Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #106340 > unrolled thread

Failed to update the os.environ with subprocess.Popen.

Back to article view | Back to comp.lang.python

Contents

  Failed to update the os.environ with subprocess.Popen. Hongyi Zhao <hongyi.zhao@gmail.com> - 2016-04-03 03:29 +0000
    Re: Failed to update the os.environ with subprocess.Popen. Steven D'Aprano <steve@pearwood.info> - 2016-04-03 14:24 +1000
      Re: Failed to update the os.environ with subprocess.Popen. Cameron Simpson <cs@zip.com.au> - 2016-04-03 18:20 +1000
        Re: Failed to update the os.environ with subprocess.Popen. Hongyi Zhao <hongyi.zhao@gmail.com> - 2016-04-03 11:24 +0000
          Re: Failed to update the os.environ with subprocess.Popen. Cameron Simpson <cs@zip.com.au> - 2016-04-04 07:49 +1000

#106340 — Failed to update the os.environ with subprocess.Popen.

Hi all,

I use the following code to update the os.environ with subprocess.Popen:

-------------
from subprocess import Popen

output = Popen("""
/bin/bash <<EOF
source ~/.profile.d/modules/modules.sh
export PATH=$MODULESHOME/../default/bin:$PATH
module add intel/parallel_studio_xe_2015
env -0
EOF
""", shell=True)

if "" in os.environ.data: del os.environ.data[""]
os.environ.clear()
os.environ.update(line.partition('=')[::2] for line in output.split('\0'))
-------------

But, I meet the following errors:

------------
Traceback (most recent call last):
  File "/home/werner/software/hpc/dft-to-study/jasp/jasp.git/jasp/bin/
runjasp.py", line 125, in <module>
    os.environ.update(line.partition('=')[::2] for line in output.split
('\0'))
AttributeError: 'Popen' object has no attribute 'split'
-------------------

How to solve this issue?
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

[toc] | [next] | [standalone]

#106343

On Sun, 3 Apr 2016 01:29 pm, Hongyi Zhao wrote:

> Hi all,
> 
> I use the following code to update the os.environ with subprocess.Popen:


I don't understand what you are trying to do here. But regardless of your
intention, the problem you have is nothing to do with updating os.environ.
Proof: change the last line from this complicated expression:

os.environ.update(line.partition('=')[::2] for line in output.split('\0'))

to this simple expression:

output.split('\0')

and you will get the same error. os.environ is irrelevant. Your problem is
only with Popen.

What makes you think that Popen objects have a split() method? They are not
documented as having this method:

https://docs.python.org/2/library/subprocess.html

https://docs.python.org/3/library/subprocess.html

https://pymotw.com/2/subprocess/




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#106356

On 03Apr2016 14:24, Steven D'Aprano <steve@pearwood.info> wrote:
>On Sun, 3 Apr 2016 01:29 pm, Hongyi Zhao wrote:
>> I use the following code to update the os.environ with subprocess.Popen:
>
>I don't understand what you are trying to do here. But regardless of your
>intention, the problem you have is nothing to do with updating os.environ.
>Proof: change the last line from this complicated expression:
>
>os.environ.update(line.partition('=')[::2] for line in output.split('\0'))
>to this simple expression:
>output.split('\0')
>
>and you will get the same error. os.environ is irrelevant. Your problem is
>only with Popen.
>
>What makes you think that Popen objects have a split() method? They are not
>documented as having this method:
>
>https://docs.python.org/2/library/subprocess.html
>https://docs.python.org/3/library/subprocess.html
>https://pymotw.com/2/subprocess/

In particular, you want the subprocess' output. As written, your code sets 
"output" to the Popen object. You actually want to set it to the .stdout 
attribute of that object, which is the output from the subcommand.

Cheers,
Cameron Simpson <cs@zip.com.au>

[toc] | [prev] | [next] | [standalone]

#106357

On Sun, 03 Apr 2016 18:20:31 +1000, Cameron Simpson wrote:

> In particular, you want the subprocess' output. As written, your code
> sets "output" to the Popen object. You actually want to set it to the
> .stdout attribute of that object, which is the output from the
> subcommand.

Based on your above hints, I try to use the following codes:

output = Popen("""
/bin/bash <<EOF
source ~/.profile.d/environment-module-systems/modules.sh
export PATH=$MODULESHOME/../default/bin:$PATH
module add intel/parallel_studio_xe_2015
env -0
EOF
""", shell=True).stdout

But I meet the following error:

Traceback (most recent call last):
  File "/home/werner/software/hpc/dft-to-study/jasp/jasp.git/jasp/bin/
runjasp.py", line 138, in <module>
    os.environ.update(line.partition('=')[::2] for line in output.split
('\0'))
AttributeError: 'NoneType' object has no attribute 'split'


While the following code will work smoothly:

output = Popen("""
/bin/bash <<EOF
source ~/.profile.d/modules/modules.sh
export PATH=$MODULESHOME/../default/bin:$PATH
module add intel/parallel_studio_xe_2015
env -0
EOF
""", shell=True, stdout=PIPE).communicate()[0]

Any hints on this issue?

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

[toc] | [prev] | [next] | [standalone]

#106415

On 03Apr2016 11:24, Hongyi Zhao <hongyi.zhao@gmail.com> wrote:
>On Sun, 03 Apr 2016 18:20:31 +1000, Cameron Simpson wrote:
>
>> In particular, you want the subprocess' output. As written, your code
>> sets "output" to the Popen object. You actually want to set it to the
>> .stdout attribute of that object, which is the output from the
>> subcommand.
>
>Based on your above hints, I try to use the following codes:
>
>output = Popen("""
>/bin/bash <<EOF
>source ~/.profile.d/environment-module-systems/modules.sh
>export PATH=$MODULESHOME/../default/bin:$PATH
>module add intel/parallel_studio_xe_2015
>env -0
>EOF
>""", shell=True).stdout
>
>But I meet the following error:
>
>Traceback (most recent call last):
>  File "/home/werner/software/hpc/dft-to-study/jasp/jasp.git/jasp/bin/
>runjasp.py", line 138, in <module>
>    os.environ.update(line.partition('=')[::2] for line in output.split
>('\0'))
>AttributeError: 'NoneType' object has no attribute 'split'

Please have a close read of the subprocess docs as Steven recommended.

You're calling:

  output.split('\0')

The message above indicates that "output" is None. This is probably because you 
have not asked to attach to the command's output in your Popen() call, and 
therefore stdout is not attached to a pipe, so it is None.

Also note that even when it is attached to a pipe, .stdout is a _file_. So you 
need to read from it to get a string to split.

>While the following code will work smoothly:
>
>output = Popen("""
>/bin/bash <<EOF
>source ~/.profile.d/modules/modules.sh
>export PATH=$MODULESHOME/../default/bin:$PATH
>module add intel/parallel_studio_xe_2015
>env -0
>EOF
>""", shell=True, stdout=PIPE).communicate()[0]
>
>Any hints on this issue?

This is because (a) you specified stdout=PIPE here, which you did not do 
earlier and (b) because .communicate reads the output for you and gives you a 
string.

Cheers,
Cameron Simpson <cs@zip.com.au>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web