Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #34870 > unrolled thread

os.system and subprocess odd behavior

Started bypy_genetic <conor.robinson@gmail.com>
First post2012-12-14 10:13 -0800
Last post2012-12-18 12:40 +0100
Articles 20 — 7 participants

Back to article view | Back to comp.lang.python


Contents

  os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-14 10:13 -0800
    Re: os.system and subprocess odd behavior Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-12-14 22:47 +0000
      Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-17 08:39 -0800
        Re: os.system and subprocess odd behavior Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-12-17 17:16 +0000
          Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-17 12:56 -0800
            Re: os.system and subprocess odd behavior Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-12-18 00:01 +0000
              Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-18 10:26 -0800
                Re: os.system and subprocess odd behavior Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-12-18 18:52 +0000
                  Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-18 12:52 -0800
                  Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-18 12:52 -0800
              Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-18 10:26 -0800
            Re: os.system and subprocess odd behavior Hans Mulder <hansmu@xs4all.nl> - 2012-12-18 23:46 +0100
          Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-17 12:56 -0800
    Re: os.system and subprocess odd behavior Dieter Maurer <dieter@handshake.de> - 2012-12-15 08:12 +0100
      Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-17 08:30 -0800
      Re: os.system and subprocess odd behavior py_genetic <conor.robinson@gmail.com> - 2012-12-17 08:30 -0800
    Re: os.system and subprocess odd behavior photonymous@gmail.com - 2012-12-17 21:10 -0800
      Re: os.system and subprocess odd behavior Hans Mulder <hansmu@xs4all.nl> - 2012-12-18 11:27 +0100
        Re: os.system and subprocess odd behavior Dave Angel <d@davea.name> - 2012-12-18 05:39 -0500
          Re: os.system and subprocess odd behavior Hans Mulder <hansmu@xs4all.nl> - 2012-12-18 12:40 +0100

#34870 — os.system and subprocess odd behavior

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-14 10:13 -0800
Subjectos.system and subprocess odd behavior
Message-ID<bb08fc10-85f0-4003-a1fb-cdc449299d19@googlegroups.com>
Example of the issue for arguments sake:

Platform Ubuntu server 12.04LTS, python 2.7

Say file1.txt has "hello world" in it.

subprocess.Popen("cat < file1 > file2", shell = True)
subprocess.call("cat < file1 > file2", shell = True)
os.system("cat < file1 > file2")


I'm finding that file2 IS created, but with 0bytes in it, this happens when I try any sort of cmd to the system of the nature where I'm putting the output into a file.

I've made sure it isn't a permission issue.  The command runs fine from the cmd line and python is being run with super user privileges. Strait from the terminal I get a hello world copy as file2... as expected.

I would like python to simply exec the cmd and move on.... I don't want to read and write the stdout ect into python and write it to a file.  Any thoughts as to why this creates file2, but no data appears?  Is there a better way to do this?

Thank you!

[toc] | [next] | [standalone]


#34876

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2012-12-14 22:47 +0000
Message-ID<50cbac6d$0$29991$c3e8da3$5496439d@news.astraweb.com>
In reply to#34870
On Fri, 14 Dec 2012 10:13:38 -0800, py_genetic wrote:

> Example of the issue for arguments sake:
> 
> Platform Ubuntu server 12.04LTS, python 2.7
> 
> Say file1.txt has "hello world" in it.
> 
> subprocess.Popen("cat < file1 > file2", shell = True)
> subprocess.call("cat < file1 > file2", shell = True) 
> os.system("cat < file1 > file2")
> 
> 
> I'm finding that file2 IS created, but with 0bytes in it, this happens
> when I try any sort of cmd to the system of the nature where I'm putting
> the output into a file.

I cannot confirm this behaviour. It works for me. When I try it, all 
three code snippets work as expected:

[steve@ando ~]$ rm file2 file3 file4
[steve@ando ~]$ cat file1
hello world
[steve@ando ~]$ python
Python 2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> import os, subprocess
py> subprocess.Popen("cat < file1 > file2", shell = True)
<subprocess.Popen object at 0xb7e8a62c>
py> subprocess.call("cat < file1 > file3", shell = True)
0
py> os.system("cat < file1 > file4")
0
py> quit()
[steve@ando ~]$ cat file2
hello world
[steve@ando ~]$ cat file3
hello world
[steve@ando ~]$ cat file4
hello world


I have run this multiple times, as an unprivileged user, as the root 
user, and as sudo. It works perfectly every time.

Please check your code. Perhaps you have over-simplified the problem.



-- 
Steven

[toc] | [prev] | [next] | [standalone]


#34991

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-17 08:39 -0800
Message-ID<8757bcac-76ac-4b15-9410-dc61d7a8c641@googlegroups.com>
In reply to#34876
Thanks for verifying this for me Steven.  I'm glad you are seeing it work.  It's really the strangest thing.

The issue seems to be with the " > outfile.txt" portion of the command.

The actual command is running a query on a verticalDB and dumping the result.  The EXACT command run from the command line runs just fine.  

Even if I use the simple cat command to and out file as just a simple test case...  The file is created with zero bytes (see below)... but its as if python moves on or gets an 0 exit code after the first part of the cmd is executed  and no data is written.

-rw-r--r-- 1 root root        0 Dec 14 15:33 QUAD_12142012203251.TXT

Any thoughts as to why on my end this may happen?

Thanks again!


[toc] | [prev] | [next] | [standalone]


#34992

FromOscar Benjamin <oscar.j.benjamin@gmail.com>
Date2012-12-17 17:16 +0000
Message-ID<mailman.970.1355764606.29569.python-list@python.org>
In reply to#34991
On 17 December 2012 16:39, py_genetic <conor.robinson@gmail.com> wrote:
> Thanks for verifying this for me Steven.  I'm glad you are seeing it work.  It's really the strangest thing.
>
> The issue seems to be with the " > outfile.txt" portion of the command.
>
> The actual command is running a query on a verticalDB and dumping the result.  The EXACT command run from the command line runs just fine.
>
> Even if I use the simple cat command to and out file as just a simple test case...  The file is created with zero bytes (see below)... but its as if python moves on or gets an 0 exit code after the first part of the cmd is executed  and no data is written.
>
> -rw-r--r-- 1 root root        0 Dec 14 15:33 QUAD_12142012203251.TXT
>
> Any thoughts as to why on my end this may happen?

Because of the root permissions on the file? What happens if you write
to a file that doesn't need privileged access?

Instead of running the "exact command", run the cat commands you
posted (that Steven has confirmed as working) and run them somewhere
in your user directory without root permissions.

Also you may want to use subprocess.check_call as this raises a Python
error if the command returns an error code.


Oscar

[toc] | [prev] | [next] | [standalone]


#35012

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-17 12:56 -0800
Message-ID<0e224780-b84f-4f72-9c95-bc55cde8b183@googlegroups.com>
In reply to#34992
Oscar, seems you may be correct.  I need to run this program as a superuser.  However, after some more tests with simple commands...  I seem to be working correctly from any permission level in python.... Except for the output write command from the database to a file.  Which runs fine if I paste it into the cmd line.  Also, subprocess.call_check() returns clean.  However, nothing is written to the output file when called from python.

so this cmd runs great from the cmd line (sudo or no) however the output file in this case is owned by the sysadmin either way... not root?

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT


When run from sudo python (other files are also created and owned by root correctly)  however no output is written from the db command zero byte file only (owned by root)... returns to python with no errors.

I'm sorta at a loss.  I'd rather still avoid having python connect to the db directly or reading the data from stdout, is a waste or mem and time for what I need.

Thanks for any more thoughts.

> 
> Because of the root permissions on the file? What happens if you write
> 
> to a file that doesn't need privileged access?
> 
> 
> 
> Instead of running the "exact command", run the cat commands you
> 
> posted (that Steven has confirmed as working) and run them somewhere
> 
> in your user directory without root permissions.
> 
> 
> 
> Also you may want to use subprocess.check_call as this raises a Python
> 
> error if the command returns an error code.
> 
> 
> 
> 
> 
> Oscar

[toc] | [prev] | [next] | [standalone]


#35031

FromOscar Benjamin <oscar.j.benjamin@gmail.com>
Date2012-12-18 00:01 +0000
Message-ID<mailman.1004.1355790699.29569.python-list@python.org>
In reply to#35012
On 17 December 2012 20:56, py_genetic <conor.robinson@gmail.com> wrote:
> Oscar, seems you may be correct.  I need to run this program as a superuser.  However, after some more tests with simple commands...  I seem to be working correctly from any permission level in python.... Except for the output write command from the database to a file.  Which runs fine if I paste it into the cmd line.  Also, subprocess.call_check() returns clean.  However, nothing is written to the output file when called from python.
>
> so this cmd runs great from the cmd line (sudo or no) however the output file in this case is owned by the sysadmin either way... not root?
>
> /usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT
>
>
> When run from sudo python (other files are also created and owned by root correctly)  however no output is written from the db command zero byte file only (owned by root)... returns to python with no errors.
>
> I'm sorta at a loss.  I'd rather still avoid having python connect to the db directly or reading the data from stdout, is a waste or mem and time for what I need.

Follow through the bash session below

$ cd /usr
$ ls
bin  games  include  lib  local  sbin  share  src
$ touch file
touch: cannot touch `file': Permission denied
$ sudo touch file
[sudo] password for oscar:
$ ls
bin  file  games  include  lib  local  sbin  share  src
$ cat < file > file2
bash: file2: Permission denied
$ sudo cat < file > file2
bash: file2: Permission denied
$ sudo cat < file > file2
bash: file2: Permission denied
$ sudo cat < file | tee file2
tee: file2: Permission denied
$ sudo cat < file | sudo tee file2
$ ls
bin  file  file2  games  include  lib  local  sbin  share  src

The problem is that when you do

  $ sudo cmd > file2

it is sort of like doing

  $ sudo cmd | this_bash_session > file2

so the permissions used to write to file2 are the same as the bash
session rather than the command cmd which has root permissions. By
piping my output into "sudo tee file2" I can get file2 to be written
by a process that has root permissions.

I suspect you have the same problem although it all complicated by the
fact that everything is a subprocess of Python. Is it possibly the
case that the main Python process does not have root permissions but
you are using it to run a command with sudo that then does have root
permissions?

Does piping through something like "sudo tee" help?


Oscar

[toc] | [prev] | [next] | [standalone]


#35064

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-18 10:26 -0800
Message-ID<6ff54b92-70f1-44fb-a5a5-17133e1a6faa@googlegroups.com>
In reply to#35031
Oscar I can confirm this behavior from terminal. 

AND this works as well, simulating exactly what I'm doing permissions wise, and calling sudo python test.py  below

f1 = open('TESTDIR/file1.txt', 'w')
f1.write('some test here\n')
f1.close()

cmd1 = 'cat < TESTDIR/file1.txt > TESTDIR/file2.txt'
P = Popen(cmd1, shell=True)
P.wait()

cmd2 = 'cat < TESTDIR/file1.txt | sudo tee TESTDIR/file3.txt'
P = Popen(cmd2, shell=True)
P.wait()

-rw-r--r-- 1 root root       15 Dec 18 12:57 file1.txt
-rw-r--r-- 1 root root       15 Dec 18 12:57 file2.txt
-rw-r--r-- 1 root root       15 Dec 18 12:57 file3.txt

HOWEVER... 

when using this command from before.... no dice

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT

OR

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL | sudo tee /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT

So it's basically as if python gets a response instantly (perhaps from the query) and closes the process, since we've verified its not permissions related.

Perhaps someone can try a mysql cmd line such as above within python?  And see if you can verify this behavior.  I believe the query returning with no errors is shutting the sub shell/process?

I've tried this with all options p.wait() ect as well as parsing the command and running shell false.

Again the exact command run perfect when pasted and run from the shell.  I'll try running it a few other ways with some diff db options.


> Follow through the bash session below
> 
> 
> 
> $ cd /usr
> 
> $ ls
> 
> bin  games  include  lib  local  sbin  share  src
> 
> $ touch file
> 
> touch: cannot touch `file': Permission denied
> 
> $ sudo touch file
> 
> [sudo] password for oscar:
> 
> $ ls
> 
> bin  file  games  include  lib  local  sbin  share  src
> 
> $ cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file | tee file2
> 
> tee: file2: Permission denied
> 
> $ sudo cat < file | sudo tee file2
> 
> $ ls
> 
> bin  file  file2  games  include  lib  local  sbin  share  src
> 
> 
> 
> The problem is that when you do
> 
> 
> 
>   $ sudo cmd > file2
> 
> 
> 
> it is sort of like doing
> 
> 
> 
>   $ sudo cmd | this_bash_session > file2
> 
> 
> 
> so the permissions used to write to file2 are the same as the bash
> 
> session rather than the command cmd which has root permissions. By
> 
> piping my output into "sudo tee file2" I can get file2 to be written
> 
> by a process that has root permissions.
> 
> 
> 
> I suspect you have the same problem although it all complicated by the
> 
> fact that everything is a subprocess of Python. Is it possibly the
> 
> case that the main Python process does not have root permissions but
> 
> you are using it to run a command with sudo that then does have root
> 
> permissions?
> 
> 
> 
> Does piping through something like "sudo tee" help?
> 
> 
> 
> 
> 
> Oscar

[toc] | [prev] | [next] | [standalone]


#35065

FromOscar Benjamin <oscar.j.benjamin@gmail.com>
Date2012-12-18 18:52 +0000
Message-ID<mailman.1024.1355856768.29569.python-list@python.org>
In reply to#35064
Can you trim content and interleave your response (instead of
top-posting) please?

On 18 December 2012 18:26, py_genetic <conor.robinson@gmail.com> wrote:
> HOWEVER...
>
> when using this command from before.... no dice
>
> /usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT
>
> OR
>
> /usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL | sudo tee /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT
>
> So it's basically as if python gets a response instantly (perhaps from the query) and closes the process, since we've verified its not permissions related.

I wouldn't say that this is verified.

Have you verified that without the last redirection (to the output
file) you can correctly read the stdout from within Python?

Does the same command work if you put it in a shell script? If so what
about making a script like so:

#!/usr/bin/env bash

jobname="$1"

/usr/local/Calpont/mysql/bin/mysql
--defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB <
"/home/myusr/jobs/APP_JOBS/JOB_${jobname}.SQL" | sudo tee
"/home/myusr/jobs/APP_JOBS/JOB_${jobname}.TXT"

If that works when you run it directly, does it work if you do:
subprocess.check_call(['myscript.sh', jobname])


Oscar

[toc] | [prev] | [next] | [standalone]


#35070

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-18 12:52 -0800
Message-ID<a9a56cbe-9686-4202-95f3-05aa04f77ce3@googlegroups.com>
In reply to#35065
Solved the issue, by injecting the query into the cmd line.  Shell script worked fine as if I was cutting and pasting to the prompt.  Seems to still be something with the subprocess receiving and exit code before or when the query finishes, just when I ask to to read from the .SQL file.

example called from in python:
mysql .... < file.txt > out.txt  <---- doesn't work (query is run 0Byte output)
mysql .... -e "my query" > out.txt <----- does work

However this isn't standard mysql as it's infinidb.  Maybe this is an esoteric issue.

Thanks for the help Oscar.  Frustrating since it seems illogical.... seems if the cmd runs in the shell it should have the exact same behavior from a subprocess shell=True cmd string call.

If I find anything else I'll update this.

[toc] | [prev] | [next] | [standalone]


#35074

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-18 12:52 -0800
Message-ID<mailman.1030.1355866669.29569.python-list@python.org>
In reply to#35065
Solved the issue, by injecting the query into the cmd line.  Shell script worked fine as if I was cutting and pasting to the prompt.  Seems to still be something with the subprocess receiving and exit code before or when the query finishes, just when I ask to to read from the .SQL file.

example called from in python:
mysql .... < file.txt > out.txt  <---- doesn't work (query is run 0Byte output)
mysql .... -e "my query" > out.txt <----- does work

However this isn't standard mysql as it's infinidb.  Maybe this is an esoteric issue.

Thanks for the help Oscar.  Frustrating since it seems illogical.... seems if the cmd runs in the shell it should have the exact same behavior from a subprocess shell=True cmd string call.

If I find anything else I'll update this.

[toc] | [prev] | [next] | [standalone]


#35067

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-18 10:26 -0800
Message-ID<mailman.1026.1355861274.29569.python-list@python.org>
In reply to#35031
Oscar I can confirm this behavior from terminal. 

AND this works as well, simulating exactly what I'm doing permissions wise, and calling sudo python test.py  below

f1 = open('TESTDIR/file1.txt', 'w')
f1.write('some test here\n')
f1.close()

cmd1 = 'cat < TESTDIR/file1.txt > TESTDIR/file2.txt'
P = Popen(cmd1, shell=True)
P.wait()

cmd2 = 'cat < TESTDIR/file1.txt | sudo tee TESTDIR/file3.txt'
P = Popen(cmd2, shell=True)
P.wait()

-rw-r--r-- 1 root root       15 Dec 18 12:57 file1.txt
-rw-r--r-- 1 root root       15 Dec 18 12:57 file2.txt
-rw-r--r-- 1 root root       15 Dec 18 12:57 file3.txt

HOWEVER... 

when using this command from before.... no dice

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT

OR

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL | sudo tee /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT

So it's basically as if python gets a response instantly (perhaps from the query) and closes the process, since we've verified its not permissions related.

Perhaps someone can try a mysql cmd line such as above within python?  And see if you can verify this behavior.  I believe the query returning with no errors is shutting the sub shell/process?

I've tried this with all options p.wait() ect as well as parsing the command and running shell false.

Again the exact command run perfect when pasted and run from the shell.  I'll try running it a few other ways with some diff db options.


> Follow through the bash session below
> 
> 
> 
> $ cd /usr
> 
> $ ls
> 
> bin  games  include  lib  local  sbin  share  src
> 
> $ touch file
> 
> touch: cannot touch `file': Permission denied
> 
> $ sudo touch file
> 
> [sudo] password for oscar:
> 
> $ ls
> 
> bin  file  games  include  lib  local  sbin  share  src
> 
> $ cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file > file2
> 
> bash: file2: Permission denied
> 
> $ sudo cat < file | tee file2
> 
> tee: file2: Permission denied
> 
> $ sudo cat < file | sudo tee file2
> 
> $ ls
> 
> bin  file  file2  games  include  lib  local  sbin  share  src
> 
> 
> 
> The problem is that when you do
> 
> 
> 
>   $ sudo cmd > file2
> 
> 
> 
> it is sort of like doing
> 
> 
> 
>   $ sudo cmd | this_bash_session > file2
> 
> 
> 
> so the permissions used to write to file2 are the same as the bash
> 
> session rather than the command cmd which has root permissions. By
> 
> piping my output into "sudo tee file2" I can get file2 to be written
> 
> by a process that has root permissions.
> 
> 
> 
> I suspect you have the same problem although it all complicated by the
> 
> fact that everything is a subprocess of Python. Is it possibly the
> 
> case that the main Python process does not have root permissions but
> 
> you are using it to run a command with sudo that then does have root
> 
> permissions?
> 
> 
> 
> Does piping through something like "sudo tee" help?
> 
> 
> 
> 
> 
> Oscar

[toc] | [prev] | [next] | [standalone]


#35078

FromHans Mulder <hansmu@xs4all.nl>
Date2012-12-18 23:46 +0100
Message-ID<50d0f233$0$6983$e4fe514c@news2.news.xs4all.nl>
In reply to#35012
On 17/12/12 21:56:50, py_genetic wrote:
> /usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT

If you're trying to interact with a MySQL database, then
you should really use the myslqdb module.  Trying to parse
the output of the command-line utility is harder, and is
too fragile for production use.


Hope this helps,

-- HansM

[toc] | [prev] | [next] | [standalone]


#35013

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-17 12:56 -0800
Message-ID<mailman.990.1355777820.29569.python-list@python.org>
In reply to#34992
Oscar, seems you may be correct.  I need to run this program as a superuser.  However, after some more tests with simple commands...  I seem to be working correctly from any permission level in python.... Except for the output write command from the database to a file.  Which runs fine if I paste it into the cmd line.  Also, subprocess.call_check() returns clean.  However, nothing is written to the output file when called from python.

so this cmd runs great from the cmd line (sudo or no) however the output file in this case is owned by the sysadmin either way... not root?

/usr/local/Calpont/mysql/bin/mysql --defaults-file=/usr/local/Calpont/mysql/my.cnf -u root myDB < /home/myusr/jobs/APP_JOBS/JOB_XXX.SQL > /home/myusr/jobs/APP_JOBS/JOB_XXX.TXT


When run from sudo python (other files are also created and owned by root correctly)  however no output is written from the db command zero byte file only (owned by root)... returns to python with no errors.

I'm sorta at a loss.  I'd rather still avoid having python connect to the db directly or reading the data from stdout, is a waste or mem and time for what I need.

Thanks for any more thoughts.

> 
> Because of the root permissions on the file? What happens if you write
> 
> to a file that doesn't need privileged access?
> 
> 
> 
> Instead of running the "exact command", run the cat commands you
> 
> posted (that Steven has confirmed as working) and run them somewhere
> 
> in your user directory without root permissions.
> 
> 
> 
> Also you may want to use subprocess.check_call as this raises a Python
> 
> error if the command returns an error code.
> 
> 
> 
> 
> 
> Oscar

[toc] | [prev] | [next] | [standalone]


#34884

FromDieter Maurer <dieter@handshake.de>
Date2012-12-15 08:12 +0100
Message-ID<mailman.901.1355555550.29569.python-list@python.org>
In reply to#34870
py_genetic <conor.robinson@gmail.com> writes:

> Example of the issue for arguments sake:
>
> Platform Ubuntu server 12.04LTS, python 2.7
>
> Say file1.txt has "hello world" in it.
      ^^^^^^^^^
Here, you speak of "file1.txt" (note the extension ".txt")

> subprocess.Popen("cat < file1 > file2", shell = True)
> subprocess.call("cat < file1 > file2", shell = True)
> os.system("cat < file1 > file2")

But in your code, you use "file1" (without extension).

If your code really references a non-existing file, you may well
get what you are observing.

[toc] | [prev] | [next] | [standalone]


#34989

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-17 08:30 -0800
Message-ID<098ece3c-5cf0-4cd9-a2d1-eb5020017597@googlegroups.com>
In reply to#34884
Thanks! I am using .txt extensions.  Sorry for being a little vague.

[toc] | [prev] | [next] | [standalone]


#34990

Frompy_genetic <conor.robinson@gmail.com>
Date2012-12-17 08:30 -0800
Message-ID<mailman.969.1355761816.29569.python-list@python.org>
In reply to#34884
Thanks! I am using .txt extensions.  Sorry for being a little vague.

[toc] | [prev] | [next] | [standalone]


#35036

Fromphotonymous@gmail.com
Date2012-12-17 21:10 -0800
Message-ID<0396f870-ab0b-4f74-a94c-8922b49ef2e9@googlegroups.com>
In reply to#34870
I hope I understand the question... but shouldn't you wait for the process to complete before exiting?

Something like:

pid = subprocess.Popen(...)
pid.wait()

Otherwise, it'll exit before the background process is done. 

[toc] | [prev] | [next] | [standalone]


#35045

FromHans Mulder <hansmu@xs4all.nl>
Date2012-12-18 11:27 +0100
Message-ID<50d04526$0$6926$e4fe514c@news2.news.xs4all.nl>
In reply to#35036
On 18/12/12 06:10:43, photonymous@gmail.com wrote:
> I hope I understand the question... but shouldn't you wait for the process to complete before exiting?
> 
> Something like:
> 
> pid = subprocess.Popen(...)
> pid.wait()
> 
> Otherwise, it'll exit before the background process is done. 

Why would that be a problem?

-- HansM

[toc] | [prev] | [next] | [standalone]


#35047

FromDave Angel <d@davea.name>
Date2012-12-18 05:39 -0500
Message-ID<mailman.1013.1355827216.29569.python-list@python.org>
In reply to#35045
On 12/18/2012 05:27 AM, Hans Mulder wrote:
> On 18/12/12 06:10:43, photonymous@gmail.com wrote:
>> I hope I understand the question... but shouldn't you wait for the process to complete before exiting?
>>
>> Something like:
>>
>> pid = subprocess.Popen(...)
>> pid.wait()
>>
>> Otherwise, it'll exit before the background process is done. 
> Why would that be a problem?
>

Because you don't want to bog the system down with a zombie task.



-- 

DaveA

[toc] | [prev] | [next] | [standalone]


#35050

FromHans Mulder <hansmu@xs4all.nl>
Date2012-12-18 12:40 +0100
Message-ID<50d0563d$0$6984$e4fe514c@news2.news.xs4all.nl>
In reply to#35047
On 18/12/12 11:39:56, Dave Angel wrote:
> On 12/18/2012 05:27 AM, Hans Mulder wrote:
>> On 18/12/12 06:10:43, photonymous@gmail.com wrote:
>>> I hope I understand the question... but shouldn't you wait for the process to complete before exiting?
>>>
>>> Something like:
>>>
>>> pid = subprocess.Popen(...)
>>> pid.wait()
>>>
>>> Otherwise, it'll exit before the background process is done. 
>> Why would that be a problem?
>>
> 
> Because you don't want to bog the system down with a zombie task.

I think you're confusing zombies with orphans.  A zombie results when
a child exits before the parent and the parent fails to wait for it.
An orphan is what you get if the parent exits before the child does.

On a Un*x system, the 'init' process (the process with ID 1) inherits
orphan processes and collects their exit status when they terminate.
Thus orphans do not become zombies.

I have no idea what would happen under Windows.


Hope this helps,

-- HansM

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.python


csiph-web