Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #45894

Piping processes works with 'shell = True' but not otherwise.

Show all headers | View raw

Hi everybody, 
I am new to the group (and relatively new to Python)
so I am sorry if this issues has been discussed (although searching for topics in the group I couldn't find a solution to my problem).

I am using Python 2.7.3 to analyse the output of two 3rd parties programs that can be launched in a linux shell as:

 program1 | program2

To do this I have written a function that pipes program1 and program2 (using subprocess.Popen) and the stdout of the subprocess, and a function that parses the output:

A basic example:

from subprocess import Popen, STDOUT, PIPE
def run():
  p1 = Popen(['program1'], stdout = PIPE, stderr = STDOUT)
  p2 = Popen(['program2'], stdin = p1.stdout, stdout = PIPE, stderr = STDOUT)
  p1.stdout.close()
  return p2.stdout


def parse(out):
  for row in out:
    print row
    #do something else with each line
  out.close()
  return parsed_output
   

# main block here

pout = run()

parsed = parse(pout)

#--- END OF PROGRAM ----#

I want to parse the output of 'program1 | program2' line by line because the output is very large.

When running the code above, occasionally some error occurs (IOERROR: [Errno 0]). However this error doesn't occur if I code the run() function as:

def run():
  p = Popen('program1 | program2', shell = True, stderr = STDOUT, stdout = PIPE)
  return p.stdout

I really can't understand why the first version causes errors, while the second one doesn't.

Can you please help me understanding what's the difference between the two cases? 

Thanks a lot in advance for the help,
Cheers, Luca

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Piping processes works with 'shell = True' but not otherwise. Luca Cerone <luca.cerone@gmail.com> - 2013-05-24 07:04 -0700
  Re: Piping processes works with 'shell = True' but not otherwise. Luca Cerone <luca.cerone@gmail.com> - 2013-05-26 03:31 -0700
  Re: Piping processes works with 'shell = True' but not otherwise. Chris Rebert <clp2@rebertia.com> - 2013-05-26 14:05 -0700
    Re: Piping processes works with 'shell = True' but not otherwise. Luca Cerone <luca.cerone@gmail.com> - 2013-05-26 16:58 -0700
      RE: Piping processes works with 'shell = True' but not otherwise. Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-27 03:14 +0300
        Re: Piping processes works with 'shell = True' but not otherwise. Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2013-05-29 19:39 +0200
          RE: Piping processes works with 'shell = True' but not otherwise. Carlos Nepomuceno <carlosnepomuceno@outlook.com> - 2013-05-29 22:31 +0300
          Re: Piping processes works with 'shell = True' but not otherwise. Cameron Simpson <cs@zip.com.au> - 2013-05-30 08:18 +1000
      Re: Piping processes works with 'shell = True' but not otherwise. Chris Angelico <rosuav@gmail.com> - 2013-05-27 18:28 +1000
        Re: Piping processes works with 'shell = True' but not otherwise. Luca Cerone <luca.cerone@gmail.com> - 2013-05-27 04:33 -0700
      Re: Piping processes works with 'shell = True' but not otherwise. Chris Rebert <clp2@rebertia.com> - 2013-05-29 10:17 -0700
        Re: Piping processes works with 'shell = True' but not otherwise. Luca Cerone <luca.cerone@gmail.com> - 2013-05-31 02:28 -0700
          Re: Piping processes works with 'shell = True' but not otherwise. Peter Otten <__peter__@web.de> - 2013-05-31 11:52 +0200

csiph-web