Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #11515 > unrolled thread

subprocess.Popen question

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  subprocess.Popen question "Danny Wong (dannwong)" <dannwong@cisco.com> - 2011-08-16 02:03 -0500
    Re: subprocess.Popen question Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-08-16 11:10 +0200
    Re: subprocess.Popen question Nobody <nobody@nowhere.com> - 2011-08-16 12:26 +0100

#11515 — subprocess.Popen question

Hi All,
	I'm executing a command which I want to capture the
standard/stderr output into a file (which I have with the code below),
but I also want the standard output to go into a variable so I can
process the information for the next command. Any ideas? Thanks.

CMD_OUTPUT = subprocess.Popen(COMMAND, stdout=PROJECT_LOGFILE,
stderr=subprocess.STDOUT)

[toc] | [next] | [standalone]

#11532

Am 16.08.2011 09:03 schrieb Danny Wong (dannwong):
> Hi All,
> 	I'm executing a command which I want to capture the
> standard/stderr output into a file (which I have with the code below),
> but I also want the standard output to go into a variable so I can
> process the information for the next command. Any ideas? Thanks.
>
> CMD_OUTPUT = subprocess.Popen(COMMAND, stdout=PROJECT_LOGFILE,
> stderr=subprocess.STDOUT)

This way it won't work - but if you do

   sp = subprocess.Popen(COMMAND, stdout=subprocess.PIPE, 
stderr=subprocess.PIPE)

you can flexibly work with both of them. But then the two streams can 
become intermixed.

After that, you would have to emulate the select() behaviour of 
communicate(), but act differently on them. If you get data on stdout, 
you capture and append to the file, for data on stderr you only append.

Maybe this could work for you:

      def communicate2(self, input):
          """ input is readable, output is a generator which yields data
          """
          read_set = []
          write_set = []
          stdout = None # Return
          stderr = None # Return

          # Translate newlines, if requested.  We cannot let the file
          # object do the translation: It is based on stdio, which is
          # impossible to combine with select (unless forcing no
          # buffering).
          if self.universal_newlines and hasattr(file, 'newlines'):
              xlate=lambda s:self._translate_newlines(s)
          else:
              xlate=lambda s:s
          if self.stdin:
              # Flush stdio buffer.  This might block, if the user has
              # been writing to .stdin in an uncontrolled fashion.
              self.stdin.flush()
          if self.stdout:
              read_set.append(self.stdout)
              stdout = []
          if self.stderr:
              read_set.append(self.stderr)
              stderr = []

          inpcrsr=0 # der ist neu...
          rest=''
          eofseen=False
          while read_set or write_set:
              rlist, wlist, xlist = select.select(read_set, write_set, [])

              if self.stdin in wlist:
                  # When select has indicated that the file is writable,
                  # we can write up to PIPE_BUF bytes without risk
                  # blocking.  POSIX defines PIPE_BUF >= 512
                  #pipebuf=512
                  pipebuf=os.fpathconf(self.stdin.fileno(),'PC_PIPE_BUF')
                  if not eofseen: buf=input.read(pipebuf-len(rest)) # 
else it stays ''
                  if not buf: eofseen=True
                  if rest or buf:
                      bytes_written = os.write(self.stdin.fileno(), 
rest+buf)
                      if bytes_written > len(rest):
                              rest=buf[bytes_written-len(rest):]
                      else:
                              rest=rest[bytes_written:]+buf
                  else:
                      self.stdin.close()
                      write_set.remove(self.stdin)

              if self.stdout in rlist:
                  data = os.read(self.stdout.fileno(), 1024)
                  if data == "":
                      self.stdout.close()
                      read_set.remove(self.stdout)
                  #stdout.append(data)
                  #yield stdout,data
                  yield 1,xlate(data)

              if self.stderr in rlist:
                  data = os.read(self.stderr.fileno(), 1024)
                  if data == "":
                      self.stderr.close()
                      read_set.remove(self.stderr)
                  #stderr.append(data)
                  yield 2,xlate(data)
          self.wait()

(Im not sure if it is to 100% correct.)

This yields tuples (1, <data>) for stdout data and tuples (2, <data>) 
for stderr data.

[toc] | [prev] | [next] | [standalone]

#11542

On Tue, 16 Aug 2011 02:03:50 -0500, Danny Wong (dannwong) wrote:

> 	I'm executing a command which I want to capture the
> standard/stderr output into a file (which I have with the code below),
> but I also want the standard output to go into a variable so I can
> process the information for the next command. Any ideas? Thanks.
> 
> CMD_OUTPUT = subprocess.Popen(COMMAND, stdout=PROJECT_LOGFILE,
> stderr=subprocess.STDOUT)

	CMD_OUTPUT.wait()
	with open(PROJECT_LOGFILE,'r') as f:
	  str = f.read()

If you're going to be writing it to a file anyhow, reading that file is
likely to be faster than the alternatives.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web