Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90579

use of subprocess module inside generator

Show key headers only | View raw

I'm using Python 3.4.3 on Windows 7 (with latest patches) to develop a 
sqlcmd module for accessing SQL Server (via Microsoft's sqlcmd.exe).  My 
goal is to develop a 100% Python 3 module that's easy to use, flexible, 
and by design shifts the majority of future SQL Server Python database 
access maintenance to Microsoft.  So far, I've successfully and quickly 
converted a few of my Python pyodbc applications/tools to use this new 
module.  However, I recently ran into an apparent Python issue which 
cost me a few hours to diagnose and work around.

I'm hoping that someone might know what the root cause of my issue was. 
  Perhaps I've hit a bug/restriction with Python generators?

My original generator function looked like this:

def _raw_data(cl, stdout, *, opath=None, timeout=timeout):
     stdout = subprocess.check_output(cl, universal_newlines=True, 
timeout=timeout)

     if opath is None:
         for line in stdout.splitlines():
             yield line.strip()
     else:
         with open(opath) as f:
             for line in f:
                 yield line.strip()

The above function appeared to work fine, if the command line directed 
sqlcmd.exe to send its output to stdout.  However, if the command line 
directed sqlcmd.exe to send its output to a file, 
subprocess.check_output would never be called when next was called on 
the returned generator.  I verified this behavior with print statements 
inside my code, as well as, inside the subprocess module.

My work around was to simply move the call to subprocess.check_output 
outside of the generator function (see below) to its caller (a 
non-generator function).  With this minor change, everything appears to 
work as expected.  OK, so am I missing something here?

def _raw_data(stdout, *, opath=None):
     if opath is None:
         for line in stdout.splitlines():
             yield line.strip()
     else:
         with open(opath) as f:
             for line in f:
                 yield line.strip()

Thank you in advance for your assistance.

Peter Santoro

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

use of subprocess module inside generator Peter <p-santoro@sbcglobal.net> - 2015-05-13 15:43 -0400

csiph-web