Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #59228 > unrolled thread

using print() with multiprocessing and pythonw

Back to article view | Back to comp.lang.python

Contents

  using print() with multiprocessing and pythonw Isaac Gerg <isaac.gerg@gergltd.com> - 2013-11-12 11:12 -0800
    Re: using print() with multiprocessing and pythonw William Ray Wing <wrw@mac.com> - 2013-11-12 15:19 -0500
      Re: using print() with multiprocessing and pythonw Isaac Gerg <isaac.gerg@gergltd.com> - 2013-11-12 12:22 -0800
        Re: using print() with multiprocessing and pythonw Chris Angelico <rosuav@gmail.com> - 2013-11-13 10:09 +1100

#59228 — using print() with multiprocessing and pythonw

I launch my program with pythonw and begin it with the code below so that all my print()'s go to the log file specified. 

if sys.executable.find('pythonw') >=0:
        # Redirect all console output to file.
        sys.stdout = open("pythonw - stdout stderr.log",'w')
        sys.stderr = sys.stdout

During the course of my program, I call multiprocessing.Process() and launch a function several times.  That function has print()'s inside (which are from warnings being printed by python).  This printing causes the multiprocess to crash.  How can I fix my code so that the print()'s are supressed. I would hate to do a warnings.filterwarnings('ignore') because when I unit test those functions, the warnings dont appear.

Thanks in advance,
Isaac

[toc] | [next] | [standalone]

#59233

On Nov 12, 2013, at 2:12 PM, Isaac Gerg <isaac.gerg@gergltd.com> wrote:

> I launch my program with pythonw and begin it with the code below so that all my print()'s go to the log file specified. 
> 
> if sys.executable.find('pythonw') >=0:
>        # Redirect all console output to file.
>        sys.stdout = open("pythonw - stdout stderr.log",'w')
>        sys.stderr = sys.stdout
> 
> During the course of my program, I call multiprocessing.Process() and launch a function several times.  That function has print()'s inside (which are from warnings being printed by python).  This printing causes the multiprocess to crash.  How can I fix my code so that the print()'s are supressed. I would hate to do a warnings.filterwarnings('ignore') because when I unit test those functions, the warnings dont appear.
> 
> Thanks in advance,
> Isaac
> -- 
> https://mail.python.org/mailman/listinfo/python-list

This may be inelegant, but it solved a similar problem for me.  Replace the print statements with logging.info statements and have each invocation of the function dump to a unique log file (with a name based on the function's input).  At least in my case, multiprocessing seemed to get its feet tangled (crash) when different subprocesses tried to print to the same output file at the same time.

-Bill

[toc] | [prev] | [next] | [standalone]

#59234

Thanks for the reply Bill.  The problem is the text i am getting is from a python warning message, not one of my own print() function calls.

[toc] | [prev] | [next] | [standalone]

#59256

On Wed, Nov 13, 2013 at 7:22 AM, Isaac Gerg <isaac.gerg@gergltd.com> wrote:
> Thanks for the reply Bill.  The problem is the text i am getting is from a python warning message, not one of my own print() function calls.

Since sys.stdout is just an object, you could replace it with
something that redirects its write() calls to the logging function.
Might be a bit hacky, but it should work.

BTW, "sys.executable.find('pythonw') >=0" should be able to be spelled
"'pythonw' in sys.executable".

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web