Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #59233

Re: using print() with multiprocessing and pythonw

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <wrw@mac.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.012
X-Spam-Evidence '*H*': 0.98; '*S*': 0.00; 'warnings': 0.04; 'output': 0.05; 'seemed': 0.09; 'statements': 0.09; 'subject:using': 0.09; 'cc:addr:python-list': 0.11; 'appear.': 0.16; 'dump': 0.16; "function's": 0.16; 'invocation': 0.16; 'received:mac.com': 0.16; 'stdout': 0.16; 'subject:skip:m 10': 0.16; 'sys.stderr': 0.16; 'sys.stdout': 0.16; 'fix': 0.17; 'wrote:': 0.18; 'received:10.0.1': 0.19; 'cc:addr:python.org': 0.22; 'print': 0.22; 'cc:2**1': 0.23; 'replace': 0.24; 'file.': 0.24; 'cc:no real name:2**0': 0.24; 'least': 0.26; 'tried': 0.27; 'function': 0.29; 'url:mailman': 0.30; '(which': 0.31; 'program,': 0.31; 'code': 0.31; 'python).': 0.31; 'file': 0.32; 'url:python': 0.33; 'problem': 0.35; 'subject:with': 0.35; 'skip:s 30': 0.35; 'case,': 0.35; 'test': 0.35; 'but': 0.35; 'url:listinfo': 0.36; 'charset :us-ascii': 0.36; 'thanks': 0.36; 'received:10.0': 0.36; 'similar': 0.36; 'url:org': 0.36; 'unit': 0.37; 'received:10': 0.37; 'being': 0.38; 'nov': 0.38; 'received:17': 0.38; 'pm,': 0.38; '12,': 0.39; 'launch': 0.39; 'url:mail': 0.40; 'how': 0.40; 'course': 0.61; 'name': 0.63; 'different': 0.65; 'dont': 0.67; 'skip:w 30': 0.69; 'header:In-reply-to:1': 0.84; 'isaac': 0.84; '2013,': 0.91; 'hate': 0.91; 'feet': 0.93
X-Proofpoint-Virus-Version vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-11-12_08:2013-11-12,2013-11-12,1970-01-01 signatures=0
X-Proofpoint-Spam-Details rule=notspam policy=default score=0 spamscore=0 suspectscore=3 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1311120166
Content-type text/plain; charset=us-ascii
MIME-version 1.0 (Mac OS X Mail 6.6 \(1510\))
Subject Re: using print() with multiprocessing and pythonw
From William Ray Wing <wrw@mac.com>
In-reply-to <f9d2f31f-e244-42c2-848c-59acf50b6e10@googlegroups.com>
Date Tue, 12 Nov 2013 15:19:25 -0500
Content-transfer-encoding quoted-printable
References <f9d2f31f-e244-42c2-848c-59acf50b6e10@googlegroups.com>
To Isaac Gerg <isaac.gerg@gergltd.com>
X-Mailer Apple Mail (2.1510)
Cc python-list@python.org, William Ray Wing <wrw@mac.com>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.2488.1384287582.18130.python-list@python.org> (permalink)
Lines 30
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1384287582 news.xs4all.nl 15926 [2001:888:2000:d::a6]:51255
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:59233

Show key headers only | View raw


On Nov 12, 2013, at 2:12 PM, Isaac Gerg <isaac.gerg@gergltd.com> wrote:

> I launch my program with pythonw and begin it with the code below so that all my print()'s go to the log file specified. 
> 
> if sys.executable.find('pythonw') >=0:
>        # Redirect all console output to file.
>        sys.stdout = open("pythonw - stdout stderr.log",'w')
>        sys.stderr = sys.stdout
> 
> During the course of my program, I call multiprocessing.Process() and launch a function several times.  That function has print()'s inside (which are from warnings being printed by python).  This printing causes the multiprocess to crash.  How can I fix my code so that the print()'s are supressed. I would hate to do a warnings.filterwarnings('ignore') because when I unit test those functions, the warnings dont appear.
> 
> Thanks in advance,
> Isaac
> -- 
> https://mail.python.org/mailman/listinfo/python-list

This may be inelegant, but it solved a similar problem for me.  Replace the print statements with logging.info statements and have each invocation of the function dump to a unique log file (with a name based on the function's input).  At least in my case, multiprocessing seemed to get its feet tangled (crash) when different subprocesses tried to print to the same output file at the same time.

-Bill

Back to comp.lang.python | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

using print() with multiprocessing and pythonw Isaac Gerg <isaac.gerg@gergltd.com> - 2013-11-12 11:12 -0800
  Re: using print() with multiprocessing and pythonw William Ray Wing <wrw@mac.com> - 2013-11-12 15:19 -0500
    Re: using print() with multiprocessing and pythonw Isaac Gerg <isaac.gerg@gergltd.com> - 2013-11-12 12:22 -0800
      Re: using print() with multiprocessing and pythonw Chris Angelico <rosuav@gmail.com> - 2013-11-13 10:09 +1100

csiph-web