Logging from a multiprocess application

Path	csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<jcasale@activenetwerx.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.003
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'handler': 0.05; 'problem?': 0.07; 'remaining': 0.07; 'socket': 0.07; 'subject:application': 0.07; 'performs': 0.09; 'worse': 0.09; 'thread': 0.14; 'handling,': 0.16; 'received:172.18.0': 0.16; 'subject:Logging': 0.16; 'task.': 0.16; 'thanks,': 0.17; 'obviously': 0.18; 'module': 0.19; 'portion': 0.22; 'to:name :python-list@python.org': 0.22; 'switched': 0.24; 'header:Received:8': 0.24; 'code': 0.31; 'subject:from': 0.34; 'instances': 0.36; 'charset:us-ascii': 0.36; 'application': 0.37; 'being': 0.38; 'server': 0.38; 'filter': 0.38; 'to:addr:python- list': 0.38; 'to:addr:python.org': 0.39; 'how': 0.40; 'received:unknown': 0.61; 'simply': 0.61; 'such': 0.63; 'special': 0.74; 'console,': 0.84
X-Cloudmark-SP-Filtered	true
X-Cloudmark-SP-Result	v=1.1 cv=ryrf5q5p6c4dCQYR2lAej91p2ozDt6sfPnRAuS/Q8hc= c=1 sm=1 a=-qo-AADLuRgA:10 a=7PYXob_7ZXMA:10 a=P90J6pEA2ccA:10 a=BLceEmwcHowA:10 a=kj9zAlcOel0A:10 a=xqWC_Br6kY4A:10 a=oNw28mxuUhXRB3mVwYQ4Ag==:17 a=U1EaSfPFy1oQR3hYnxIA:9 a=CjuIK1q_8ugA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/simple; d=activenetwerx.com; s=default; t=1391711059; bh=b9DtUns1GKAeDJBru15UjRv9PhMqQ8GyieGbuDDbBrI=; h=From:To:Subject:Date; b=0bn5hmfJCZswWGudNlBAI/VltoK+J3PKYD8syq+8UDMmzUomt/U54eqKuI5nD6qXV /Ons1Oi5oIaBYdNQLANe48WjS7czHqlPq8hf7Q2mS3kqe5WH99ZfJDJpzzOgcYHgvj GoC+eQMABrvNEe+LIL6UWiOwsD6bpi4k/W0CnTgM=
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/simple; d=activenetwerx.com; s=default; t=1391711059; bh=b9DtUns1GKAeDJBru15UjRv9PhMqQ8GyieGbuDDbBrI=; h=From:To:Subject:Date; b=0bn5hmfJCZswWGudNlBAI/VltoK+J3PKYD8syq+8UDMmzUomt/U54eqKuI5nD6qXV /Ons1Oi5oIaBYdNQLANe48WjS7czHqlPq8hf7Q2mS3kqe5WH99ZfJDJpzzOgcYHgvj GoC+eQMABrvNEe+LIL6UWiOwsD6bpi4k/W0CnTgM=
X-Spam-Checker-Version	SpamAssassin 3.3.1 (2010-03-16) on mail.activenetwerx.com
X-Spam-Level
X-Spam-Status	No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=ham version=3.3.1
From	"Joseph L. Casale" <jcasale@activenetwerx.com>
To	"python-list@python.org" <python-list@python.org>
Subject	Logging from a multiprocess application
Thread-Topic	Logging from a multiprocess application
Thread-Index	AQHPI2g5N/9yoWvf30GcQM7Csnp4Kw==
Date	Thu, 6 Feb 2014 18:24:17 +0000
Accept-Language	en-US
Content-Language	en-US
X-MS-Has-Attach
X-MS-TNEF-Correlator
x-originating-ip	[172.18.0.4]
Content-Type	text/plain; charset="us-ascii"
Content-Transfer-Encoding	quoted-printable
MIME-Version	1.0
X-Virus-Scanned	ClamAV using ClamSMTP
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.6452.1391711131.18130.python-list@python.org> (permalink)
Lines	17
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1391711131 news.xs4all.nl 2906 [2001:888:2000:d::a6]:54993
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:65548

Show key headers only | View raw

I have a module that has one operation that benefits greatly from being multiprocessed.
Its a console based module and as such I have a stream handler and filter associated to
the console, obviously the mp based instances need special handling, so I have been
experimenting with a socket server in a thread in order for the remaining application to
carry on.

How have others tackled this problem? The portion of the code made to use multiprocessing
can not be switched to threading as it performs worse than simply serializing each task.

Thanks,
jlc

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Logging from a multiprocess application "Joseph L. Casale" <jcasale@activenetwerx.com> - 2014-02-06 18:24 +0000
  Re: Logging from a multiprocess application Mark Betz <betz.mark@gmail.com> - 2014-02-06 11:45 -0800
    RE: Logging from a multiprocess application "Joseph L. Casale" <jcasale@activenetwerx.com> - 2014-02-06 21:38 +0000

csiph-web