Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!newsfeed.eweka.nl!eweka.nl!feeder3.eweka.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <tahoemph@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'space.': 0.07; 'python': 0.08; 'tends': 0.09; 'am,': 0.12; '16,': 0.15; 'cc:addr:python- list': 0.15; 'containers': 0.16; 'cores': 0.16; 'heavier': 0.16; 'os.': 0.16; 'scheduled.': 0.16; 'subject:threads': 0.16; 'sucks': 0.16; 'switching': 0.16; 'wed,': 0.17; 'wrote:': 0.18; 'seems': 0.19; 'trying': 0.21; 'cc:no real name:2**0': 0.21; 'maybe': 0.21; 'asked': 0.22; "doesn't": 0.23; 'received:209.85.212.46': 0.23; 'received:mail-vw0-f46.google.com': 0.23; 'weight': 0.23; "shouldn't": 0.23; '\xa0if': 0.23; 'header:In-Reply-To:1': 0.23; 'separate': 0.28; 'cc:addr:python.org': 0.29; 'message- id:@mail.gmail.com': 0.29; 'problem': 0.29; 'cc:addr:gmail.com': 0.30; 'generally': 0.30; 'false.': 0.30; 'processes.': 0.30; 'threads': 0.30; 'least': 0.31; 'nov': 0.31; 'pm,': 0.31; 'michael': 0.31; 'cases': 0.32; 'comment': 0.32; 'operating': 0.33; 'received:209.85.212': 0.34; 'scheduling': 0.34; 'things': 0.35; 'external': 0.35; 'core': 0.36; 'cc:2**1': 0.36; 'thread': 0.36; 'but': 0.37; 'run': 0.38; 'received:google.com': 0.38; 'some': 0.38; 'received:209.85': 0.38; 'starting': 0.38; 'basic': 0.39; 'subject:: ': 0.39; 'change': 0.39; 'received:209': 0.40; 'really': 0.40; 'within': 0.60; 'managing': 0.60; 'address': 0.61; '2011': 0.62; 'overhead.': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=qNgGt0AOQHBIQ+DrfjbZSmKKFsZC6lA/pO13QNRg44c=; b=wxdweInlQ0VPspNGOV7m8xW1YQ+2OG5b0bnDOn0n4t/ZXj4NEsNCeCXxTDRt683qIM N92pvMOmfkcKI3gI8dfRz2vy4EFJFrjzyRW9WzEy2rdmuloT7G+RpmjYr2Prg7ROXBLu qztU1oqdYbHYaK/x0Z4CxUiv4GtG935Cy6TNQ=
MIME-Version: 1.0
In-Reply-To: <4EC3F298.1030604@davea.name>
References: <31766634.4.1321451296410.JavaMail.geo-discussion-forums@yqcm23> <CAPTjJmoAWni4-wyKB4HgAJ5KvTbz42_BC3f22MOnugUmmXzVtA@mail.gmail.com> <CAAdqOWF=irJXFO2q2+o8jpM09GMLf8N+NTNGTOtMhwKgtG6jag@mail.gmail.com> <4EC3F298.1030604@davea.name>
Date: Wed, 16 Nov 2011 09:55:24 -0800
Subject: Re: Multiple threads
From: Michael Hunter <tahoemph@gmail.com>
To: d@davea.name
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2776.1321466127.27778.python-list@python.org>
Lines: 25
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1321466127 news.xs4all.nl 6940 [2001:888:2000:d::a6]:56349
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:15781

On Wed, Nov 16, 2011 at 9:27 AM, Dave Angel <d@davea.name> wrote:
> On 11/16/2011 12:00 PM, Jack Keegan wrote:
>[...] Processes [...] and the OS is generally better at scheduling them th=
an it is at
> scheduling threads within a single process. =A0If you have multiple cores=
, the
> processes can really run simultaneously, frequently with very small
> overhead. =A0[...]

Maybe you are trying to simplify things but in a lot of cases this is
just false.  In at least some operating systems these days a thread is
the basic unit that is scheduled.  Processes are thread containers
that provide other things (fds, separate address space, etc.).  The
comment about multiple cores can be extended to multiple threads on a
core (CMT) but applies to threads as well as processes.  Switching
between processes tends to be heavier weight then switching between
threads in a process because of the needs to change the address space.

Just because Python sucks at threads doesn't make them heavier for the OS.

That doesn't mean you shouldn't use multiprocessing.  The problem
asked about seems a good fit to me to a single python process starting
and managing a set of external converter processes.

              Michael