Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #2296

Re: multiprocessing Pool.imap broken?

Path csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.albasani.net!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <yanghatespam@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'received:209.85.212.46': 0.03; 'received:mail-vw0-f46.google.com': 0.03; 'pypi': 0.04; 'responding': 0.04; 'ubuntu': 0.04; 'wed,': 0.04; 'interpreter': 0.07; 'url:mit': 0.07; 'python': 0.07; 'advance.': 0.09; '>>>>': 0.09; '[0,': 0.09; 'interpreter.': 0.09; 'subject:broken': 0.09; 'pm,': 0.11; 'examples': 0.12; 'am,': 0.14; 'described': 0.14; 'wrote:': 0.14; '"copyright",': 0.16; '"credits"': 0.16; '"license"': 0.16; '[gcc': 0.16; 'importable': 0.16; 'jumps': 0.16; 'linux2': 0.16; 'python2.6': 0.16; 'case.': 0.16; 'tue,': 0.20; 'cc:no real name:2**0': 0.20; 'cc:2**0': 0.20; 'programming': 0.20; 'header:In-Reply-To:1': 0.22; 'cc:addr :python-list': 0.22; 'received:209.85.212': 0.25; 'cases': 0.25; 'pointing': 0.25; 'says': 0.25; 'guess': 0.26; 'url:mailman': 0.27; 'tried': 0.27; 'message-id:@mail.gmail.com': 0.28; 'url:edu': 0.28; 'problem': 0.29; 'subject:?': 0.29; 'cc:addr:python.org': 0.31; 'anyone': 0.31; 'import': 0.32; "i've": 0.33; 'url:listinfo': 0.33; "isn't": 0.34; 'using': 0.34; 'actually': 0.34; 'requires': 0.35; 'correctly': 0.35; 'suggestion': 0.35; 'print': 0.35; 'running': 0.36; 'some': 0.37; 'however': 0.37; 'case': 0.37; 'received:209.85': 0.37; 'url:python': 0.37; 'issue': 0.37; 'apr': 0.38; 'received:google.com': 0.38; 'url:org': 0.38; 'received:209': 0.39; 'how': 0.39; 'behavior': 0.40; 'header:Received:5': 0.40; 'note:': 0.61; '2011': 0.62; 'worth': 0.64; 'mar': 0.64; 'here.': 0.68; 'states': 0.69; 'children.': 0.77; '30,': 0.80; "(i've": 0.84; '12:12': 0.84; 'hints?': 0.84; 'imap': 0.84; 'zhang': 0.84
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=QuNNktzvML7unlXQGPT/Yllrpe8yMVH4NE8UGhAbne8=; b=RQDP/ZTZPp558fn4cnVJvNkNyNANbcCO4n9FD2Jp4yDwMHFewzxXH+vQHc9PRUXlDP uIwwiNqcRo1EHCMzFLLOjj3Sr8o8c2kZ+UqkW5rRTOcNWvnKav8+JDXvqoaWJVszHSO2 GnnUFAVZ7Y1/9u6FESGslvFPmpG2EOyAhY0Pg=
DomainKey-Signature a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=hpjS7rQa2LuEiwNSABfzCTdvSTSsqoL98mfgdtJ8nGki+raFWKgkyLq01upnz6pA6N aFhhbIMTGJzzINXg67BuHXrAW8fUFUdsv44G+7IEHyVf5NauZq+1o4UXu2l9pakwelcC kEzuZSNgbrS84Dr3s5dmN9ml8N5zsdVICCD0c=
MIME-Version 1.0
In-Reply-To <AANLkTimckqBVed4uTVEDdxDva6bf+7x0c1ztWiwNvi5V@mail.gmail.com>
References <AANLkTinoHq9+E_+Q5Fbv7=6uihw0965GbFr--EPgckxW@mail.gmail.com> <AANLkTin6iLF4EBSt+FdR+qbdPWB8woxb6i8SYgdexaCj@mail.gmail.com> <AANLkTimckqBVed4uTVEDdxDva6bf+7x0c1ztWiwNvi5V@mail.gmail.com>
From Yang Zhang <yanghatespam@gmail.com>
Date Thu, 31 Mar 2011 11:47:04 -0700
Subject Re: multiprocessing Pool.imap broken?
To "kyle.j.conroy@gmail.com" <kyle.j.conroy@gmail.com>
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding quoted-printable
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.12
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.41.1301597246.2990.python-list@python.org> (permalink)
Lines 76
NNTP-Posting-Host 82.94.164.166
X-Trace 1301597246 news.xs4all.nl 41113 [::ffff:82.94.164.166]:34469
X-Complaints-To abuse@xs4all.nl
Xref x330-a1.tempe.blueboxinc.net comp.lang.python:2296

Show key headers only | View raw

My self-reply tried to preempt your suggestion :)

On Wed, Mar 30, 2011 at 12:12 AM, kyle.j.conroy@gmail.com
<kyle.j.conroy@gmail.com> wrote:
> Yang,
>
> My guess is that you are running into a problem using multiprocessing with
> the interpreter. The documentation states that Pool may not work correctly
> in this case.
>>
>> Note: Functionality within this package requires that the __main__ method
>> be importable by the children. This is covered in Programming guidelines
>> however it is worth pointing out here. This means that some examples, such
>> as the multiprocessing.Pool examples will not work in the interactive
>> interpreter.
>
> Hope this helps,
>
> Kyle
>
>
> On Tue, Mar 29, 2011 at 6:51 PM, Yang Zhang <yanghatespam@gmail.com> wrote:
>>
>> On Tue, Mar 29, 2011 at 6:44 PM, Yang Zhang <yanghatespam@gmail.com>
>> wrote:
>> > I've tried both the multiprocessing included in the python2.6 Ubuntu
>> > package (__version__ says 0.70a1) and the latest from PyPI (2.6.2.1).
>> > In both cases I don't know how to use imap correctly - it causes the
>> > entire interpreter to stop responding to ctrl-C's.  Any hints?  Thanks
>> > in advance.
>> >
>> > $ python
>> > Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
>> > [GCC 4.4.3] on linux2
>> > Type "help", "copyright", "credits" or "license" for more information.
>> >>>> import multiprocessing as mp
>> >>>> mp.Pool(1).map(abs, range(3))
>> > [0, 1, 2]
>> >>>> list(mp.Pool(1).imap(abs, range(3)))
>> > ^C^C^C^C^\Quit
>> >
>>
>> In case anyone jumps on this, this isn't an issue with running from the
>> console:
>>
>> $ cat /tmp/go3.py
>> import multiprocessing as mp
>> print mp.Pool(1).map(abs, range(3))
>> print list(mp.Pool(1).imap(abs, range(3)))
>>
>> $ python /tmp/go3.py
>> [0, 1, 2]
>> ^C^C^C^C^C^\Quit
>>
>> (I've actually never seen the behavior described in the corresponding
>> Note at the top of the multiprocessing documentation.)
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>



-- 
Yang Zhang
http://yz.mit.edu/

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: multiprocessing Pool.imap broken? Yang Zhang <yanghatespam@gmail.com> - 2011-03-31 11:47 -0700

csiph-web