Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.029
X-Spam-Evidence: '*H*': 0.94; '*S*': 0.00; 'below)': 0.09; 'subject:Why': 0.09; 'subject:module': 0.09; 'subject:using': 0.09; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'subject: \n ': 0.16; 'subject:run': 0.16; 'subject:skip:m 10': 0.16; 'subject:when': 0.16; 'subject:python': 0.16; 'sat,': 0.16; 'wrote:': 0.18; 'bit': 0.19; 'trying': 0.19; 'fine': 0.24; '(see': 0.26; 'shown': 0.26; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'message-id:@mail.gmail.com': 0.30; 'code': 0.31; 'run': 0.32; 'subject:time': 0.33; "can't": 0.35; 'but': 0.35; 'received:google.com': 0.35; 'version': 0.36; 'subject:?': 0.36; 'confirmed': 0.38; 'to:addr:python-list': 0.38; 'does': 0.39; 'use.': 0.39; 'to:addr:python.org': 0.39; 'details.': 0.61; 'hang': 0.67; '26,': 0.68; 'below.': 0.71; 'subject:long': 0.84; 'sorry.': 0.91; 'subject:very': 0.91; '2013': 0.98
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Q7rBo4OMQFi5/8yMo1JnaWiYFB8ylPC3al9UkRseHng=; b=M/NqlycVMZg/fHAcNgNHAR96Gxu108PFmKihuAWjD4IaiNdX48xMWk/Y+k34gW/D98 6pM+wbE3ACt2m8P6/u0EfWSJ0pVf90rOp8dcO/w+uAi4TW3QB6jS9UDfYNpZY8V4ig+w 2vsHQrMzMkAQvT1fGHhxtoeEQ9XsvCCkcB3ERVxtzvyC2wKct8/CYwYZNW+2B9M8ykLs D0hVMVoLAigYUWMB411slpP+sAHij+PoVEIJK3jZfcNnDrDJFM9wi7dlhR2cT3xZqWRg LGhmSvwtcv7490L+lJnxK/DTNjF0hZoc+dzQnltzHAUlM6lMENwfUnb6JL3duLZayzEK T+HQ==
MIME-Version: 1.0
X-Received: by 10.68.189.101 with SMTP id gh5mr9442167pbc.39.1382748492080; Fri, 25 Oct 2013 17:48:12 -0700 (PDT)
In-Reply-To: <CACxeJGrM7L3i323fshfm-NjEtXomwtNEbNepVtJhmv9EnMqixA@mail.gmail.com>
References: <CACxeJGrM7L3i323fshfm-NjEtXomwtNEbNepVtJhmv9EnMqixA@mail.gmail.com>
Date: Sat, 26 Oct 2013 11:48:12 +1100
Subject: Re: Why does lzma hangs for a very long time when run in parallel using python's muptiprocessing module?
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.1567.1382748495.18130.python-list@python.org>
Lines: 10
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1382748495 news.xs4all.nl 15915 [2001:888:2000:d::a6]:48340
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:57586

On Sat, Oct 26, 2013 at 3:21 AM, cantor cantor <cantormath2@gmail.com> wrote:
> When trying to run lzma in parallel (see the code below) it hangs for a very
> long time. The non-parallel version of the code using map() works fine as
> shown in the code below.

Confirmed that your code does indeed hang as you describe, but I can't
help much with the details. I poked around with it a bit but without
finding much of use. Sorry. :(

ChrisA