Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #56598

Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing

Path csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <nedbat@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.003
X-Spam-Evidence '*H*': 0.99; '*S*': 0.00; 'subject:not': 0.03; 'subject:Python': 0.06; 'mentioned,': 0.07; 'subject: -- ': 0.07; 'filename': 0.09; 'pgp': 0.09; 'skip:# 30': 0.09; 'subject:files': 0.09; 'cc:addr:python-list': 0.11; 'posted': 0.15; '"with"': 0.16; 'processes.': 0.16; 'thursday,': 0.16; 'sender:addr:gmail.com': 0.17; 'wrote:': 0.18; '>>>': 0.22; 'example': 0.22; 'cc:addr:python.org': 0.22; 'header:User-Agent:1': 0.23; 'convenient': 0.24; 'sorry,': 0.24; 'text,': 0.24; 'fine': 0.24; 'looks': 0.24; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; 'this:': 0.26; 'van': 0.27; 'header:In-Reply-To:1': 0.27; 'function': 0.29; "doesn't": 0.30; 'statement': 0.30; 'work.': 0.31; 'code': 0.31; "skip:' 10": 0.31; 'you?': 0.31; 'large.': 0.31; 'writes:': 0.31; 'file': 0.32; 'probably': 0.32; 'stuff': 0.32; 'quite': 0.32; 'problem': 0.35; 'something': 0.35; 'received:google.com': 0.35; 'should': 0.36; 'skip:[ 10': 0.38; 'issue': 0.38; 'pm,': 0.38; 'remove': 0.60; 'most': 0.60; 'providing': 0.61; 'here:': 0.62; 'complete': 0.62; 'more': 0.64; 'reply': 0.66; 'obvious': 0.74; 'isaac': 0.84; 'subject:skip:M 10': 0.84; '2013': 0.98
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=nYQsy5x0dM/9Nkg4qtY1HxtZusXvE5lFdg0itLZYuDY=; b=kzAnSCiKmqGRZDYpL/IEl6jB66TU4sYWh0+51dvN2Vxkz03Ce9u20JbMd6rVQ4cDX3 vxsbcOMKqG7h7t+Un76ty1rw7yx1uIXbOOqwlygJqiny/jsHYnsZOafF0Zs14QR+RJbQ bCwMYREJ4lGGn7zvHMZWHyYa1BiCHApk1/ZXRffG0Aytp6LHCbBwu2kVpj26tPDgBodB qqPzePRUmCww/X9PLhZH9herfnZUmmK5Cys97LZ2+E0SlO+oBlTowp5Pr2IlfGCoY8D9 MPnHXCtn/1JH2bh2Xmlokr4edYBOYk59xrMx5t9AERughNP0UJHm+O959RnZrJE/WIbA hPpw==
X-Received by 10.49.24.74 with SMTP id s10mr15055751qef.24.1381430467476; Thu, 10 Oct 2013 11:41:07 -0700 (PDT)
Sender Ned Batchelder <nedbat@gmail.com>
Date Thu, 10 Oct 2013 14:41:05 -0400
From Ned Batchelder <ned@nedbatchelder.com>
User-Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8
MIME-Version 1.0
To Isaac Gerg <isaac.gerg@gergltd.com>
Subject Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
References <c9e59679-f9de-41e8-99e9-58a0d656b683@googlegroups.com> <m261t5xero.fsf@cochabamba.vanoostrum.org> <73f03f7c-6a0a-449c-af60-ab204844d578@googlegroups.com>
In-Reply-To <73f03f7c-6a0a-449c-af60-ab204844d578@googlegroups.com>
Content-Type text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding 7bit
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.956.1381430477.18130.python-list@python.org> (permalink)
Lines 59
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1381430477 news.xs4all.nl 15919 [2001:888:2000:d::a6]:34809
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:56598

Show key headers only | View raw

On 10/10/13 12:44 PM, Isaac Gerg wrote:
> Sorry, I am just providing pseudo code since I the code i have is quite large.
>
> As I mentioned, the code works fine when I remove the multirpcessing stuff so the filename is not the issue (though you are right in your correction).
>
> Someone with the same problem posted a smaller, more complete example here:
>
> http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib
>
> None of the solutions posted work.

(BTW: it's better form to reply beneath the original text, not above it.)

None of the solutions try the obvious thing of closing the file before 
spawning more processes.  Would that work for you?  A "with" statement 
is a convenient way to do this:

     with open(filename,'r') as f:
         data = f.read()

The file is closed automatically when the with statement ends.

--Ned.

>
> On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote:
>> Isaac Gerg <isaac.gerg@gergltd.com> writes:
>>
>>
>>
>>> I have a function that looks like the following:
>>
>>
>> That doesn't look like a function
>>
>>
>>
>>> #---------------------------------
>>> filename = 'c:\testfile.h5'
>>
>>
>> Your filename is most probably wrong. It should be something like:
>>
>>
>>
>> filename = 'c:/testfile.h5'
>>
>> filename = 'c:\\testfile.h5'
>>
>> filename = r'c:\testfile.h5'
>>
>> -- 
>>
>> Piet van Oostrum <piet@vanoostrum.org>
>>
>> WWW: http://pietvanoostrum.com/
>>
>> PGP key: [8DAE142BE17999C4]

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 08:31 -0700
  Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Piet van Oostrum <piet@vanoostrum.org> - 2013-10-10 12:38 -0400
    Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 09:44 -0700
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Ned Batchelder <ned@nedbatchelder.com> - 2013-10-10 14:41 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 14:49 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 15:00 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Piet van Oostrum <piet@vanoostrum.org> - 2013-10-10 15:38 -0400
        Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 14:15 -0700
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Terry Reedy <tjreedy@udel.edu> - 2013-10-10 21:45 -0400
  Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing mapoe <matthias@kill.at> - 2013-10-10 18:16 -0500
    Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing mapoe <matthias@kill.at> - 2013-10-10 18:29 -0500

csiph-web