Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #56601

Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Thu, Oct 10, 2013 at 2:49 PM, Isaac Gerg <isaac.gerg@gergltd.com> wrote:

>
>
>
> On Thu, Oct 10, 2013 at 2:41 PM, Ned Batchelder <ned@nedbatchelder.com>wrote:
>
>> On 10/10/13 12:44 PM, Isaac Gerg wrote:
>>
>>> Sorry, I am just providing pseudo code since I the code i have is quite
>>> large.
>>>
>>> As I mentioned, the code works fine when I remove the multirpcessing
>>> stuff so the filename is not the issue (though you are right in your
>>> correction).
>>>
>>> Someone with the same problem posted a smaller, more complete example
>>> here:
>>>
>>> http://stackoverflow.com/**questions/948119/preventing-**
>>> file-handle-inheritance-in-**multiprocessing-lib<http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib>
>>>
>>> None of the solutions posted work.
>>>
>>
>> (BTW: it's better form to reply beneath the original text, not above it.)
>>
>> None of the solutions try the obvious thing of closing the file before
>> spawning more processes.  Would that work for you?  A "with" statement is a
>> convenient way to do this:
>>
>>     with open(filename,'r') as f:
>>         data = f.read()
>>
>> The file is closed automatically when the with statement ends.
>>
>> --Ned.
>>
>>
>>> On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote:
>>>
>>>> Isaac Gerg <isaac.gerg@gergltd.com> writes:
>>>>
>>>>
>>>>
>>>>  I have a function that looks like the following:
>>>>>
>>>>
>>>>
>>>> That doesn't look like a function
>>>>
>>>>
>>>>
>>>>  #-----------------------------**----
>>>>> filename = 'c:\testfile.h5'
>>>>>
>>>>
>>>>
>>>> Your filename is most probably wrong. It should be something like:
>>>>
>>>>
>>>>
>>>> filename = 'c:/testfile.h5'
>>>>
>>>> filename = 'c:\\testfile.h5'
>>>>
>>>> filename = r'c:\testfile.h5'
>>>>
>>>> --
>>>>
>>>> Piet van Oostrum <piet@vanoostrum.org>
>>>>
>>>> WWW: http://pietvanoostrum.com/
>>>>
>>>> PGP key: [8DAE142BE17999C4]
>>>>
>>>
>>
> I will try what you suggest and see if it works.
>
> Additionally, is there a place on the web to view this conversation and
> reply?  Currently, I am only able to access this list through email.
>


Ned, I am unable to try what you suggest.  The multiprocess.Process call is
within a class but its target is a static method outside of the class thus
no pickling.  I cannot close the file and then reopen after the
multiprocess.Process call because other threads may be reading from the
file during that time.  Is there a way in Python 3.2 to prevent the
multiprocess.Process from inheriting the file descriptors from the parent
process OR, is there a way to ensure that the multiprocess is completely
closed and garbaged collected by the time I want to use os.remove()?

Isaac

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 08:31 -0700
  Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Piet van Oostrum <piet@vanoostrum.org> - 2013-10-10 12:38 -0400
    Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 09:44 -0700
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Ned Batchelder <ned@nedbatchelder.com> - 2013-10-10 14:41 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 14:49 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 15:00 -0400
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Piet van Oostrum <piet@vanoostrum.org> - 2013-10-10 15:38 -0400
        Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Isaac Gerg <isaac.gerg@gergltd.com> - 2013-10-10 14:15 -0700
      Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing Terry Reedy <tjreedy@udel.edu> - 2013-10-10 21:45 -0400
  Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing mapoe <matthias@kill.at> - 2013-10-10 18:16 -0500
    Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing mapoe <matthias@kill.at> - 2013-10-10 18:29 -0500

csiph-web