Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #68700 > unrolled thread

Implement multiprocessing without inheriting parent file handle

Back to article view | Back to comp.lang.python

Contents

  Implement multiprocessing without inheriting parent file handle Antony Joseph <antonyjoseph89@gmail.com> - 2014-03-21 20:58 +0530
    Re: Implement multiprocessing without inheriting parent file handle Mark H Harris <harrismh777@gmail.com> - 2014-03-21 12:25 -0500
    Re: Implement multiprocessing without inheriting parent file handle Grant Edwards <invalid@invalid.invalid> - 2014-03-21 17:37 +0000
    Re: Implement multiprocessing without inheriting parent file handle Marko Rauhamaa <marko@pacujo.net> - 2014-03-21 19:42 +0200
      Re: Implement multiprocessing without inheriting parent file handle Mark H Harris <harrismh777@gmail.com> - 2014-03-21 13:16 -0500
        Re: Implement multiprocessing without inheriting parent file handle Chris Angelico <rosuav@gmail.com> - 2014-03-22 09:50 +1100

#68700 — Implement multiprocessing without inheriting parent file handle

[Multipart message — attachments visible in raw view] — view raw

Hi all,

How can i implement multiprocessing without inherit file descriptors from
my parent process?

Please help me.

regards,
Antony

[toc] | [next] | [standalone]

#68703

On 3/21/14 10:28 AM, Antony Joseph wrote:
> How can i implement multiprocessing without inherit file descriptors
> from my parent process?

I'll bite...

If what you mean by 'multiprocessing' is forking a process, to get a 
child process, which will then do some parallel processing for some 
reason, the answer is two-fold: 1) you can't, and 2) you might try using 
threads.  (there are other answers too)

When you fork a process the only differences between the child and the 
parent is 1) process number, and 2) one is the 'parent' and one is the 
'child'.  They are 'exact' copies of one another.

In socket daemons the parent always listens on a given port, then forks 
a child process to establish a secondary port(s) to handle the comm 
link;  at that point (instant) both the parent and the child ask a 
simple question, "Am I the parent?"  If so, the code goes back to 
listening... if not, the code (the child) establishes the necessary comm 
ports and handles the server request.  When finished the child notifies 
the parent and ends.

You can try parent/child arrangements (maybe use rpc, or shared memory 
segments for inter process comm), or you might try threads... which are 
a separate unit of execution (not a full process copy) but with access 
to the parent's memory.

Otherwise, you might want to tell the list some of the background of 
your difficulty, or put the difficulty in pythonic terms, or simply ask 
your python related question with as much simplified but complete detail 
as possible.

Is there a python question in there someplace?

marcus

[toc] | [prev] | [next] | [standalone]

#68704

On 2014-03-21, Antony Joseph <antonyjoseph89@gmail.com> wrote:

> How can i implement multiprocessing without inherit file descriptors from
> my parent process?

What one typically does if that is desired is to call fork() and then
in the child process close all open file descriptors before doing any
other processsing (such as exec()ing another program).

-- 
Grant Edwards               grant.b.edwards        Yow! My life is a patio
                                  at               of fun!
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#68705

Antony Joseph <antonyjoseph89@gmail.com>:

> How can i implement multiprocessing without inherit file descriptors
> from my parent process?

Take a look at the subprocess module:

  <URL:
  http://docs.python.org/3/library/subprocess.html#popen-constructor>

It's got the optional close_fds parameter, which is True by default.

IOW, you don't need to do anything if you use subprocess.Popen() to
start your child process. Incidentally, that's the preferred way.


Marko

[toc] | [prev] | [next] | [standalone]

#68709

On 3/21/14 12:42 PM, Marko Rauhamaa wrote:
>    http://docs.python.org/3/library/subprocess.html#popen-constructor>

> It's got the optional close_fds parameter, which is True by default.

> IOW, you don't need to do anything if you use subprocess.Popen() to
> start your child process. Incidentally, that's the preferred way.

hi Marko, of course this depends on the environment; if unix-like, then 
yes, elif windows well, not so much...

There are caveats about stdin, stdout, and stderr;  as well, there are 
caveats about passing fds between the two. Well, Popen() is implemented 
on unix and windows differently, so there is some study needed here if 
code is going to be run in both environments.

marcus

[toc] | [prev] | [next] | [standalone]

#68727

On Sat, Mar 22, 2014 at 5:16 AM, Mark H Harris <harrismh777@gmail.com> wrote:
> On 3/21/14 12:42 PM, Marko Rauhamaa wrote:
>>
>>    http://docs.python.org/3/library/subprocess.html#popen-constructor>
>
>
>> It's got the optional close_fds parameter, which is True by default.
>
>
>> IOW, you don't need to do anything if you use subprocess.Popen() to
>> start your child process. Incidentally, that's the preferred way.
>
>
> hi Marko, of course this depends on the environment; if unix-like, then yes,
> elif windows well, not so much...
>
> There are caveats about stdin, stdout, and stderr;  as well, there are
> caveats about passing fds between the two. Well, Popen() is implemented on
> unix and windows differently, so there is some study needed here if code is
> going to be run in both environments.

There were some recent changes, though, including making CLOEXEC the
default, which means that fork/exec on Unix will retain only the fds
you actually want.

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web