Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #90867

Re: fork/exec & close file descriptors

Path csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!feeder1.cambriumusenet.nl!feed.tweaknews.nl!194.109.133.81.MISMATCH!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <ethan@stoneleaf.us>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'url:pypi': 0.03; 'else:': 0.03; 'elif': 0.05; 'level,': 0.07; 'method.': 0.07; 'subject:file': 0.07; 'from:addr:ethan': 0.09; 'from:addr:stoneleaf.us': 0.09; 'from:name:ethan furman': 0.09; 'message-id:@stoneleaf.us': 0.09; 'restart': 0.09; 'try:': 0.09; 'yeah,': 0.09; '~ethan~': 0.09; 'python': 0.11; 'def': 0.12; '-1):': 0.16; '-1,': 0.16; 'descriptors': 0.16; 'file))': 0.16; 'number?': 0.16; 'os.close(fd)': 0.16; 'set()': 0.16; 'underlying': 0.16; 'wrote:': 0.18; 'library': 0.18; 'basically': 0.19; 'header:User-Agent:1': 0.23; 'skip': 0.24; 'subject:/': 0.26; 'header:In-Reply-To:1': 0.27; '[1]': 0.29; 'am,': 0.29; 'raise': 0.29; '[2]': 0.30; "i'm": 0.30; 'file': 0.32; 'stuff': 0.32; 'open': 0.33; 'url:python': 0.33; 'bugs': 0.33; 'basic': 0.35; 'except': 0.35; 'something': 0.35; 'but': 0.35; 'dance': 0.36; 'object,': 0.36; 'url:org': 0.36; 'should': 0.36; 'so,': 0.37; 'received:10': 0.37; 'ben': 0.38; 'to:addr:python-list': 0.38; 'sure': 0.39; 'to:addr:python.org': 0.39; 'either': 0.39; 'how': 0.40; 'received:10.1': 0.61; 'new': 0.61; 'received:173': 0.61; 'simply': 0.61; 'hours': 0.66; 'due': 0.66; 'close': 0.67; 'believe': 0.68; 'skip:r 40': 0.68; 'subject: & ': 0.68; 'discover': 0.82; 'received:10.1.10': 0.84
Date Tue, 19 May 2015 09:35:58 -0700
From Ethan Furman <ethan@stoneleaf.us>
User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0
MIME-Version 1.0
To python-list@python.org
Subject Re: fork/exec & close file descriptors
References <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
In-Reply-To <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
Content-Type text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding 7bit
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.143.1432053377.17265.python-list@python.org> (permalink)
Lines 39
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1432053377 news.xs4all.nl 2828 [2001:888:2000:d::a6]:36062
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:90867

Show key headers only | View raw


On 05/19/2015 05:59 AM, Skip Montanaro wrote:

> Due to presumed bugs in an underlying library over which I have no control, I'm considering a restart in the wee hours of the morning. The basic fork/exec dance is not a problem, but how do I discover
> all the open file descriptors in the new child process to make sure they get closed? Do I simply start at fd 3 and call os.close() on everything up to some largish fd number? Some of these file
> descriptors will have been opened by stuff well below the Python level, so I don't know them a priori.

Pandaemonium [1] (and I believe Ben Finney's daemon [2]) use something akin to the following:

def close_open_files(exclude):
     max_files = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
     keep = set()
     for file in exclude:
         if isinstance(file, baseint):
             keep.add(file)
         elif hasattr(file, 'fileno'):
             keep.add(file.fileno())
         else:
             raise ValueError(
                     'files to not close should be either an file descriptor, '
                     'or a file-type object, not %r (%s)' % (type(file), file))
     for fd in range(max_files, -1, -1):
         if fd in keep:
             continue
         try:
             os.close(fd)
         except OSError:
             exc = sys.exc_info()[1]
             if exc.errno == errno.EBADF:
                 continue
             raise

So, yeah, basically a brute-force method.

--
~Ethan~


[1] https://pypi.python.org/pypi/pandaemonium
[2] https://pypi.python.org/pypi/python-daemon

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: fork/exec & close file descriptors Ethan Furman <ethan@stoneleaf.us> - 2015-05-19 09:35 -0700

csiph-web