Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!tudelft.nl!txtfeed1.tudelft.nl!feeder2.cambriumusenet.nl!feed.tweaknews.nl!62.179.104.142.MISMATCH!amsnews11.chello.com!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <tobias.oberstein@tavendo.de>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'bsd': 0.05; 'flags': 0.05; 'happily': 0.07; 'python': 0.08; 'descriptors': 0.09; 'files:': 0.09; 'sockets': 0.09; 'subject:files': 0.09; 'files.': 0.09; 'to:name:python-list@python.org': 0.15; 'tries': 0.15; 'limit.': 0.16; 'open()': 0.16; 'strange:': 0.16; 'twisted': 0.16; 'linux': 0.17; 'contribute': 0.18; 'to:2**1': 0.19; 'connections': 0.21; 'file,': 0.21; 'options.': 0.21; "doesn't": 0.23; 'header:In- Reply-To:1': 0.23; "i'm": 0.26; 'beyond': 0.27; 'subject:number': 0.30; 'point,': 0.32; 'familiar': 0.32; 'to:addr:python-list': 0.32; 'received:169': 0.34; 'received:169.254': 0.34; 'thus,': 0.34; 'similar': 0.35; 'subject:/': 0.35; 'file': 0.36; 'charset :us-ascii': 0.36; 'doing': 0.37; 'machine': 0.37; 'but': 0.37; 'options': 0.37; 'configured': 0.38; 'open': 0.38; 'else': 0.38; "it's": 0.39; 'why': 0.39; 'subject:: ': 0.39; 'might': 0.39; 'to:addr:python.org': 0.39; 'more': 0.60; 'total': 0.61; 'dedicated': 0.63; 'high': 0.66; 'limit': 0.67; '100': 0.69; 'soon': 0.72; 'service)': 0.84
From: Tobias Oberstein <tobias.oberstein@tavendo.de>
To: Christian Heimes <lists@cheimes.de>, "python-list@python.org" <python-list@python.org>
Date: Mon, 14 Nov 2011 08:36:37 -0800
Subject: AW: Py2.7/FreeBSD: maximum number of open files
Thread-Topic: Py2.7/FreeBSD: maximum number of open files
Thread-Index: Acyi6m0OulAiwHhtQ8m6S375sA1/EgAAIDWw
References: <634914A010D0B943A035D226786325D42D0C264772@EXVMBX020-12.exch020.serverdata.net> <j9rfda$mfs$1@dough.gmane.org>
In-Reply-To: <j9rfda$mfs$1@dough.gmane.org>
Accept-Language: de-DE, en-US
Content-Language: de-DE
acceptlanguage: de-DE, en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2700.1321289211.27778.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1321289211 news.xs4all.nl 6881 [2001:888:2000:d::a6]:37347
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:15675

> I'm not familiar with BSD but Linux has similar Kernel options. The kerne=
l
> options might be *global* flags to set the total upper limit of open file
> descriptors for the entire system, not for a single process.
> Also on Linux "ulimit" doesn't display the fd limit. You have to use "uli=
mit -n".

This is a dedicated machine doing nothing else .. I'm monitoring global FD =
usage

sysctl kern.openfiles

and it's way beyond the configured limit

$ ulimit -n
200000


>=20
> Why do you need more than 32k file descriptors anyway? It's an insanely h=
igh

It's not for files:

This is a network service .. I tested it with up to 50k TCP connections .. =
however
at this point, when the service tries to open a file, it'll bail out.

Sockets+Files both contribute to open FDs.

I need 50k sockets + 100 files.

Thus, this is even more strange: the Python (a Twisted service) will happil=
y
accept 50k sockets, but as soon as you do open() a file, it'll bail out.