Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #75840

Re: Test for an empty directory that could be very large if it is not empty?

Path csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin1!goblin2!goblin.stu.neva.ru!newsfeed1.swip.net!uio.no!news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path <python.list@tim.thechases.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.027
X-Spam-Evidence '*H*': 0.95; '*S*': 0.00; 'subject:not': 0.03; 'algorithm': 0.04; 'cc:addr:python-list': 0.11; '"is': 0.16; '-tkc': 0.16; 'assumed,': 0.16; 'chasing': 0.16; 'finney': 0.16; 'from:addr:python.list': 0.16; 'from:addr:tim.thechases.com': 0.16; 'from:name:tim chase': 0.16; 'iterable': 0.16; 'iteration': 0.16; 'latter,': 0.16; 'sees': 0.16; 'subject: \n ': 0.16; 'wrote:': 0.18; 'code.': 0.18; '(in': 0.22; 'cc:addr:python.org': 0.22; 'cc:2**0': 0.24; 'cc:no real name:2**0': 0.24; 'header:In- Reply-To:1': 0.27; 'matching': 0.30; "i'm": 0.30; 'subject:that': 0.31; 'file': 0.32; 'quite': 0.32; '(e.g.': 0.33; 'guess': 0.33; 'common': 0.35; 'requirement': 0.35; 'but': 0.35; 'subject:?': 0.36; 'so,': 0.37; 'list': 0.37; 'performance': 0.37; 'ben': 0.38; 'files': 0.38; 'structure': 0.39; 'easy': 0.60; 'serving': 0.60; 'different': 0.65; 'forward': 0.65; 'surprise': 0.74; '"look': 0.84; 'ethan': 0.84; 'optimisation': 0.84; 'received:50.22': 0.84; 'subject:Test': 0.84; 'terrible': 0.84; 'timings': 0.84; 'subject:very': 0.91
Date Thu, 7 Aug 2014 06:07:37 -0500
From Tim Chase <python.list@tim.thechases.com>
To Ben Finney <ben+python@benfinney.id.au>
Subject Re: Test for an empty directory that could be very large if it is not empty?
In-Reply-To <85ppgd9jio.fsf@benfinney.id.au>
References <53E2A22A.10906@it.uu.se> <854mxpb6fq.fsf@benfinney.id.au> <20140806174441.3a2e16e1@bigbox.christie.dr> <85ppgd9jio.fsf@benfinney.id.au>
X-Mailer Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu)
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8
Content-Transfer-Encoding quoted-printable
X-AntiAbuse This header was added to track abuse, please include it with any abuse report
X-AntiAbuse Primary Hostname - boston.accountservergroup.com
X-AntiAbuse Original Domain - python.org
X-AntiAbuse Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse Sender Address Domain - tim.thechases.com
X-Get-Message-Sender-Via boston.accountservergroup.com: authenticated_id: tim@thechases.com
Cc python-list@python.org
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.12723.1407409742.18130.python-list@python.org> (permalink)
Lines 26
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1407409742 news.xs4all.nl 2868 [2001:888:2000:d::a6]:43674
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:75840

Show key headers only | View raw


On 2014-08-07 11:27, Ben Finney wrote:
> > The difference in timings when serving a web-request are
> > noticeable (in my use-case, I had to change my algorithm and
> > storage structure to simplify/avoid heavily-populated
> > directories)  
> 
> So, if the requirement is “test whether the directory is empty
> faster than N microseconds”, that's quite different from “without
> the generation of a list of the file names”.
> 
> The former may entail the latter, but that's not to be assumed, and
> chasing an optimisation prematurely is a common cause of terrible
> code.

I guess my surprise in the 2->3 non-iterator'ization of
os.listdir() is that it's very easy to wrap an iterable in list()
if you want the whole bunch, but it's much harder to get the
performance characteristics of interruptible iteration (e.g. "is the
directory empty" or "look at files until you find one matching
$CRITERIA").  Looking forward to scandir() arriving for just those
reasons.

-tkc
(who sees Ethan Furman's excellent followup-post as I'm about to hit
Send)

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread


Thread

Re: Test for an empty directory that could be very large if it is not empty? Tim Chase <python.list@tim.thechases.com> - 2014-08-07 06:07 -0500

csiph-web