Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #75814 > unrolled thread

Re: Test for an empty directory that could be very large if it is not empty?

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Test for an empty directory that could be very large if it is not empty? Ben Finney <ben+python@benfinney.id.au> - 2014-08-07 08:26 +1000
    Re: Test for an empty directory that could be very large if it is not empty? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-07 13:25 +1000

#75814 — Re: Test for an empty directory that could be very large if it is not empty?

Virgil Stokes <vs@it.uu.se> writes:

> Suppose I have a directory C:/Test that is either empty or contains
> more than 2000000 files, all with the same extension (e.g. *.txt). How
> can I determine if the directory is empty WITHOUT the generation of a
> list of the file names in it (e.g. using os.listdir('C:/Test')) when
> it is not empty?

What is your goal for that? Have you measured the performance difference
and decided *based on objective observation* that it's too expensive?

Certainly ‘os.listdir(foo)’ is the simplest way to determine the entries
in a directory, and thereby to test whether it is empty. That simplicity
is very valuable, and you should have a compelling, *measured* reason to
do something more complicated. What is it?

-- 
 \         “The most dangerous man to any government is the man who is |
  `\       able to think things out for himself, without regard to the |
_o__)          prevailing superstitions and taboos.” —Henry L. Mencken |
Ben Finney

[toc] | [next] | [standalone]

#75824

Ben Finney wrote:

> Virgil Stokes <vs@it.uu.se> writes:
> 
>> Suppose I have a directory C:/Test that is either empty or contains
>> more than 2000000 files, all with the same extension (e.g. *.txt). How
>> can I determine if the directory is empty WITHOUT the generation of a
>> list of the file names in it (e.g. using os.listdir('C:/Test')) when
>> it is not empty?
> 
> What is your goal for that? Have you measured the performance difference
> and decided *based on objective observation* that it's too expensive?

Normally I would agree with you, but this is one case where there is no need
to measure, we can tell in advance that at least sometimes there will be a
severe performance hit simply by considering the nature of file systems. In
particular, consider the case where the directory is a remote file system
on the other side of the world over a link with many dropped packets or
other noise. Waiting for 200 thousand file names to be transmitted, only to
throw them away, is surely going to be slower than (say) the results of a
call to os.stat. (Assuming that gives the answer.)

The difficult question then becomes: is it reasonable to (potentially) slow
down the common case of local file systems by a tiny amount, in order to
protect against the (rare) case where it will give a big speed things up?



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web