Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #34987

Re: Iterating over files of a huge directory

Show all headers | View raw

Le lundi 17 décembre 2012 16:52:19 UTC+1, Oscar Benjamin a écrit :
> On 17 December 2012 15:28, Gilles Lenfant <...> wrote:
> 
> 
> In the last couple of months there has been a lot of discussion (on
> 
> python-list or python-dev - not sure) about creating a library to more
> 
> efficiently iterate over the files in a directory. The result so far
> 
> is this library on github:
> 
> https://github.com/benhoyt/betterwalk
> 
> 
> 
> It says there that
> 
> """
> 
> Somewhat relatedly, many people have also asked for a version of
> 
> os.listdir() that yields filenames as it iterates instead of returning
> 
> them as one big list.
> 
> 
> 
> So as well as a faster walk(), BetterWalk adds iterdir_stat() and
> 
> iterdir(). They're pretty easy to use, but see below for the full API
> 
> docs.
> 
> """
> 
> 
> 
> Does that code work for you? If so, I imagine the author would be
> 
> interested to get some feedback on how well it works.
> 
> 
> 
> Alternatively, perhaps consider calling an external utility.
> 

Many thanks for this pointer Oscar.

"betterwalk" is exactly what I was looking for. More particularly iterdir(...) and iterdir_stat(...)
I'll get a deeper look at betterwalk and provide (hopefully successful) feedback.

Cheers
-- 
Gilles Lenfant

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Iterating over files of a huge directory Gilles Lenfant <gilles.lenfant@gmail.com> - 2012-12-17 07:28 -0800
  Re: Iterating over files of a huge directory Chris Angelico <rosuav@gmail.com> - 2012-12-18 02:41 +1100
    Re: Iterating over files of a huge directory Paul Rudin <paul.nospam@rudin.co.uk> - 2012-12-17 17:27 +0000
      Re: Iterating over files of a huge directory MRAB <python@mrabarnett.plus.com> - 2012-12-17 18:29 +0000
      Re: Iterating over files of a huge directory Chris Angelico <rosuav@gmail.com> - 2012-12-18 08:10 +1100
  Re: Iterating over files of a huge directory Tim Golden <mail@timgolden.me.uk> - 2012-12-17 15:48 +0000
  Re: Iterating over files of a huge directory Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-12-17 15:52 +0000
    Re: Iterating over files of a huge directory Gilles Lenfant <gilles.lenfant@gmail.com> - 2012-12-17 08:06 -0800
    Re: Iterating over files of a huge directory Gilles Lenfant <gilles.lenfant@gmail.com> - 2012-12-17 08:06 -0800
  Re: Iterating over files of a huge directory marduk <marduk@python.net> - 2012-12-17 10:50 -0500
  Re: Re: Iterating over files of a huge directory Evan Driscoll <driscoll@cs.wisc.edu> - 2012-12-17 12:40 -0600
  Re: Re: Iterating over files of a huge directory Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2012-12-17 19:50 +0000
  Re: Iterating over files of a huge directory Evan Driscoll <driscoll@cs.wisc.edu> - 2012-12-17 14:09 -0600
  Re: Iterating over files of a huge directory Terry Reedy <tjreedy@udel.edu> - 2012-12-17 16:27 -0500

csiph-web