Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!us.feeder.erje.net!feeder.erje.net!eu.feeder.erje.net!eternal-september.org!feeder.eternal-september.org!mx04.eternal-september.org!.POSTED!not-for-mail From: Jim Janney Newsgroups: comp.lang.java.programmer Subject: Re: The first 10 files Date: Sat, 26 Jan 2013 20:57:03 -0700 Organization: fat-tailed contagion Lines: 48 Message-ID: References: <51041ff8$0$284$14726298@news.sunsite.dk> <1iop8bl8ysrfg$.rdxcxhgxuj1r$.dlg@40tude.net> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Injection-Info: mx04.eternal-september.org; posting-host="3c5d835c9a0219e1946f4dd3740c28b0"; logging-data="26391"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18NjkGkEC6qyoNTXapbZkQp" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) Cancel-Lock: sha1:ZFTgHU2dn8JWTGSebHYof687DAg= sha1:nHH9vOpfN3LWSxr9poKe+eiEJGA= Xref: csiph.com comp.lang.java.programmer:21785 Arved Sandstrom writes: > On 01/26/2013 09:42 PM, Eric Sosman wrote: >> On 1/26/2013 6:21 PM, Peter Duniho wrote: >>> On Sat, 26 Jan 2013 17:06:07 -0500, Eric Sosman wrote: >>> >>>> On 1/26/2013 4:15 PM, Robert Klemme wrote: >>>>> On 26.01.2013 19:26, Arne Vajhøj wrote: >>>>> >>>>>> But I am a bit skeptical about whether a String[] with 30K elements >>>>>> is really the bottleneck. >>>>>> >>>>>> If the real bottleneck is the OS calls to get next file, then >>>>>> a filter like this will not help. >>>>> >>>>> Why? >>>> >>>> Because the listFiles() method will fetch the information >>>> for all 30K files from the O/S, will construct 30K File objects >>>> to represent them, and will submit all 30K File objects to the >>>> FileFilter, one by one. The FileFilter will (very quickly) >>>> reject 29.99K of the 30K Files, but ... >>> >>> Will it? >> >> Necessarily. As far as listFiles() knows, the FileFilter >> might accept the very last File object given to it. Therefore, >> listFiles() cannot fail to present that very last File -- and >> every other File -- for inspection. > [ SNIP ] > > I'd have to agree. A simple test shows this to be the case, but your > reasoning precludes having to run such a test in the first place. > > My code "gets' the first N files from listFiles(), for some definition > of "first", but it certainly doesn't only get N files from the OS. > > Based on Wojtek's later post, I'd be examining the entire problem in > more detail before arriving at a decent solution. I don't think most > of the problem pertaining to offering reasonable batches of files to a > Java program for processing is something that I'd address in Java > anyway. There's also the problem of starvation, since we have no guarantees concerning the order of entries in the directory. -- Jim Janney