Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #12718

Re: I Need to search over 100 largeish text documents efficiently. What's the best approach?

Show all headers | View raw

Am 27.01.2014 02:43, schrieb Denis McMahon:

> On Sun, 26 Jan 2014 05:34:21 -0800, rob.bradford2805 wrote:
> 
>> What is the best/fastest approach to scan 100+ largish text files for
>> word strings
> 
> A quick googling finds:
> 
> http://sourceforge.net/projects/php-grep/
> http://net-wrench.com/download-tools/php-grep.php
> 
> Claims to be able to search 1000 files in under 10 secs

Under ideal conditions - maybe. But if each file is more than 1 MB, it
is barely possible to even read this amount of data in just 10 seconds
(assuming around 80 MB/s and 1000 MB of data to be searched).

Even using a simple word index (word plus the name of the file(s) and
the position(s) where the word is located) would be the better solution.


-- 
Arno Welzel
http://arnowelzel.de
http://de-rec-fahrrad.de

Back to comp.lang.php | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

I Need to search over 100 largeish  text documents efficiently. What's the best approach? rob.bradford2805@gmail.com - 2014-01-26 05:34 -0800
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Richard Damon <Richard@Damon-Family.org> - 2014-01-26 09:09 -0500
    Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Michael Vilain <vilain@NOspamcop.net> - 2014-01-26 13:34 -0800
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Jerry Stuckle <jstucklex@attglobal.net> - 2014-01-26 10:29 -0500
    Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Denis McMahon <denismfmcmahon@gmail.com> - 2014-01-26 20:14 +0000
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? The Natural Philosopher <tnp@invalid.invalid> - 2014-01-26 19:56 +0000
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-26 20:55 +0000
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Denis McMahon <denismfmcmahon@gmail.com> - 2014-01-27 01:43 +0000
    Re: I Need to search over 100 largeish  text documents efficiently.   What's the best approach? Arno Welzel <usenet@arnowelzel.de> - 2014-01-27 10:58 +0100
      Re: I Need to search over 100 largeish  text documents efficiently.   What's the best approach? Denis McMahon <denismfmcmahon@gmail.com> - 2014-01-27 12:23 +0000
      Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-01-27 17:05 +0000
  Re: I Need to search over 100 largeish  text documents efficiently. What's the best approach? rob.bradford2805@gmail.com - 2014-01-27 13:14 -0800

csiph-web