Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #18848

Re: poor performance while processing a file one byte a time

Show all headers | View raw

On Thu, 20 Jan 2022 11:08:23 -0500
John-Paul Stewart <jpstewart@personalprojects.net> wrote:

> > function fn($fname) {
> >   $fd = fopen($fname, 'rb');
> >   if ($fd === false) return(0);
> > 
> >   $result = 0;
> > 
> >   while (!feof($fd)) {
> > 
> >     $buff = fread($fd, 1024 * 1024);
> > 
> >     foreach (str_split($buff) as $b) {
> >       $result += ord($b);
> >       $result &= 0xffff;
> >     }
> >   }
> > 
> >   fclose($fd);
> >   return($result);
> > }
> 
> Your foreach(str_split) line is an obvious place to start.
> str_split() creates an array from a string.  In your case, the input
> buffer is 1024*1024 bytes long, so you're splitting that megabyte
> string and (re-)creating an array of more than a million elements for
> _each iteration of the loop_.  (Which will be a million+ times.)

Are you really sure about that? My understanding is that the foreach()
argument is processed only once... Isn't that the case?

> Why?  The very first thing you should consider is pulling that out
> and doing it just once:
> 
> $buff = fread($fd, 1024 * 1024);
> $whatever = str_split($buff);
> foreach ($whatever as $b)

Initially I had such version of the code, but it was throwing such
error on the str_split() line:

"PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted
(tried to allocate 4096 bytes)"

I reduced the fread() buffer to 64K and the error went away. But the
speed is still the same. Code is this:

  while (!feof($fd)) {

    $buffstr = fread($fd, 64 * 1024);
    $buffarr = str_split($buffstr);

    foreach ($buffarr as $b) {
      $result += ord($b);
      $result &= 0xffff;
    }
  }

> (There's nothing specific to PHP about that advice either.  It's
> equally applicable to C, although modern C compilers _may_ make that
> optimization for you.)

That's hardly applicable in C, since there is no "foreach" in C. There
is a "for", and it's initialization argument is processed exactly once.


Mateusz

Back to comp.lang.php | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 15:16 +0100
  Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 11:08 -0500
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 17:23 +0100
      Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 12:38 -0500
        Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:49 +0100
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 17:37 +0100
      Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 12:41 -0500
        Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:47 +0100
  Re: poor performance while processing a file one byte a time Arno Welzel <usenet@arnowelzel.de> - 2022-01-20 19:23 +0100
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:45 +0100
      Re: poor performance while processing a file one byte a time Arno Welzel <usenet@arnowelzel.de> - 2022-01-20 20:27 +0100

csiph-web