Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #18852

Re: poor performance while processing a file one byte a time

Show all headers | View raw

Mateusz Viste:

> Hello,
> 
> I am processing some files using php. Basically I read every byte of
> the file and perform a simple operation on it to compute a sum.
> 
> My initial implementation was in C, but now I am trying re-doing the
> same in PHP. This is how my PHP code looks like:
> 
> 
> function fn($fname) {
>   $fd = fopen($fname, 'rb');
>   if ($fd === false) return(0);
> 
>   $result = 0;
> 
>   while (!feof($fd)) {
> 
>     $buff = fread($fd, 1024 * 1024);
> 
>     foreach (str_split($buff) as $b) {
>       $result += ord($b);
>       $result &= 0xffff;
>     }
>   }
> 
>   fclose($fd);
>   return($result);
> }
> 
> It works, but it is really slow (approximately 100x slower than the
> original C code). I know that I should not expect much performance
> from interpreted PHP code, but still - is there any trick I could use to
> speed this up?

By *not* using arrays.

str_split() creates an array based on a string:
<https://www.php.net/manual/en/function.str-split.php>

> I have also tried to replace str_split() and ord() with unpack('C*'),
> but it was even slower. Anything else I could try?

Just use the string as it is:

function fn($fname) {
  $fd = fopen($fname, 'rb');
  if ($fd === false) return(0);

  $result = 0;

  while (!feof($fd)) {

    $buff = fread($fd, 1024 * 1024);

    $len = strlen($buff);
    $pos = 0;
    while ($pos < $len) {
      $result += ord($buff[$pos]);
      $result &= 0xffff;
      $pos++;
    }
  }

  fclose($fd);
  return($result);
}

However this may still be quite slow on larger files. Since it seems you
want to create some kind of checksum based on the file content,  you may
want to use something else like hash_file(), sha1_file() or md5_file()
and use the result of these calls instead processing the whole file
content in a loop.


-- 
Arno Welzel
https://arnowelzel.de

Back to comp.lang.php | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 15:16 +0100
  Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 11:08 -0500
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 17:23 +0100
      Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 12:38 -0500
        Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:49 +0100
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 17:37 +0100
      Re: poor performance while processing a file one byte a time John-Paul Stewart <jpstewart@personalprojects.net> - 2022-01-20 12:41 -0500
        Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:47 +0100
  Re: poor performance while processing a file one byte a time Arno Welzel <usenet@arnowelzel.de> - 2022-01-20 19:23 +0100
    Re: poor performance while processing a file one byte a time Mateusz Viste <mateusz@xyz.invalid> - 2022-01-20 19:45 +0100
      Re: poor performance while processing a file one byte a time Arno Welzel <usenet@arnowelzel.de> - 2022-01-20 20:27 +0100

csiph-web