Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.php > #9702

Re: Exhaustive memory allocation using arrays

Show all headers | View raw

tombert.at@gmail.com wrote:
^^^^^^^^^^^^^^^^^^^^
Please get a real name.

> Why is the following code using > 100MB?
> 
> for ($i=0; $i < 1000000; $i++) {
>     $value[$i] = 23.4567890123;
> }
> echo (memory_get_usage(TRUE) / (1024 * 1024)) . "MB <br>";
> 
> It looks like that each item takes 100 Byte for allocation??

First of all, 1 MB (megabyte) is 1'000'000 (one million, 10⁶) bytes.  What 
you call “MB” here actually is 1 _MiB_ (mega-binary byte; short: mebibyte) 
or 2²⁰ bytes = 1'048'576 bytes (as you divided by 2¹⁰ × 2¹⁰ = 2²⁰ to arrive 
at your “MB” value):

<http://en.wikipedia.org/wiki/Mebibyte>

So your calculation

  100 “MB” ∕ 1'000'000 items = 100 bytes∕item

is wrong to begin with.  Assuming that you got something close to 100 as a 
result of

  memory_get_usage(TRUE) / (1024 * 1024)

each item would take about

  104'857'600 bytes / 1'000'000 = 105 bytes

<http://php.net/memory_get_usage>

Second, the value you assigned is a value of the float type.  In PHP, the 
precision of the float type, thus the number of bits required to store such 
a value, is platform-dependent:

<http://php.net/manual/en/language.types.float.php>

The manual says that IEEE-754 “double precision” floating-point values are a 
common implementation (as they are in other equally dynamic programming 
languages), but we cannot assume that to be the case on your platform.  And 
you have not said which platform and PHP version you are using, so we cannot 
look into the pertinent source code.

Third, you are not just storing the float value.  You are storing a key-
value relationship:

,-<http://www.php.net/manual/en/language.types.array.php>
| 
| An array in PHP is actually an ordered map. A map is a type that
| associates values to keys.  […]

Fourth, that PHP arrays can be associative, i. e. can have string keys, and 
that using the string representation of a numeric key accesses the same item 
as the number (ibid.), suggests implementations in which all keys are 
actually string values (as it is in other equally dynamic programming 
languages, like ECMAScript implementations; if not like that, you would have 
to store the type of the key which is even more memory-expensive), and

,-<http://www.php.net/manual/en/language.types.string.php>
| 
| The string in PHP is implemented as an array of bytes and an integer
| indicating the length of the buffer.  […]

Fast access to array items further suggests that the keys are not only 
stored as-is (for array_keys()), but that the implementation includes a hash 
table.

Fifth, PHP arrays do have an internal pointer – <http://php.net/current> – 
that can be advanced: <http://php.net/next>.  Combined with the dynamic size 
of PHP arrays this suggests that the items of an array are at least arranged 
in a linked list.  The existence of a function to rewind the internal 
pointer – <http://php.net/prev> – further suggests that they are arranged in 
a double-linked list.

Finally, you are not just measuring the memory that is needed for the array, 
but for executing the preceding program.  There is *at least* the memory for 
$i variable to consider.  So “your math does not add up”.

Probably there are even more factors to consider.  See also the 
memory_get_usage() documentation and user comments.

So consider this: A PHP array could be implemented *at least* as a hash 
table (for quick item access) in which the string keys are stored in a 
linked list (so that array_keys() works), and the stored values could be in 
a double-linked list (so that internal pointer movements work).  In this 
hash table you are storing items whose plain values require *at least* 64 
bits = 8 bytes of memory each.  Add to that the memory required to run the 
program that fills the array.

I know of no native way to determine the memory it takes to store the PHP 
array; it can perhaps be obtained with a PHP profiler.  In any case, I 
presume it will be smaller than what you measured with memory_get_usage().


HTH

PointedEars
-- 
var bugRiddenCrashPronePieceOfJunk = (
    navigator.userAgent.indexOf('MSIE 5') != -1
    && navigator.userAgent.indexOf('Mac') != -1
)  // Plone, register_function.js:16

Back to comp.lang.php | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Exhaustive memory allocation using arrays tombert.at@gmail.com - 2013-05-28 23:27 -0700
  Re: Exhaustive memory allocation using arrays tombert.at@gmail.com - 2013-05-28 23:42 -0700
  Re: Exhaustive memory allocation using arrays tombert.at@gmail.com - 2013-05-29 00:45 -0700
    Re: Exhaustive memory allocation using arrays Christoph Becker <cmbecker69@gmx.de> - 2013-05-29 13:00 +0200
  Re: Exhaustive memory allocation using arrays Denis McMahon <denismfmcmahon@gmail.com> - 2013-05-29 08:22 +0000
    Re: Exhaustive memory allocation using arrays The Natural Philosopher <tnp@invalid.invalid> - 2013-05-29 09:54 +0100
  Re: Exhaustive memory allocation using arrays Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2013-05-29 15:30 +0200

csiph-web