Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.development.apps > #634 > unrolled thread

Re: memory use

Back to article view | Back to comp.os.linux.development.apps

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: memory use Grant Edwards <invalid@invalid.invalid> - 2011-02-02 19:42 +0000
    Re: memory use Grant Edwards <invalid@invalid.invalid> - 2011-02-02 21:16 +0000
    Re: memory use Bob Tennent <BobT@cs.queensu.ca> - 2011-02-02 20:22 +0000

#634 — Re: memory use

On 2011-02-02, Bob Tennent <BobT@cs.queensu.ca> wrote:

> A C program called musixflx is used as the second pass of a
> three-pass TeX-based system called musixtex.  The program was the
> first C program written by a Fortran programmer.  It has definitions
> as follows:
>
>   # define MAX_BARS    2048  /* max number of bars */
>
>   int zbar[MAX_BARS], lr_repeat[MAX_BARS], raggedline[MAX_BARS],
>       l_repeat[MAX_BARS], barno[MAX_BARS], ...
>
>   double hardbarlength[MAX_BARS], softbarlength[MAX_BARS],
>          width_leftrightrepeat[MAX_BARS], width_leftrepeat[MAX_BARS],
>          ...
>
> Typically the number of bars in a piece is much smaller than MAX_BARS.
>
> Would it be best to
>
>  + leave the code alone and let the virtual-memory functions of the
>    operating system manage the memory
>
>  + use "extensible" arrays (i.e., use realloc to copy small arrays to
>    larger ones when necessary)
>
> And would there be any benefit to using a single array of structures
> instead of many parallel arrays?

You're going to have to define "best" and "benefit" before anybody can
attempt to answer your question.  Is the current design causing some
sort of problem for you?  If so, what is that problem?

-- 
Grant Edwards               grant.b.edwards        Yow! In Newark the
                                  at               laundromats are open 24
                              gmail.com            hours a day!

[toc] | [next] | [standalone]

#635

On 2011-02-02, Bob Tennent <BobT@cs.queensu.ca> wrote:
>On Wed, 2 Feb 2011 19:42:08 +0000 (UTC), Grant Edwards wrote:
>> On 2011-02-02, Bob Tennent <BobT@cs.queensu.ca> wrote:
>>
>>> A C program called musixflx is used as the second pass of a
>>> three-pass TeX-based system called musixtex.  The program was the
>>> first C program written by a Fortran programmer.  It has definitions
>>> as follows:
>>>
>>>   # define MAX_BARS    2048  /* max number of bars */
>>>
>>>   int zbar[MAX_BARS], lr_repeat[MAX_BARS], raggedline[MAX_BARS],
>>>       l_repeat[MAX_BARS], barno[MAX_BARS], ...
>>>
>>>   double hardbarlength[MAX_BARS], softbarlength[MAX_BARS],
>>>          width_leftrightrepeat[MAX_BARS], width_leftrepeat[MAX_BARS],
>>>          ...
>>>
>>> Typically the number of bars in a piece is much smaller than MAX_BARS.
>>>
>>> Would it be best to
>>>
>>>  + leave the code alone and let the virtual-memory functions of the
>>>    operating system manage the memory
>>>
>>>  + use "extensible" arrays (i.e., use realloc to copy small arrays to
>>>    larger ones when necessary)
>>>
>>> And would there be any benefit to using a single array of structures
>>> instead of many parallel arrays?
>>
>> You're going to have to define "best" and "benefit" before anybody can
>> attempt to answer your question.  Is the current design causing some
>> sort of problem for you?  If so, what is that problem?
>
> Naively, it looks like the program uses a huge amount of memory

Huge?

The declarations you showed declare a total of 120KB of memory. That's
"KB" not "MB".  That's less than 0.01% of RAM on a machine that has 1GB.
What's your target enviroment like?

> and it's conceivable that extensible arrays would avoid this. But I
> suspect virtual memory will keep unused array space out of RAM.

That depends on access patterns, but in general converting code like
that into an array of structs will provide more access locality and
better page utilization.  The existing code spreads the accesses out
over a more pages compared to an array of structs.

> so there's no significant performance penalty in the existing code.  
>
> But even if the above is true, it may be that one big array of
> structs will have less overhead than many parallel arrays.

The storage requirements will be the same.  What sort of overhead are
you concerned about?

If the problem you're trying to solve is that the code is hard to read
and maintain, then go head and re-write it.  If you're worring about
reducing memory usage from 120K to 60K, then you're probably wasting
your time.

OTOH, if the program has a RSS of hundreds of megabytes, then you've
gotsome thing to work on.  What does "top" tell you about the program
while it's running?

-- 
Grant Edwards               grant.b.edwards        Yow! It's a hole all the
                                  at               way to downtown Burbank!
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#639

On Wed, 2 Feb 2011 19:42:08 +0000 (UTC), Grant Edwards wrote:
 > On 2011-02-02, Bob Tennent <BobT@cs.queensu.ca> wrote:
 >
 >> A C program called musixflx is used as the second pass of a
 >> three-pass TeX-based system called musixtex.  The program was the
 >> first C program written by a Fortran programmer.  It has definitions
 >> as follows:
 >>
 >>   # define MAX_BARS    2048  /* max number of bars */
 >>
 >>   int zbar[MAX_BARS], lr_repeat[MAX_BARS], raggedline[MAX_BARS],
 >>       l_repeat[MAX_BARS], barno[MAX_BARS], ...
 >>
 >>   double hardbarlength[MAX_BARS], softbarlength[MAX_BARS],
 >>          width_leftrightrepeat[MAX_BARS], width_leftrepeat[MAX_BARS],
 >>          ...
 >>
 >> Typically the number of bars in a piece is much smaller than MAX_BARS.
 >>
 >> Would it be best to
 >>
 >>  + leave the code alone and let the virtual-memory functions of the
 >>    operating system manage the memory
 >>
 >>  + use "extensible" arrays (i.e., use realloc to copy small arrays to
 >>    larger ones when necessary)
 >>
 >> And would there be any benefit to using a single array of structures
 >> instead of many parallel arrays?
 >
 > You're going to have to define "best" and "benefit" before anybody can
 > attempt to answer your question.  Is the current design causing some
 > sort of problem for you?  If so, what is that problem?

Naively, it looks like the program uses a huge amount of memory and
it's conceivable that extensible arrays would avoid this. But I suspect
virtual memory will keep unused array space out of RAM so there's no
significant performance penalty in the existing code.  

But even if the above is true, it may be that one big array of structs
will have less overhead than many parallel arrays.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.development.apps

csiph-web