Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #58631 > unrolled thread

local arrays and stack

Back to article view | Back to comp.lang.c

Contents

  local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-22 01:51 -0700
    local arrays and stack asetofsymbols@gmail.com - 2015-03-22 02:23 -0700
    Re: local arrays and stack Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-03-22 13:22 -0700
      Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 01:55 -0700
    Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-22 15:07 -0700
      Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 01:50 -0700
        Re: local arrays and stack Robert Wessel <robertwessel2@yahoo.com> - 2015-03-23 04:26 -0500
          Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 06:53 -0700
          Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 07:29 -0700
            Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 08:34 -0700
            Re: local arrays and stack Robert Wessel <robertwessel2@yahoo.com> - 2015-03-23 19:57 -0500
            Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-24 04:01 -0700
              Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-24 04:37 -0700
              Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 13:04 +0100
                Re: local arrays and stack glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-03-24 17:29 +0000
                  Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 20:01 +0100
          Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-23 08:24 -0700
            Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 08:42 -0700
              Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-23 16:07 +0000
                Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 09:15 -0700
                Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-23 09:16 -0700
                  Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-23 16:34 +0000
                    Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 10:12 -0700
                      Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-23 17:44 +0000
                        Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 11:47 -0700
                        Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 12:06 -0700
                          Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-23 20:25 +0000
                            Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 14:10 -0700
                              Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-23 22:22 +0000
                                Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 23:57 -0700
                                Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 10:28 +0100
                                  Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-24 10:44 +0000
                                    Re: local arrays and stack Martin Shobe <martin.shobe@yahoo.com> - 2015-03-24 06:38 -0500
                                      Re: local arrays and stack Martin Shobe <martin.shobe@yahoo.com> - 2015-03-24 09:57 -0500
                                    Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 13:22 +0100
                                      Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-25 16:07 +0000
                                        Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-25 17:42 +0100
                                          Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-25 19:06 +0000
                                            Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-25 12:26 -0700
                                              Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-25 13:20 -0700
                                            Re: local arrays and stack Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-03-25 12:28 -0700
                                              Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-25 21:45 +0100
                                            Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-25 12:50 -0700
                                              Re: local arrays and stack Bartc <bc@freeuk.com> - 2015-03-25 22:33 +0000
                                                Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-25 23:57 +0100
                                            Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-25 21:42 +0100
                                        Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-25 09:55 -0700
                                        Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-25 12:36 -0700
                                          Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-25 13:04 -0700
                                          Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-25 21:52 +0100
                                            Re: local arrays and stack Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-03-25 15:18 -0700
                                              Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-26 00:02 +0100
                                              Re: local arrays and stack Melzzzzz <mel@zzzzz.com> - 2015-03-26 00:47 +0100
                                    Re: local arrays and stack Mel <mel@zzzzz.com> - 2015-03-24 14:58 +0100
                                Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-24 06:58 -0700
                            Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-23 14:34 -0700
                            Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 10:16 +0100
                        Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 09:55 +0100
                        Re: local arrays and stack Noob <root@127.0.0.1> - 2015-03-24 10:23 +0100
                  Re: local arrays and stack Tim Rentsch <txr@alumni.caltech.edu> - 2015-03-24 12:18 -0700
                    Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-24 13:42 -0700
                      Re: local arrays and stack Tim Rentsch <txr@alumni.caltech.edu> - 2015-03-25 08:47 -0700
              Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 09:32 -0700
                Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-23 09:57 -0700
                Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-23 10:37 -0700
                  Re: local arrays and stack Xcott Craver <xcottcraver@gmail.com> - 2015-03-23 10:49 -0700
                    Re: local arrays and stack Keith Thompson <kst-u@mib.org> - 2015-03-23 12:48 -0700
                      Re: local arrays and stack jt@toerring.de (Jens Thoms Toerring) - 2015-03-23 22:29 +0000
                        Re: local arrays and stack James Kuyper <jameskuyper@verizon.net> - 2015-03-23 19:03 -0400
                          Re: local arrays and stack jt@toerring.de (Jens Thoms Toerring) - 2015-03-23 23:36 +0000
                          Re: local arrays and stack fir <profesor.fir@gmail.com> - 2015-03-24 02:17 -0700
                  Re: local arrays and stack Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-03-23 11:56 -0700
            Re: local arrays and stack Robert Wessel <robertwessel2@yahoo.com> - 2015-03-23 20:00 -0500
              Re: local arrays and stack David Brown <david.brown@hesbynett.no> - 2015-03-24 10:39 +0100

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#58949

On Wednesday, March 25, 2015 at 7:06:51 PM UTC, Bart wrote:
>
> (In fact, I didn't realise until this thread that VLAs were so unsafe. 
> You know about the usually remote risks of stack overflow when some 
> recursive algorithm goes out of control, but with VLAs you don't even 
> need to call one function, just declaring an array in the middle of 
> main() could be enough to bring down an application!)
> 
It's quite common to have a bug which means that recursion never 
terminates. You want the program to crash as soon as it has gobbled
a reasonable chunk of memory, not hog system resources and start
paging itself out to disk.
VLAs are worse because normally recursive algorithms split each time,
so you need 2^N inputs to make them crash. If you've got about 50
levels to play with, which even a small computer will usually handle,
that's impossible to provide. With a VLA you only need N and the
stack is broken.

[toc] | [prev] | [next] | [standalone]

#58962

On 25/03/15 20:28, Malcolm McLean wrote:
> On Wednesday, March 25, 2015 at 7:06:51 PM UTC, Bart wrote:
>>
>> (In fact, I didn't realise until this thread that VLAs were so unsafe.
>> You know about the usually remote risks of stack overflow when some
>> recursive algorithm goes out of control, but with VLAs you don't even
>> need to call one function, just declaring an array in the middle of
>> main() could be enough to bring down an application!)
>>
> It's quite common to have a bug which means that recursion never
> terminates. You want the program to crash as soon as it has gobbled
> a reasonable chunk of memory, not hog system resources and start
> paging itself out to disk.
> VLAs are worse because normally recursive algorithms split each time,
> so you need 2^N inputs to make them crash. If you've got about 50
> levels to play with, which even a small computer will usually handle,
> that's impossible to provide. With a VLA you only need N and the
> stack is broken.
>

That's a complicated way of saying that uncontrolled recursion will 
crash faster if you allocate more data on the stack at each step.  While 
true, it is not exactly a revelation.

[toc] | [prev] | [next] | [standalone]

#58953

Bartc <bc@freeuk.com> writes:
[...]
> (In fact, I didn't realise until this thread that VLAs were so
> unsafe. You know about the usually remote risks of stack overflow when
> some recursive algorithm goes out of control, but with VLAs you don't
> even need to call one function, just declaring an array in the middle
> of main() could be enough to bring down an application!)

Just declaring a fixed-size array in the middle of main() could do
exactly the same thing.  The only thing VLAs add is the ability to
determine the size at run time.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#58973

On 25/03/2015 19:50, Keith Thompson wrote:
> Bartc <bc@freeuk.com> writes:
> [...]
>> (In fact, I didn't realise until this thread that VLAs were so
>> unsafe. You know about the usually remote risks of stack overflow when
>> some recursive algorithm goes out of control, but with VLAs you don't
>> even need to call one function, just declaring an array in the middle
>> of main() could be enough to bring down an application!)
>
> Just declaring a fixed-size array in the middle of main() could do
> exactly the same thing.  The only thing VLAs add is the ability to
> determine the size at run time.

A massive fixed-size array is more likely to be picked up early on as a 
problem by you and/or the compiler.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#58977

On 25/03/15 23:33, Bartc wrote:
> On 25/03/2015 19:50, Keith Thompson wrote:
>> Bartc <bc@freeuk.com> writes:
>> [...]
>>> (In fact, I didn't realise until this thread that VLAs were so
>>> unsafe. You know about the usually remote risks of stack overflow when
>>> some recursive algorithm goes out of control, but with VLAs you don't
>>> even need to call one function, just declaring an array in the middle
>>> of main() could be enough to bring down an application!)
>>
>> Just declaring a fixed-size array in the middle of main() could do
>> exactly the same thing.  The only thing VLAs add is the ability to
>> determine the size at run time.
>
> A massive fixed-size array is more likely to be picked up early on as a
> problem by you and/or the compiler.
>

Except in embedded systems with specific memory limitations, you'll 
probably find that the compiler will be quite happy with any fixed sized 
arrays, whether they are on the stack, heap or statically allocated. 
It's only when running that you will find the problem.  (Admittedly for 
statically allocated arrays you might see the problem as soon as the 
program tries to start.)

[toc] | [prev] | [next] | [standalone]

#58960

On 25/03/15 20:06, Bartc wrote:
> On 25/03/2015 16:42, David Brown wrote:
>> On 25/03/15 17:07, Bartc wrote:
>
>>> Suppose a function has input consisting two strings of given lengths M
>>> and N, and it needs to concatenate them into a composite work string of
>>> length M+N before any further processing, then that composite string is
>>> discarded.
>>>
>>> It sounds very appealing to just define:
>>>
>>>   char workstring[M+N+1];
>>>
>>> except that M could be a million or a billion.
>
>> A fancy compiler that puts local data on the heap will not protect you
>> from bad software design!
>
> What bad software design? It is declaring 'int array[N];' without being
> pretty certain that N is within reasonable range, that is bad design.

/Exactly/ the point - doing /anything/ with /any/ data is bad design if 
you require certain properties but don't know that these hold.  If a 
stack size of 8 MB is a reasonable limit for your target systems, then 
it is bad design to allocate an array on the stack that could take you 
over that limit.  But that is no different from any other checks or 
limits on incoming data - and a compiler will not make your program 
safe, reliable, or well-designed by deciding to put the array on the 
heap instead of the stack.

>
> (In fact, I didn't realise until this thread that VLAs were so unsafe.
> You know about the usually remote risks of stack overflow when some
> recursive algorithm goes out of control, but with VLAs you don't even
> need to call one function, just declaring an array in the middle of
> main() could be enough to bring down an application!)
>

VLA's are not unsafe.

Allocating an unreasonable amount of data (from any storage) is unsafe. 
  Working with unknown quantities without checking them is unsafe. 
Overflowing buffers is unsafe.  Undefined behaviour is unsafe.  In 
general, writing bad code is unsafe.

But using a tool such as VLA's on the stack, or fixed-size arrays on the 
stack, is perfectly safe as long as you use them properly.  And since 
stack-allocated arrays will be deallocated automatically, you could 
argue that they are less error-prone than heap allocated data as there 
is no risk of leaks.

[toc] | [prev] | [next] | [standalone]

#58927

W dniu środa, 25 marca 2015 17:07:39 UTC+1 użytkownik Bart napisał:
> On 24/03/2015 12:22, David Brown wrote:
> > On 24/03/15 11:44, Bartc wrote:
> 
> >> Suddenly, heap-allocated VLAs don't seem such a bad option!
> >
> > Yes, they /are/ a bad option unless they are what the /user/ asks for in
> > their source code.
> >
> > Of course, it is generally considered good programming practice to
> > allocate large objects on the heap rather than the stack, but it should
> > be the programmer who decides.
> 
> No. The programmer just writes this:
> 
> int array[N];
> 
> without needing to know where the space is going to come from. That 
> might be from the main hardware stack, from the main heap, or some 
> combination as I suggest below.
> 
> >> And because of the problem of crashing, VLAs can't be too big. In which
> >> case, just allocate a generous /fixed/ array.
> >
> > VLAs do not take any more or less space than a fixed-size array of the
> > same size, and they go in the same area (stack, heap, or static memory
> > according to how they are defined in the source code).
> 
> With a fixed-size array imposing an upper limit, the behaviour of a bit 
> of code is going to be more predictable. But you will have to think 
> about that happens if you actually need a considerably larger size.
> 
> Suppose a function has input consisting two strings of given lengths M 
> and N, and it needs to concatenate them into a composite work string of 
> length M+N before any further processing, then that composite string is 
> discarded.
> 
> It sounds very appealing to just define:
> 
>   char workstring[M+N+1];
> 
> except that M could be a million or a billion. (For this, I might use a 
> 256- or 1024-char workstring when M+N is small enough, and use the heap 
> for bigger strings. Most strings are going to be small. But the 
> occasional massive one won't now crash the application. Well, unless 
> it's really massive, but typically heap sizes are far bigger than stack 
> sizes. Even then, there will be an opportunity to fail gracefully, or 
> just to return an error code and continue.)
> 
> >> It's one C feature I wouldn't bother with in a parallel language (and I
> >> haven't).
> >>
> >
> > I can't say I have had any real use of VLA's either.  C++ does not
> > support them, and I don't think anyone feels the lack very strongly.
> 
> I didn't know that C++ doesn't do C-style VLAs. However C++ allows you 
> to apply custom construct/destruct handlers to certain kinds of objects, 
> which presumably can also include allocating and deallocating space from 
> a heap.
> 
> In other words, just the sort of scheme I stated *could* be used to do C 
> VLAs too, ie. the ones that no-one seems to use but go to some pains to 
> point out how easy and efficient they would be if somebody did use them!
> 
> And anyway the choice needn't simply be between putting a flat, packed 
> array directly on the stack, or allocating it using malloc(). If the 
> pattern of allocation is a nested style as it is for VLAs, then there's 
> no reason why a fast stack-like allocator can't be used, implemented on 
> heap memory. Then some of the inefficiences of malloc/free can be 
> eliminated.
> 
> But it wouldn't be /quite/ as crude as direct stack VLAs, because you 
> will want to take advantage of the better out-of-memory handling that is 
> now possible, and the easier possibility of expanding the available 
> space as needed.
> 
from all those three options static areas, stack (or something close) and heap, heap is like most crude/less cleaver

though can be considered as an option, why not (you could use it right now for local storage this is salled malloc and free

autmatic support can be also considered (as the one with keyword heap as i was givin example) - got no terrible 
step up regerding to explicitely putting malloc free but "nice
for completion" of 'modes' of storage imo


as a c spirit man i am not giving a very big regard to heap as heep imo has something like too narriw interface and controll, (same like c file api)  something more low level would be better imo - windows has more of it though i was nat yet ready to learn it yet ;o

(all this above is obvious)

[toc] | [prev] | [next] | [standalone]

#58951

Bartc <bc@freeuk.com> writes:
> On 24/03/2015 12:22, David Brown wrote:
>> On 24/03/15 11:44, Bartc wrote:
>
>>> Suddenly, heap-allocated VLAs don't seem such a bad option!
>>
>> Yes, they /are/ a bad option unless they are what the /user/ asks for in
>> their source code.
>>
>> Of course, it is generally considered good programming practice to
>> allocate large objects on the heap rather than the stack, but it should
>> be the programmer who decides.
>
> No. The programmer just writes this:
>
> int array[N];
>
> without needing to know where the space is going to come from. That
> might be from the main hardware stack, from the main heap, or some
> combination as I suggest below.

If you're describing what programmers could do in an ideal environment,
I won't disagree.

If you're attempting to describe current reality, you've missed.
Current C implementations (at least the ones I'm familiar with) place
locally declared non-static array objects on the stack, which is
typically limited in size.  If you need a very large array object, you
need to define it as static (and live with the fact that its lifetime is
the entire execution of the program), or you need to allocate it with
malloc or equivalent and manage its lifetime yourself.

This *is* "generally considered good programming practice".

[...]

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#58955

W dniu środa, 25 marca 2015 20:36:46 UTC+1 użytkownik Keith Thompson napisał:
> Bartc <bc@freeuk.com> writes:
> > On 24/03/2015 12:22, David Brown wrote:
> >> On 24/03/15 11:44, Bartc wrote:
> >
> >>> Suddenly, heap-allocated VLAs don't seem such a bad option!
> >>
> >> Yes, they /are/ a bad option unless they are what the /user/ asks for in
> >> their source code.
> >>
> >> Of course, it is generally considered good programming practice to
> >> allocate large objects on the heap rather than the stack, but it should
> >> be the programmer who decides.
> >
> > No. The programmer just writes this:
> >
> > int array[N];
> >
> > without needing to know where the space is going to come from. That
> > might be from the main hardware stack, from the main heap, or some
> > combination as I suggest below.
> 
> If you're describing what programmers could do in an ideal environment,
> I won't disagree.
> 
> If you're attempting to describe current reality, you've missed.
> Current C implementations (at least the ones I'm familiar with) place
> locally declared non-static array objects on the stack, which is
> typically limited in size.  If you need a very large array object, you
> need to define it as static (and live with the fact that its lifetime is
> the entire execution of the program), or you need to allocate it with
> malloc or equivalent and manage its lifetime yourself.
> 
> This *is* "generally considered good programming practice".
> 
> [...]
> 
aiuc not "very large" but larger then stack
(if stack is default 1MB (or 2MB, a bit confussion some said that it is 1 MB on win32 but it seems to be 2 maybe)  then this is micro stack, far below the average array buffers i use)

[toc] | [prev] | [next] | [standalone]

#58964

On 25/03/15 20:36, Keith Thompson wrote:
> Bartc <bc@freeuk.com> writes:
>> On 24/03/2015 12:22, David Brown wrote:
>>> On 24/03/15 11:44, Bartc wrote:
>>
>>>> Suddenly, heap-allocated VLAs don't seem such a bad option!
>>>
>>> Yes, they /are/ a bad option unless they are what the /user/ asks for in
>>> their source code.
>>>
>>> Of course, it is generally considered good programming practice to
>>> allocate large objects on the heap rather than the stack, but it should
>>> be the programmer who decides.
>>
>> No. The programmer just writes this:
>>
>> int array[N];
>>
>> without needing to know where the space is going to come from. That
>> might be from the main hardware stack, from the main heap, or some
>> combination as I suggest below.
>
> If you're describing what programmers could do in an ideal environment,
> I won't disagree.
>
> If you're attempting to describe current reality, you've missed.
> Current C implementations (at least the ones I'm familiar with) place
> locally declared non-static array objects on the stack, which is
> typically limited in size.  If you need a very large array object, you
> need to define it as static (and live with the fact that its lifetime is
> the entire execution of the program), or you need to allocate it with
> malloc or equivalent and manage its lifetime yourself.
>
> This *is* "generally considered good programming practice".
>

Also note that for an unknown size N, allocating an "array[N]" object on 
the heap is just as bad practice as allocating it on the stack.  The 
heap /generally/ has more space than the stack (on many of my embedded 
systems, the heap is very small or completely non-existent - and on most 
others the heap and the stack are in the same memory block, starting at 
different ends).  But the heap is not unlimited - and allocating too big 
a block from the heap has more chance of disaster (causing system 
instability) than allocating too big a block from the stack (which will 
typically cause a segment fault).

Unless you are programming Turing machines, any allocation should have 
an idea of the size it is dealing with.

[toc] | [prev] | [next] | [standalone]

#58971

On Wednesday, March 25, 2015 at 8:52:26 PM UTC, David Brown wrote:
>
> Also note that for an unknown size N, allocating an "array[N]" object on 
> the heap is just as bad practice as allocating it on the stack.  
>
You check the return from malloc(). If null, you can't do the 
calculation.
However a lot of people write (it's even been described as the clc-
approved way)

EMPLOYEE *ptr = malloc(N * sizeof *ptr);

that of course will overflow for sufficiently large N, and may appear
to succeed.

Some bad operating systems / C compilers allow user code to allocate 
too much memory, causing problems elsewhere. User processes shouldn't
be allowed to grab resources that prevent housekeeping programs from
functioning, whether that be mouse, keyboard, screen, disk, processor,
memory or anything else.  

[toc] | [prev] | [next] | [standalone]

#58979

On 25/03/15 23:18, Malcolm McLean wrote:
> On Wednesday, March 25, 2015 at 8:52:26 PM UTC, David Brown wrote:
>>
>> Also note that for an unknown size N, allocating an "array[N]" object on
>> the heap is just as bad practice as allocating it on the stack.
>>
> You check the return from malloc(). If null, you can't do the
> calculation.
> However a lot of people write (it's even been described as the clc-
> approved way)
>
> EMPLOYEE *ptr = malloc(N * sizeof *ptr);
>
> that of course will overflow for sufficiently large N, and may appear
> to succeed.
>
> Some bad operating systems / C compilers allow user code to allocate
> too much memory, causing problems elsewhere. User processes shouldn't
> be allowed to grab resources that prevent housekeeping programs from
> functioning, whether that be mouse, keyboard, screen, disk, processor,
> memory or anything else.
>

It's a debatable issue how much memory is "too much" in such cases. 
Sometimes you really want the OS to give you more memory than you have - 
that's part of the reason for having swap space.  But overuse of memory 
introduces risks that everything else will slow to a crawl, and may make 
the system unstable or unusable.  The best an OS can do is impose limits 
that reduce the risk of problems.  (Thus you have "ulimit" in *nix, to 
make this configurable.)

Also, an OS might give the app all the memory it asks for, but not 
actually allocate pages of memory (or swap) until the data is accessed.

[toc] | [prev] | [next] | [standalone]

#58988

On Wed, 25 Mar 2015 15:18:21 -0700 (PDT)
Malcolm McLean <malcolm.mclean5@btinternet.com> wrote:

> On Wednesday, March 25, 2015 at 8:52:26 PM UTC, David Brown wrote:
> >
> > Also note that for an unknown size N, allocating an "array[N]"
> > object on the heap is just as bad practice as allocating it on the
> > stack.  
> >
> You check the return from malloc(). If null, you can't do the 
> calculation.
> However a lot of people write (it's even been described as the clc-
> approved way)
> 
> EMPLOYEE *ptr = malloc(N * sizeof *ptr);
> 
> that of course will overflow for sufficiently large N, and may appear
> to succeed.
> 
> Some bad operating systems / C compilers allow user code to allocate 
> too much memory, causing problems elsewhere.

Problem are GC's, not compilers. They mmap too much RAM initially...
Other thing is forking. Without overcommit one couldn't run
Java or anything that maps initially large amount of RAM. 

 User processes shouldn't
> be allowed to grab resources that prevent housekeeping programs from
> functioning, whether that be mouse, keyboard, screen, disk, processor,
> memory or anything else.  

Problem is that user processes nowadays allocate too much
in front...

> 

[toc] | [prev] | [next] | [standalone]

#58837

On Tue, 24 Mar 2015 10:44:10 +0000, Bartc <bc@freeuk.com> wrote:
> On 24/03/2015 09:28, David Brown wrote:
> I tried Linux, and yes that seemed to just directly increment the 
stack 
> pointer. However, trying to allocate something big, or lots of 
smaller 
> allocations in recursive functions, made it crash!
You can link with ld.gold which dynamicaly groes stack, that is if 
you build gcc with ld.gold. 
I have tested this with gccgo but it should grow
stack dynamically for gcc, too, I guess...

-- 
Press any key to continue or any other to quit 

[toc] | [prev] | [next] | [standalone]

#58836

On Monday, March 23, 2015 at 6:22:50 PM UTC-4, Bart wrote:

> (3) I said it wouldn't be that different from the clean-up needed when 
> you jump out of a complex set of nested VLAs

I'm missing the post where you said anything about a "complex set of 
nested VLAs."

I only saw this remark about a simple VLA in a loop:

| I don't know what gcc in particular does. But if allocating a VLA,
| perhaps inside a block which is the body of a loop, and the size is
| different each time, then wouldn't it have to take of such details
| anyway? Including dealing with goto, break, longjmp and return. 

And the answer to that question is "no."  If you allocate a local array on
the stack, you don't have to clean it up before a longjmp() as you would
if you allocated from the heap.  A call to longjmp() restores the stack 
as saved in setjmp(), which takes care of subsequent stack-allocated locals.

As for goto, break and return:  if you are leaving a scope with local 
variables reserved on the stack, the stack pointer must be amended.  
But this is true of any local variable, not just VLAs; and it is not a  
particularly complicated task either.  It is almost certainly simpler to 
alter a register than it is to make however many calls to free().

Elsewhere you write:

> But the data may be more complicated than it is in C: you can't just add
> something to a stack pointer; there may be nested pointers to other
> allocated data that need attending to first. 

That sounds like a data structures problem, unrelated to any issues about
how a local variable is allocated.  If you have a data structure that must
be processed before it is deallocated, then you perform that processing;
the deallocation of stack locals is still just a matter of updating the stack
pointer.

Perhaps I would understand your objections better if you provided an example
of a situation either with a complex set of nested VLAs, or where an object
contains "nested pointers" that would hinder its allocation as a stack
when declared as a local variable.

--S

[toc] | [prev] | [next] | [standalone]

#58788

Bartc <bc@freeuk.com> writes:
[...]
> But if a simple pointer is to be used, then just using malloc and free
> starts to sound attractive, if chkstk_ms() needs to be called anyway.

If we're counting CPU instructions, compare the number of instructions
executed to allocate a VLA on the stack vs. the number of instructions
executed for a call to malloc() followed by a call to free().

Figuring out how to generate code to allocate VLAs correctly might be a
little tricky -- but that work has already been done.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#58823

On 23/03/15 21:25, Bartc wrote:
> On 23/03/2015 19:06, Xcott Craver wrote:
>> On Monday, March 23, 2015 at 1:45:05 PM UTC-4, Bart wrote:
> 
>>> How? The VLA size is a runtime expression.
>>
>> Amending the stack pointer is a runtime operation.
> 
> But with a constant value, or a variable value? That makes a difference.

It makes a difference to the exact instructions used, but not the
principle.  You are either subtracting a compile-time constant from SP,
or a run-time calculated value.

> 
>> Maybe we're talking about different things here.  I'm imagining code like
>>
>> int func( int n ) {
>>     int i;
>>     for( i=0; i<30; i++ ) {
>>         int barray[i*i+1000*i+n];
>>         /** code to use barray[] **/
>>     }
>>     return 0;
>> }
>>
>> When I compile this and disassemble func(), I see instructions at
>> the top of the loop that decrement %rsp by an amount corresponding to
>> i*i+1000*i+n ints; and then instructions at the end of the loop that
>> reset %rsp to its previous value.  In the case of this specific function,
>> allocating and deallocating the array is really that simple.
> 
> Your system obviously is confident enough to push the stack by any
> amount without checking. When I compile with gcc for Windows, I get the
> following. Notice the lines marked saved_stack, and the call to
> ___chkstk_ms. (The 64-bit version is similar.)
> 

That looks like a Windows issue - you just have to live with it if
that's your target OS.  Whether or not ___chkstk_ms is a time-consuming
function is perhaps worth investigating - if it is, then you'll know
that using large local stack objects on Windows might be slower than
putting them on the heap.  On Linux, however, any issues about paging,
memory allocation, etc., are done behind the scenes when necessary, and
therefore stack allocated memory is very efficient.

When you compile test code, use -O1 (or perhaps -Os or -O2), since the
code is clearer than with optimisation disabled.

> 
>> Why would it be awkward to push the stack pointer itself?
> 
> It just is, although you can do it indirectly. (For example are you
> pushing rsp before or after it's been decremented?)

The cpu architecture will define the answer to that question, and the
compiler authors will know that answer.  Thus pushing (or popping) the
stack pointer is not a problem.  There is rarely any reason to do it,
since you almost always want a base pointer in cases when the stack
pointer would be stored, but it is perfectly possible.

> 
>> In fact, the stack pointer is routinely pushed on the stack, albeit a
>> little
>> bit indirectly:  at the start of a function it is customary to push the
>> current value of the base pointer, and then copy the stack pointer
>> into the
>> base pointer.
> 
> That's pushing a frame pointer which is very common. Although gcc seems
> to have done away with that, having everything as offsets from the stack
> pointer instead.

Most compilers skip the generation of full stack frames (with a base
pointer) when they don't need it, since the code is smaller and faster
without a frame pointer.  The advantage of frame pointers is that local
variables live at fixed (bp+offset) offsets, while their (sp+offset)
offset will change as the stack pointer moves up and down - less
advanced compilers, and in particular less advanced debuggers, found
(bp+offset) much easier to handle.  But modern compilers and modern
debuggers don't have a problem with changing offsets.

> 
>> My above code compiles to assembly that just saves the original value of
>> %rsp in a register, but there is no reason why it would be more
>> complicated
>> to save %rsp in a spare bit of stack itself.
> 
> Suppose you first push the stack pointer (ignoring the problem of when
> it is incremented), then allocate N bytes of a VLA.
> 
> The trouble is, that saved stack pointer is now the other side of that
> VLA! We would need to save N somewhere in order to access it. So we
> might as well just store N.
> 
> Or maybe the stack pointer is pushed after the VLA, but it is necessary
> to push the old value before the VLA is allocated.

That's why the base pointer register is used!  Put the old value of SP
into BP, then adjust SP to allocate space for N.  You now have easy
access to things on the stack before the allocation (via BP+offset),
things after the stack (via SP+offset), and N (via SP for a decreasing
stack, or BP for an increasing stack).  And as BP is considered a
"preserved" register in most x86 ABIs, it is stacked and restored just
like any other preserved register.

> 
> So it can be a little tricky. There is also the question of how to
> access that array: it's not known at compile-time what the offset will
> be, so it might be necessary to store that into a pointer, something
> else that needs to go onto the stack (but at a known fixed offset this
> time).
> 
> But if a simple pointer is to be used, then just using malloc and free
> starts to sound attractive, if chkstk_ms() needs to be called anyway.
> 

See above.

And for comparison to your code, this is compiled with gcc on 64-bit Linux:

extern void bar(int* x);

void foo100(void) {
    int ar[100];
    bar(ar);
}

void foo1000(void) {
    int ar[1000];
    bar(ar);
}

void foo1000000(void) {
    int ar[1000000];
    bar(ar);
}

void fooX(int x) {
    int ar[x];
    bar(ar);
}


   1              		.file	"a.c"
   2              		.text
   3              		.globl	foo100
   5              	foo100:
   6              	.LFB0:
   7              		.cfi_startproc
   8 0000 4881EC98 		subq	$408, %rsp
   8      010000
   9              		.cfi_def_cfa_offset 416
  10 0007 4889E7   		movq	%rsp, %rdi
  11 000a E8000000 		call	bar
  11      00
  12 000f 4881C498 		addq	$408, %rsp
  12      010000
  13              		.cfi_def_cfa_offset 8
  14 0016 C3       		ret
  15              		.cfi_endproc
  16              	.LFE0:
  18              		.globl	foo1000
  20              	foo1000:
  21              	.LFB1:
  22              		.cfi_startproc
  23 0017 4881ECA8 		subq	$4008, %rsp
  23      0F0000
  24              		.cfi_def_cfa_offset 4016
  25 001e 4889E7   		movq	%rsp, %rdi
  26 0021 E8000000 		call	bar
  26      00
  27 0026 4881C4A8 		addq	$4008, %rsp
  27      0F0000
  28              		.cfi_def_cfa_offset 8
  29 002d C3       		ret
  30              		.cfi_endproc
  31              	.LFE1:
  33              		.globl	foo1000000
  35              	foo1000000:
  36              	.LFB2:
  37              		.cfi_startproc
  38 002e 4881EC08 		subq	$4000008, %rsp
  38      093D00
  39              		.cfi_def_cfa_offset 4000016
  40 0035 4889E7   		movq	%rsp, %rdi
  41 0038 E8000000 		call	bar
  41      00
  42 003d 4881C408 		addq	$4000008, %rsp
  42      093D00
  43              		.cfi_def_cfa_offset 8
  44 0044 C3       		ret
  45              		.cfi_endproc
  46              	.LFE2:
  48              		.globl	fooX
  50              	fooX:
  51              	.LFB3:
  52              		.cfi_startproc
  53 0045 55       		pushq	%rbp
  54              		.cfi_def_cfa_offset 16
  55              		.cfi_offset 6, -16

GAS LISTING /tmp/ccp42rdO.s 			page 2


  56 0046 4889E5   		movq	%rsp, %rbp
  57              		.cfi_def_cfa_register 6
  58 0049 4863FF   		movslq	%edi, %rdi
  59 004c 488D04BD 		leaq	18(,%rdi,4), %rax
  59      12000000
  60 0054 4883E0F0 		andq	$-16, %rax
  61 0058 4829C4   		subq	%rax, %rsp
  62 005b 4889E7   		movq	%rsp, %rdi
  63 005e E8000000 		call	bar
  63      00
  64 0063 C9       		leave
  65              		.cfi_def_cfa 7, 8
  66 0064 C3       		ret
  67              		.cfi_endproc
  68              	.LFE3:
  70              		.ident	"GCC: (GNU) 4.9.1"
  71              		.section	.note.GNU-stack,"",@progbits

[toc] | [prev] | [next] | [standalone]

#58822

On 23/03/15 18:44, Bartc wrote:
> On 23/03/2015 17:12, Xcott Craver wrote:
>> On Monday, March 23, 2015 at 12:35:07 PM UTC-4, Bart wrote:
>>>
>>> I don't know what gcc in particular does. But if allocating a VLA,
>>> perhaps inside a block which is the body of a loop, and the size is
>>> different each time, then wouldn't it have to take of such details
>>> anyway? Including dealing with goto, break, longjmp and return.
>>
>> If doing this on the stack, allocating a VLA means decrementing the
>> stack pointer, and deallocating it means resetting the stack pointer
>> to a previous value.
> 
> How? The VLA size is a runtime expression. If it's a complicated one,
> then it would need to be stored somewhere (presumably also on the
> stack), as it would be awkward to push the stack pointer itself.

Just assume that the VLA is allocated with alloca() rather than malloc()
- with all the advantages and disadvantages that implies.  Ideally, this
means copying the SP to the BP register (so that the compiler can access
data already on the stack with BP+offset after SP changes), then change
SP to make enough space on the stack for the VLA.  Deallocation is done
by copying the BP back into the SP.  This is exactly the same procedure
as used whenever a function needs a full stack frame (rather than the
simplified versions typically used these days).

Jumping around inside the function (goto, break, etc.) or leaving the
function is handled just like normal.  Longjmp and setjmp are messy as
always, but not worse than usual.

> 
> See, already it's a bit more elaborate than it looks at first ...
> 
>> Even if you are declaring an array in a loop with a different length
>> each time, the allocation code should not require anything more
>> complicated than altering a register at the beginning of the loop,
>> and altering it again at the end.
> 
> ... and it can get worse:
> 
> With gcc, local arrays with a fixed size over about 4KB involve a
> function call to ensure the remaining stack is big enough (so that there
> is no page fault).

That will depend on the target and the OS - maybe Windows requires some
sort of check like that.  In general, however, a compiler will simply
pretend that the stack is as big as needed and rely on the OS and MMU to
sort out the details (such as killing the program if it breaks the rules
of the OS).

> 
> What about VLAs over 4KB? The trouble is it won't know whether a VLA is
> bigger than 4KB, and whether to call that __chkstk_ms() function, until
> runtime. Now it gets complicated enough that using malloc()/free() might
> be a simpler solution.
> 

[toc] | [prev] | [next] | [standalone]

#58825

On 23/03/2015 18:44, Bartc wrote:

> With gcc, local arrays with a fixed size over about 4KB involve a
> function call to ensure the remaining stack is big enough (so that
> there is no page fault).
>
> What about VLAs over 4KB? The trouble is it won't know whether a VLA
> is bigger than 4KB, and whether to call that __chkstk_ms() function,
> until runtime. Now it gets complicated enough that using
> malloc()/free() might be a simpler solution.

The problem is that you are trying to extrapolate how gcc works
on every supported platform, based on your understanding of a
single (quirky!) implementation. This is like trying to learn C
by studying the output from your compiler.

https://xkcd.com/605/

I wouldd guess the "ms" in ___chkstk_ms() stands for "Microsoft".

http://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/src/libgcc/dll_dependency.S

Regards.

[toc] | [prev] | [next] | [standalone]

#58865

Keith Thompson <kst-u@mib.org> writes:

> Bartc <bc@freeuk.com> writes:
>> [snip]
>
> Does gcc *ever* (generate code to) allocate local non-static
> arrays on the heap or in static memory?  It would have to generate
> code to explicitly deallocate the array, even in the presence of a
> goto or longjmp() out of the scope.  [snip]

Certainly it would be nice if a compiler did manage to deallocate
such arrays even in the presence of longjmp(), but strictly
speaking I'm not sure there is a requirement to do so.  So I
think that's a QoI issue, not a conformance problem.  Note that
for VLA's there is an (informative) explicit remark in the
Standard about this, in 7.13.2.1 p5.  I don't see any text in the
Standard that makes arrays that are not VLA's different in this
regard (or for that matter, non-arrays either - if we had a big
struct we might very well want to allocate it using non-stack
memory).

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web