Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #107198

Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit?

Show all headers | View raw

On Monday, March 27, 2017 at 12:38:08 PM UTC-5, Gordon Burditt wrote:
> A fact of life is that programs often request memory they don't
> ever use.  A lot of this comes from fork(), which allocates copies
> of a lot of memory in the parent process for the child, (copy-on-write
> means that unless it's written, one copy of a page serves for both
> processes) which is often followed by exec*(), which throws out all
> that memory and replaces it with another program.

How often would programs request large amounts of memory that they never
use outside of situations involving fork?  How often is fork() used in
situations which wouldn't better be served by a system call that starts
a new process along with calls to create, attach, and detach mutable or
immutable shareable objects?

> A process which has allocated more than half of allowed memory
> will not be able to fork() without overcommitting memory or
> having the fork() fail.  Calling free() will normally not help.

Except in the cases where a spawned process would need a copy of everything
the parent had in memory, using a forking process that creates a logical
copy of everything seems silly to me, and I question how often copy-on-write
would be needed at the process level otherwise.

> > Now I saw some cases where a program is trying to load memory in
> > 1 MB chunks in a while loop. In that case I thought if the heap
> > itself is used totally the program should crash as it do not have
> > permission to access other application's memory segments.
> 
> A refused memory allocation is *NOT* a crash.  The program should
> check whether it got the memory and, as a worst case, exit gracefully
> with an error message if it cannot allocate memory instead of
> attempting to use it anyway.  This is not a "crash".  Perhaps a
> better response is to continue with a smaller buffer.

If a program won't be able to do anything useful unless it can get all the
memory it needs, a guarantee that a memory allocation request will either
succeed or force some kind of program termination can often eliminate the
need to have a lot of error-checking scattered throughout a program.  It
would have been helpful to have a standard means of setting up cleanup
handlers that could be used in conjunction with setjmp() to support more
systematic error-recovery behavior.

If a program will be able to adapt itself to varying amounts of available
memory, having it limit usage only when allocations fail may not yield the
most useful behavior, since such a program might grab memory which could
otherwise have been used by a program that needs it more.

> Linux will overcommit memory (that is, let you allocate memory for
> the heap without any physical resource to back it) *IF* it is set
> to do that.  This is the default setting.  You can turn it off if
> the sysctl vm.overcommit_memory = 2.  Typically making setting means
> you also need more swap space.  You can also tune vm.overcommit_ratio
> and vm.overcommit_kbytes to regulate *how much* it will overcommit.

> The response to attempting to write memory when the system has run
> out of physical memory to back it is to SIGTERM the program trying
> to write the memory (which is not necessarily the same program as
> the "big memory hog" program - it might be some innocent program
> just trying to grow its stack or use a small amount of memory it
> allocated).  The program dies on a SIGTERM signal.  Not "hang".  If
> memory use is high and programs are competing heavily for RAM, the
> system will spend a lot of time swapping and paging and it may
> appear to "hang".

That doesn't seem very helpful.  What happens if the SIGTERM handler
tries to write to a sig_atomic_t which is sitting in a shared
copy-on-write page?

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? acidlategamer@gmail.com - 2017-03-27 01:23 -0700
  Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? "James R. Kuyper" <jameskuyper@verizon.net> - 2017-03-27 11:06 -0400
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Ken Brody <kenbrody@spamcop.net> - 2017-03-27 14:50 -0400
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-03-27 21:17 +0100
      Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 19:08 -0500
  Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? scott@slp53.sl.home (Scott Lurndal) - 2017-03-27 15:23 +0000
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-03-27 09:57 -0700
      Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? BartC <bc@freeuk.com> - 2017-03-27 18:14 +0100
  Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? gordonb.l1p2g@burditt.org (Gordon Burditt) - 2017-03-27 12:37 -0500
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? "James R. Kuyper" <jameskuyper@verizon.net> - 2017-03-27 13:53 -0400
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? scott@slp53.sl.home (Scott Lurndal) - 2017-03-27 18:06 +0000
    Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? supercat@casperkitty.com - 2017-03-27 12:29 -0700
      Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 19:04 -0500
        Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? supercat@casperkitty.com - 2017-03-27 17:31 -0700
          Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Ian Collins <ian-news@hotmail.com> - 2017-03-28 13:35 +1300
        Re: Does heap segment of a C program increases in the amount of RAM or is it fixed to a limit? Ian Collins <ian-news@hotmail.com> - 2017-03-28 13:37 +1300

csiph-web