Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #15663

Re: fork() 50 times slower on some Linux kernels?

Show all headers | View raw

On 2012-01-09, Stephen Sprunk <stephen@sprunk.org> wrote:
> On 09-Jan-12 14:25, Elmar wrote:
>> a previous discussion in this group mentioned the problem that fork()
>> is a resource bottleneck because it needs to (virtually) duplicate a
>> process' memory space:
>> 
>> http://groups.google.com/group/comp.lang.c/browse_thread/thread/911978685974e041/bdd37113d7b43e0c
>> ...
>> Google didn't help much, so my hope was that one of you has heard of/
>> encountered this problem, knows which kernels are affected, and when/
>> how it was solved, maybe there is a configuration trick etc.?
>
> Did you read that thread?  The answer is in there, and it has nothing to
> do with the kernel versions; it's about whether vm overcommitment is
> enabled (fast but unsafe) or disabled (safe but unfast).

That's going to be moot in this issue that he's having. Strict overcommit
accounting only makes a difference to whether a call to mmap succeeds or
fails. Allocation of the mmaps is still lazy. The OS merely ensures (sort of)
that it has the "cash" to back the "check" it has written (i.e. that's exactly
why this is called accounting!) ``If we hand out this virtual space, will there
later be enough pages available to populate it?'' If the answer is no, then the
mmap is denied (ENOMEM).  

(This accounting method isn't accurate, because the kernel OS can allocate pages
for itself which are not accounted.  That's why there is the percentage fudge
factor ``make the total virtual memory outlay no more than XXX% relative
to total core and swap''. If you make this 100, it is not strict enough because
it allows as much total virtual memory as there is physical storage, with no
space for kernel pages. But there is no guarantee that, say, 80% is enough,
either, in every conceivable situation. Why not just add the kernel pages
to the accounting? Then VM will be counted twice, because mmap regions are
populated by the same kernel allocations! You would have to categorize kernel
get_free_page calls into those which go into populating anonymous mmaps, and
those which go elsewhere.)

Overcommit accounting is also moot if you memset the memory, because then
you're committing it immediately (taking the check written by mmap straight to
the bank to get it in cash).

In any case, did you notice in the original posting that this 1 second
pause occurs on a system with 8GB of RAM?

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

fork() 50 times slower on some Linux kernels? Elmar <elmar.krieger@gmail.com> - 2012-01-09 12:25 -0800
  Re: fork() 50 times slower on some Linux kernels? Ian Collins <ian-news@hotmail.com> - 2012-01-10 09:41 +1300
  Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 21:32 +0000
    [OT] Re: fork() 50 times slower on some Linux kernels? jt@toerring.de (Jens Thoms Toerring) - 2012-01-09 22:12 +0000
      Re: [OT] Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 22:32 +0000
        Re: [OT] Re: fork() 50 times slower on some Linux kernels? Ian Collins <ian-news@hotmail.com> - 2012-01-10 11:56 +1300
  Re: fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 16:11 -0600
    Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 22:30 +0000
      Re: fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 17:20 -0600
        Re: [OT] fork() 50 times slower on some Linux kernels? jt@toerring.de (Jens Thoms Toerring) - 2012-01-09 23:57 +0000
          Re: [OT] fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-10 00:22 +0000
            Re: [OT] fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 20:39 -0600
              Re: [OT] fork() 50 times slower on some Linux kernels? William Ahern <william@wilbur.25thandClement.com> - 2012-01-09 19:43 -0800
                Re: [OT] fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-10 00:31 -0600
          Re: fork() 50 times slower on some Linux kernels? Elmar <elmar.krieger@gmail.com> - 2012-01-11 14:39 -0800
        Re: fork() 50 times slower on some Linux kernels? Nobody <nobody@nowhere.com> - 2012-01-10 23:07 +0000

csiph-web