Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #15675

Re: [OT] fork() 50 times slower on some Linux kernels?

Show all headers | View raw

Stephen Sprunk <stephen@sprunk.org> wrote:
> On 09-Jan-12 18:22, Kaz Kylheku wrote:
<snip>
> > Even with strict accounting, fork should not be duplicating the pages
> > and committing memory for them, but be implementing a copy-on-write
> > optimization. Strict accounting would mean that fork should fail if
> > creating the (empty!) maps for the process would bring the total VM
> > footprint of the system over the limit.
> > 
> > That's the theory.

> If you don't back those pages during the fork() operation, then you're
> back to the original problem.

> For instance, imagine a process that allocates a thousand pages.  If
> overcommitment is disabled, then the kernel will actually back them (or
> allocation will fail).  The process forks, and then both parent and
> child write to each of the thousand pages, triggering a thousand
> copy-on-writes and the need to find a thousand backing pages.  If even
> one of those pages can't be backed, then one of the processes will
> fail--something that should have been impossible.  (Or worse, the OOM
> Killer turns its attention elsewhere to obtain backing pages...)

> A simpler implementation would be to reject copy-on-write entirely,
> duplicating the pages.  However, that would unnecessarily double the
> number of active pages and pollute the cache even if no writes occur,
> which is the normal case.

Maybe I misunderstand what you mean by "duplicate", but it seems to me you
don't need to reject copy-on-write. You just need to reserve the number of
pages needed to do any future copying. You could do this with simple
counters: # of pages in use, # of pages reserved.

Like Ken, I also think that the bogeyman here--of processes which allocate a
ton of memory upfront--is a poor excuse and overhyped. Those applications
are few and far between. Sometimes important, maybe, but theoretically
easily fixed. Most common--at least open source--malloc() implementation use
mmap(); so teach them to create their own swap by mmaping space backed by
anonymous /tmp or /var/tmp files or something similar. You could even make
it COW after a fork and have malloc() catch a signal to copy and swap
regions, aborting the process if no more disk space is available. On a
modern Linux system all of this could be done from userspace without any
kernel help, albeit maybe not optimally.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

fork() 50 times slower on some Linux kernels? Elmar <elmar.krieger@gmail.com> - 2012-01-09 12:25 -0800
  Re: fork() 50 times slower on some Linux kernels? Ian Collins <ian-news@hotmail.com> - 2012-01-10 09:41 +1300
  Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 21:32 +0000
    [OT] Re: fork() 50 times slower on some Linux kernels? jt@toerring.de (Jens Thoms Toerring) - 2012-01-09 22:12 +0000
      Re: [OT] Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 22:32 +0000
        Re: [OT] Re: fork() 50 times slower on some Linux kernels? Ian Collins <ian-news@hotmail.com> - 2012-01-10 11:56 +1300
  Re: fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 16:11 -0600
    Re: fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-09 22:30 +0000
      Re: fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 17:20 -0600
        Re: [OT] fork() 50 times slower on some Linux kernels? jt@toerring.de (Jens Thoms Toerring) - 2012-01-09 23:57 +0000
          Re: [OT] fork() 50 times slower on some Linux kernels? Kaz Kylheku <kaz@kylheku.com> - 2012-01-10 00:22 +0000
            Re: [OT] fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-09 20:39 -0600
              Re: [OT] fork() 50 times slower on some Linux kernels? William Ahern <william@wilbur.25thandClement.com> - 2012-01-09 19:43 -0800
                Re: [OT] fork() 50 times slower on some Linux kernels? Stephen Sprunk <stephen@sprunk.org> - 2012-01-10 00:31 -0600
          Re: fork() 50 times slower on some Linux kernels? Elmar <elmar.krieger@gmail.com> - 2012-01-11 14:39 -0800
        Re: fork() 50 times slower on some Linux kernels? Nobody <nobody@nowhere.com> - 2012-01-10 23:07 +0000

csiph-web