Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1443408

Re: System freezes after OOM

From Mikulas Patocka <mpatocka@redhat.com>
Newsgroups linux.kernel
Subject Re: System freezes after OOM
Date 2016-07-14 14:30 +0200
Message-ID <rUNWa-1dc-17@gated-at.bofh.it> (permalink)
References (7 earlier) <rUsI1-3Kr-5@gated-at.bofh.it> <rUtkK-4fl-25@gated-at.bofh.it> <rUtNL-4qs-1@gated-at.bofh.it> <rUu78-4Nq-7@gated-at.bofh.it> <rUCel-1CC-7@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On Wed, 13 Jul 2016, David Rientjes wrote:

> On Wed, 13 Jul 2016, Mikulas Patocka wrote:
> 
> > What are the real problems that f9054c70d28bc214b2857cf8db8269f4f45a5e23 
> > tries to fix?
> > 
> 
> It prevents the whole system from livelocking due to an oom killed process 
> stalling forever waiting for mempool_alloc() to return.  No other threads 
> may be oom killed while waiting for it to exit.
> 
> > Do you have a stacktrace where it deadlocked, or was just a theoretical 
> > consideration?
> > 
> 
> schedule
> schedule_timeout
> io_schedule_timeout
> mempool_alloc
> __split_and_process_bio
> dm_request
> generic_make_request
> submit_bio
> mpage_readpages
> ext4_readpages
> __do_page_cache_readahead
> ra_submit
> filemap_fault
> handle_mm_fault
> __do_page_fault
> do_page_fault
> page_fault

Device mapper should be able to proceed if there is no available memory. 
If it doesn't proceed, there is a bug in it.

I'd like to ask - what device mapper targets did you use in this case? Are 
there some other deadlocked processes? (show sysrq-t, sysrq-w when this 
happened)

Did the machine lock up completely with that stacktrace, or was it just 
slowed down?

> > Mempool users generally (except for some flawed cases like fs_bio_set) do 
> > not require memory to proceed. So if you just loop in mempool_alloc, the 
> > processes that exhasted the mempool reserve will eventually return objects 
> > to the mempool and you should proceed.
> > 
> 
> That's obviously not the case if we have hundreds of machines timing out 
> after two hours waiting for that fault to succeed.  The mempool interface 
> cannot require that users return elements to the pool synchronous with all 
> allocators so that we can happily loop forever, the only requirement on 

Mempool users must return objects to the mempool.

> the interface is that mempool_alloc() must succeed.  If the context of the 
> thread doing mempool_alloc() allows access to memory reserves, this will 
> always be allowed by the page allocator.  This is not a mempool problem.

Mikulas

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-11 17:50 +0200
  Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-12 08:50 +0200
    Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 01:50 +0200
      Re: System freezes after OOM Jerome Marchand <jmarchan@redhat.com> - 2016-07-13 10:40 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 13:20 +0200
          Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 16:30 +0200
      Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 13:20 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 15:00 +0200
          Re: System freezes after OOM Milan Broz <gmazyland@gmail.com> - 2016-07-13 15:50 +0200
            Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:40 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 11:10 +0200
                Re: System freezes after OOM Milan Broz <gmazyland@gmail.com> - 2016-07-14 11:50 +0200
        Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:10 +0200
          Re: [dm-devel] System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 13:00 +0200
          Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 15:00 +0200
            Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 16:10 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:00 +0200
                Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 17:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 10:40 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 14:20 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 14:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 19:10 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-18 09:30 +0200
            Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 17:30 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:40 +0200
                Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 19:10 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-15 13:50 +0200
      Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-13 15:30 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 15:50 +0200
          Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 16:30 +0200
            Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 17:00 +0200
              Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:20 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 02:00 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-14 13:10 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 14:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:30 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-14 23:50 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 00:10 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 13:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 14:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 13:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 23:50 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-16 00:00 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-16 02:00 +0200
                Re: System freezes after OOM Johannes Weiner <hannes@cmpxchg.org> - 2016-07-18 17:20 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 09:30 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 10:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 14:10 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:50 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-18 09:40 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-18 23:10 +0200
        Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 02:10 +0200

csiph-web