Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1444687

Re: System freezes after OOM

From Mikulas Patocka <mpatocka@redhat.com>
Newsgroups linux.kernel
Subject Re: System freezes after OOM
Date 2016-07-16 02:00 +0200
Message-ID <rVlbs-4PI-11@gated-at.bofh.it> (permalink)
References (10 earlier) <rUVqG-5SB-19@gated-at.bofh.it> <rV9tD-6hc-13@gated-at.bofh.it> <rViQh-3y0-5@gated-at.bofh.it> <rVj9E-3EM-23@gated-at.bofh.it> <rVjjj-3Ig-7@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On Fri, 15 Jul 2016, David Rientjes wrote:

> Kworkers are processing writeback, ext4_writepages() relies on kmem that 

ext4_writepages is above device mapper, not below, so how it could block 
device mapper progress?

Do you use device mapper on the top of block loop device? Writing to loop 
is prone to deadlock anyway, you should avoid that in production code.

> is reclaiming memory itself through kmem_getpages() and they are waiting 
> on the oom victim to exit so they endlessly loop in the page allocator 
> themselves.  Same situation with __alloc_skb() so we can intermittently 
> lose access to hundreds of the machines over the network.  There are no 
> custom drivers required for this to happen, the stack trace has already 
> been posted of the livelock victim and this can happen for anything in 
> filemap_fault() that has TIF_MEMDIE set.

Again - filemap_failt() is above device mapper, not below (unless you use 
loop).

> > > frankly don't care about your patch reviewing of dm mempool usage when 
> > > dm_request() livelocked our kernel.
> > 
> > If it livelocked, it is a bug in some underlying block driver, not a bug 
> > in mempool_alloc.
> > 
> 
> Lol, the interface is quite clear and can be modified to allow mempool 
> users to set __GFP_NOMEMALLOC on their mempool_alloc() request if they can 
> guarantee elements will be returned to the freelist in all situations, 

You still didn't post configuration of your block stack, so I have no clue 
why entries are not returned to the mempool.

> including system oom situations.  We may revert that ourselves if our 
> machines time out once we use a post-4.7 kernel and report that as 
> necessary.

Mikulas

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-11 17:50 +0200
  Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-12 08:50 +0200
    Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 01:50 +0200
      Re: System freezes after OOM Jerome Marchand <jmarchan@redhat.com> - 2016-07-13 10:40 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 13:20 +0200
          Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 16:30 +0200
      Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 13:20 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 15:00 +0200
          Re: System freezes after OOM Milan Broz <gmazyland@gmail.com> - 2016-07-13 15:50 +0200
            Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:40 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 11:10 +0200
                Re: System freezes after OOM Milan Broz <gmazyland@gmail.com> - 2016-07-14 11:50 +0200
        Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:10 +0200
          Re: [dm-devel] System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 13:00 +0200
          Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 15:00 +0200
            Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 16:10 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:00 +0200
                Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 17:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 10:40 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 14:20 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 14:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 19:10 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-18 09:30 +0200
            Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 17:30 +0200
              Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:40 +0200
                Re: System freezes after OOM Ondrej Kozina <okozina@redhat.com> - 2016-07-14 19:10 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 19:40 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-15 13:50 +0200
      Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-13 15:30 +0200
        Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 15:50 +0200
          Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 16:30 +0200
            Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-13 17:00 +0200
              Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-13 17:20 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 02:00 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-14 13:10 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 14:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:30 +0200
                Re: System freezes after OOM Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-07-14 23:50 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 00:10 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 13:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-14 14:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 13:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 23:50 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-16 00:00 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-16 02:00 +0200
                Re: System freezes after OOM Johannes Weiner <hannes@cmpxchg.org> - 2016-07-18 17:20 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-14 17:30 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 22:40 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 09:30 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-15 10:30 +0200
                Re: System freezes after OOM Mikulas Patocka <mpatocka@redhat.com> - 2016-07-15 14:10 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-15 23:50 +0200
                Re: System freezes after OOM Michal Hocko <mhocko@kernel.org> - 2016-07-18 09:40 +0200
                Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-18 23:10 +0200
        Re: System freezes after OOM David Rientjes <rientjes@google.com> - 2016-07-14 02:10 +0200

csiph-web