[RFC PATCH 0/2] mempool vs. page allocator interaction

From	Michal Hocko <mhocko@kernel.org>
Newsgroups	linux.kernel
Subject	[RFC PATCH 0/2] mempool vs. page allocator interaction
Date	2016-07-18 10:40 +0200
Message-ID	<rWcfL-3Nz-13@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi,
there have been two issues identified when investigating dm-crypt
backed swap recently [1]. The first one looks like a regression from
f9054c70d28b ("mm, mempool: only set __GFP_NOMEMALLOC if there are free
elements") because swapout path can now deplete all the available memory
reserves. The first patch tries to address that issue by dropping
__GFP_NOMEMALLOC only to TIF_MEMDIE tasks.

The second issue is that dm writeout path which relies on mempool
allocator gets throttled by the direct reclaim in throttle_vm_writeout
which just makes the whole memory pressure problem even worse. The
patch2 just makes sure that we annotate mempool users to be throttled
less by PF_LESS_THROTTLE flag and prevent from throttle_vm_writeout for
that path. mempool users are usually the IO path and throttle them less
sounds like a reasonable way to go.

I do not have any more complicated dm setup available so I would
appreciate if dm people (CCed) could give these two a try.

Also it would be great to iron out concerns from David. He has posted a
deadlock stack trace [2] which has led to f9054c70d28b which is bio
allocation lockup because the TIF_MEMDIE process cannot make a forward
progress without access to memory reserve. This case should be fixed by
patch 1 AFAICS. There are other potential cases when the stuck mempool
is called from PF_MEMALLOC context and blocks the oom victim indirectly
(over a lock) but I believe those are much less likely and we have the
oom reaper to make a forward progress.

Sorry of pulling the discussion outside of the original email thread
but there were more lines of dicussion there and I felt discussing
particualr solution with its justification has a greater chance of
moving towards a solution. I am sending this as an RFC because this
needs a deep review as there might be other side effects I do not see
(especially about patch 2).

Any comments, suggestions are welcome.

---
[1] http://lkml.kernel.org/r/alpine.LRH.2.02.1607111027080.14327@file01.intranet.prod.int.rdu2.redhat.com
[2] http://lkml.kernel.org/r/alpine.DEB.2.10.1607131644590.92037@chino.kir.corp.google.com

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[RFC PATCH 0/2] mempool vs. page allocator interaction Michal Hocko <mhocko@kernel.org> - 2016-07-18 10:40 +0200
  [RFC PATCH 1/2] mempool: do not consume memory reserves from the reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-18 10:50 +0200
    [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks Michal Hocko <mhocko@kernel.org> - 2016-07-18 10:50 +0200
      Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE  tasks Mikulas Patocka <mpatocka@redhat.com> - 2016-07-20 00:00 +0200
      Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks NeilBrown <neilb@suse.de> - 2016-07-22 10:50 +0200
        Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks NeilBrown <neilb@suse.com> - 2016-07-22 11:10 +0200
        Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE  tasks Michal Hocko <mhocko@kernel.org> - 2016-07-22 11:20 +0200
          Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks NeilBrown <neilb@suse.com> - 2016-07-23 02:20 +0200
            Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE  tasks Michal Hocko <mhocko@kernel.org> - 2016-07-25 10:40 +0200
              Re: [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE  tasks Michal Hocko <mhocko@kernel.org> - 2016-07-25 21:30 +0200
    Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path David Rientjes <rientjes@google.com> - 2016-07-19 04:10 +0200
      Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-19 09:50 +0200
    Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Johannes Weiner <hannes@cmpxchg.org> - 2016-07-19 16:00 +0200
      Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-19 16:30 +0200
        Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path Mikulas Patocka <mpatocka@redhat.com> - 2016-07-20 00:10 +0200
      Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path David Rientjes <rientjes@google.com> - 2016-07-19 22:50 +0200
        Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-20 10:20 +0200
          Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path David Rientjes <rientjes@google.com> - 2016-07-20 23:10 +0200
            Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-21 11:00 +0200
              Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Johannes Weiner <hannes@cmpxchg.org> - 2016-07-21 14:20 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-21 17:00 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Johannes Weiner <hannes@cmpxchg.org> - 2016-07-21 17:30 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the reclaim path NeilBrown <neilb@suse.com> - 2016-07-22 03:50 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-22 08:40 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Vlastimil Babka <vbabka@suse.cz> - 2016-07-22 14:30 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path Andrew Morton <akpm@linux-foundation.org> - 2016-07-22 21:50 +0200
                Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Vlastimil Babka <vbabka@suse.cz> - 2016-07-23 21:00 +0200
    Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from  the reclaim path Mikulas Patocka <mpatocka@redhat.com> - 2016-07-20 00:00 +0200
      Re: [RFC PATCH 1/2] mempool: do not consume memory reserves from the  reclaim path Michal Hocko <mhocko@kernel.org> - 2016-07-20 08:50 +0200

csiph-web