Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1467503 > unrolled thread

Re: OOM detection regressions since 4.7

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: OOM detection regressions since 4.7 Michal Hocko <mhocko@kernel.org> - 2016-08-22 11:40 +0200
    Re: OOM detection regressions since 4.7 Greg KH <gregkh@linuxfoundation.org> - 2016-08-22 12:20 +0200
      Re: OOM detection regressions since 4.7 Michal Hocko <mhocko@kernel.org> - 2016-08-22 13:00 +0200
        Re: OOM detection regressions since 4.7 Greg KH <gregkh@linuxfoundation.org> - 2016-08-22 15:40 +0200
          Re: OOM detection regressions since 4.7 Michal Hocko <mhocko@kernel.org> - 2016-08-22 15:50 +0200
            Re: OOM detection regressions since 4.7 Greg KH <gregkh@linuxfoundation.org> - 2016-08-22 16:10 +0200
            Re: OOM detection regressions since 4.7 Andrew Morton <akpm@linux-foundation.org> - 2016-08-23 00:10 +0200
              Re: OOM detection regressions since 4.7 Michal Hocko <mhocko@kernel.org> - 2016-08-23 11:00 +0200
                Re: OOM detection regressions since 4.7 Olaf Hering <olaf@aepfle.de> - 2016-08-25 09:30 +0200
                Re: OOM detection regressions since 4.7 Michal Hocko <mhocko@kernel.org> - 2016-08-25 09:40 +0200

#1467503 — Re: OOM detection regressions since 4.7

[ups, fixing up Greg's email]

On Mon 22-08-16 11:32:49, Michal Hocko wrote:
> Hi, 
> there have been multiple reports [1][2][3][4][5] about pre-mature OOM
> killer invocations since 4.7 which contains oom detection rework. All of
> them were for order-2 (kernel stack) alloaction requests failing because
> of a high fragmentation and compaction failing to make any forward
> progress. While investigating this we have found out that the compaction
> just gives up too early. Vlastimil has been working on compaction
> improvement for quite some time and his series [6] is already sitting
> in mmotm tree. This already helps a lot because it drops some heuristics
> which are more aimed at lower latencies for high orders rather than
> reliability. Joonsoo has then identified further problem with too many
> blocks being marked as unmovable [7] and Vlastimil has prepared a patch
> on top of his series [8] which is also in the mmotm tree now.
> 
> That being said, the regression is real and should be fixed for 4.7
> stable users. [6][8] was reported to help and ooms are no longer
> reproducible. I know we are quite late (rc3) in 4.8 but I would vote
> for mergeing those patches and have them in 4.8. For 4.7 I would go
> with a partial revert of the detection rework for high order requests
> (see patch below). This patch is really trivial. If those compaction
> improvements are just too large for 4.8 then we can use the same patch
> as for 4.7 stable for now and revert it in 4.9 after compaction changes
> are merged.
> 
> Thoughts?
> 
> [1] http://lkml.kernel.org/r/20160731051121.GB307@x4
> [2] http://lkml.kernel.org/r/201608120901.41463.a.miskiewicz@gmail.com
> [3] http://lkml.kernel.org/r/20160801192620.GD31957@dhcp22.suse.cz
> [4] https://lists.opensuse.org/opensuse-kernel/2016-08/msg00021.html
> [5] https://bugzilla.opensuse.org/show_bug.cgi?id=994066
> [6] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> [7] http://lkml.kernel.org/r/20160816031222.GC16913@js1304-P5Q-DELUXE
> [8] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> 
> ---
> From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> From: Michal Hocko <mhocko@suse.com>
> Date: Mon, 22 Aug 2016 10:52:06 +0200
> Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
>  order request
> 
> There have been several reports about pre-mature OOM killer invocation
> in 4.7 kernel when order-2 allocation request (for the kernel stack)
> invoked OOM killer even during basic workloads (light IO or even kernel
> compile on some filesystems). In all reported cases the memory is
> fragmented and there are no order-2+ pages available. There is usually
> a large amount of slab memory (usually dentries/inodes) and further
> debugging has shown that there are way too many unmovable blocks which
> are skipped during the compaction. Multiple reporters have confirmed that
> the current linux-next which includes [1] and [2] helped and OOMs are
> not reproducible anymore. A simpler fix for the stable is to simply
> ignore the compaction feedback and retry as long as there is a reclaim
> progress for high order requests which we used to do before. We already
> do that for CONFING_COMPACTION=n so let's reuse the same code when
> compaction is enabled as well.
> 
> [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> 
> Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> Signed-off-by: Michal Hocko <mhocko@suse.com>
> ---
>  mm/page_alloc.c | 50 ++------------------------------------------------
>  1 file changed, 2 insertions(+), 48 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 8b3e1341b754..6e354199151b 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3254,53 +3254,6 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  	return NULL;
>  }
>  
> -static inline bool
> -should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
> -		     enum compact_result compact_result, enum migrate_mode *migrate_mode,
> -		     int compaction_retries)
> -{
> -	int max_retries = MAX_COMPACT_RETRIES;
> -
> -	if (!order)
> -		return false;
> -
> -	/*
> -	 * compaction considers all the zone as desperately out of memory
> -	 * so it doesn't really make much sense to retry except when the
> -	 * failure could be caused by weak migration mode.
> -	 */
> -	if (compaction_failed(compact_result)) {
> -		if (*migrate_mode == MIGRATE_ASYNC) {
> -			*migrate_mode = MIGRATE_SYNC_LIGHT;
> -			return true;
> -		}
> -		return false;
> -	}
> -
> -	/*
> -	 * make sure the compaction wasn't deferred or didn't bail out early
> -	 * due to locks contention before we declare that we should give up.
> -	 * But do not retry if the given zonelist is not suitable for
> -	 * compaction.
> -	 */
> -	if (compaction_withdrawn(compact_result))
> -		return compaction_zonelist_suitable(ac, order, alloc_flags);
> -
> -	/*
> -	 * !costly requests are much more important than __GFP_REPEAT
> -	 * costly ones because they are de facto nofail and invoke OOM
> -	 * killer to move on while costly can fail and users are ready
> -	 * to cope with that. 1/4 retries is rather arbitrary but we
> -	 * would need much more detailed feedback from compaction to
> -	 * make a better decision.
> -	 */
> -	if (order > PAGE_ALLOC_COSTLY_ORDER)
> -		max_retries /= 4;
> -	if (compaction_retries <= max_retries)
> -		return true;
> -
> -	return false;
> -}
>  #else
>  static inline struct page *
>  __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
> @@ -3311,6 +3264,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  	return NULL;
>  }
>  
> +#endif /* CONFIG_COMPACTION */
> +
>  static inline bool
>  should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_flags,
>  		     enum compact_result compact_result,
> @@ -3337,7 +3292,6 @@ should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_fla
>  	}
>  	return false;
>  }
> -#endif /* CONFIG_COMPACTION */
>  
>  /* Perform direct synchronous page reclaim */
>  static int
> -- 
> 2.8.1
> 
> -- 
> Michal Hocko
> SUSE Labs

-- 
Michal Hocko
SUSE Labs

[toc] | [next] | [standalone]

#1467526

On Mon, Aug 22, 2016 at 11:37:07AM +0200, Michal Hocko wrote:
> [ups, fixing up Greg's email]
> 
> On Mon 22-08-16 11:32:49, Michal Hocko wrote:
> > Hi, 
> > there have been multiple reports [1][2][3][4][5] about pre-mature OOM
> > killer invocations since 4.7 which contains oom detection rework. All of
> > them were for order-2 (kernel stack) alloaction requests failing because
> > of a high fragmentation and compaction failing to make any forward
> > progress. While investigating this we have found out that the compaction
> > just gives up too early. Vlastimil has been working on compaction
> > improvement for quite some time and his series [6] is already sitting
> > in mmotm tree. This already helps a lot because it drops some heuristics
> > which are more aimed at lower latencies for high orders rather than
> > reliability. Joonsoo has then identified further problem with too many
> > blocks being marked as unmovable [7] and Vlastimil has prepared a patch
> > on top of his series [8] which is also in the mmotm tree now.
> > 
> > That being said, the regression is real and should be fixed for 4.7
> > stable users. [6][8] was reported to help and ooms are no longer
> > reproducible. I know we are quite late (rc3) in 4.8 but I would vote
> > for mergeing those patches and have them in 4.8. For 4.7 I would go
> > with a partial revert of the detection rework for high order requests
> > (see patch below). This patch is really trivial. If those compaction
> > improvements are just too large for 4.8 then we can use the same patch
> > as for 4.7 stable for now and revert it in 4.9 after compaction changes
> > are merged.
> > 
> > Thoughts?
> > 
> > [1] http://lkml.kernel.org/r/20160731051121.GB307@x4
> > [2] http://lkml.kernel.org/r/201608120901.41463.a.miskiewicz@gmail.com
> > [3] http://lkml.kernel.org/r/20160801192620.GD31957@dhcp22.suse.cz
> > [4] https://lists.opensuse.org/opensuse-kernel/2016-08/msg00021.html
> > [5] https://bugzilla.opensuse.org/show_bug.cgi?id=994066
> > [6] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > [7] http://lkml.kernel.org/r/20160816031222.GC16913@js1304-P5Q-DELUXE
> > [8] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > 
> > ---
> > From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> > From: Michal Hocko <mhocko@suse.com>
> > Date: Mon, 22 Aug 2016 10:52:06 +0200
> > Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
> >  order request
> > 
> > There have been several reports about pre-mature OOM killer invocation
> > in 4.7 kernel when order-2 allocation request (for the kernel stack)
> > invoked OOM killer even during basic workloads (light IO or even kernel
> > compile on some filesystems). In all reported cases the memory is
> > fragmented and there are no order-2+ pages available. There is usually
> > a large amount of slab memory (usually dentries/inodes) and further
> > debugging has shown that there are way too many unmovable blocks which
> > are skipped during the compaction. Multiple reporters have confirmed that
> > the current linux-next which includes [1] and [2] helped and OOMs are
> > not reproducible anymore. A simpler fix for the stable is to simply
> > ignore the compaction feedback and retry as long as there is a reclaim
> > progress for high order requests which we used to do before. We already
> > do that for CONFING_COMPACTION=n so let's reuse the same code when
> > compaction is enabled as well.
> > 
> > [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > 
> > Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > ---
> >  mm/page_alloc.c | 50 ++------------------------------------------------
> >  1 file changed, 2 insertions(+), 48 deletions(-)

So, if this goes into Linus's tree, can you let stable@vger.kernel.org
know about it so we can add it to the 4.7-stable tree?  Otherwise
there's not much I can do here now, right?

thanks,

greg k-h

[toc] | [prev] | [next] | [standalone]

#1467547

On Mon 22-08-16 06:05:28, Greg KH wrote:
> On Mon, Aug 22, 2016 at 11:37:07AM +0200, Michal Hocko wrote:
[...]
> > > From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> > > From: Michal Hocko <mhocko@suse.com>
> > > Date: Mon, 22 Aug 2016 10:52:06 +0200
> > > Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
> > >  order request
> > > 
> > > There have been several reports about pre-mature OOM killer invocation
> > > in 4.7 kernel when order-2 allocation request (for the kernel stack)
> > > invoked OOM killer even during basic workloads (light IO or even kernel
> > > compile on some filesystems). In all reported cases the memory is
> > > fragmented and there are no order-2+ pages available. There is usually
> > > a large amount of slab memory (usually dentries/inodes) and further
> > > debugging has shown that there are way too many unmovable blocks which
> > > are skipped during the compaction. Multiple reporters have confirmed that
> > > the current linux-next which includes [1] and [2] helped and OOMs are
> > > not reproducible anymore. A simpler fix for the stable is to simply
> > > ignore the compaction feedback and retry as long as there is a reclaim
> > > progress for high order requests which we used to do before. We already
> > > do that for CONFING_COMPACTION=n so let's reuse the same code when
> > > compaction is enabled as well.
> > > 
> > > [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > > [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > > 
> > > Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> > > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > > ---
> > >  mm/page_alloc.c | 50 ++------------------------------------------------
> > >  1 file changed, 2 insertions(+), 48 deletions(-)
> 
> So, if this goes into Linus's tree, can you let stable@vger.kernel.org
> know about it so we can add it to the 4.7-stable tree?  Otherwise
> there's not much I can do here now, right?

My plan would be actually to not push this to Linus because we have a
proper fix for Linus tree. It is just that the fix is quite large and I
felt like the stable should get the most simple fix possible, which is
this partial revert. So, what I am trying to tell is to push a non-linus
patch to stable as it is simpler.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1467631

On Mon, Aug 22, 2016 at 12:54:41PM +0200, Michal Hocko wrote:
> On Mon 22-08-16 06:05:28, Greg KH wrote:
> > On Mon, Aug 22, 2016 at 11:37:07AM +0200, Michal Hocko wrote:
> [...]
> > > > From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> > > > From: Michal Hocko <mhocko@suse.com>
> > > > Date: Mon, 22 Aug 2016 10:52:06 +0200
> > > > Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
> > > >  order request
> > > > 
> > > > There have been several reports about pre-mature OOM killer invocation
> > > > in 4.7 kernel when order-2 allocation request (for the kernel stack)
> > > > invoked OOM killer even during basic workloads (light IO or even kernel
> > > > compile on some filesystems). In all reported cases the memory is
> > > > fragmented and there are no order-2+ pages available. There is usually
> > > > a large amount of slab memory (usually dentries/inodes) and further
> > > > debugging has shown that there are way too many unmovable blocks which
> > > > are skipped during the compaction. Multiple reporters have confirmed that
> > > > the current linux-next which includes [1] and [2] helped and OOMs are
> > > > not reproducible anymore. A simpler fix for the stable is to simply
> > > > ignore the compaction feedback and retry as long as there is a reclaim
> > > > progress for high order requests which we used to do before. We already
> > > > do that for CONFING_COMPACTION=n so let's reuse the same code when
> > > > compaction is enabled as well.
> > > > 
> > > > [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > > > [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > > > 
> > > > Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> > > > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > > > ---
> > > >  mm/page_alloc.c | 50 ++------------------------------------------------
> > > >  1 file changed, 2 insertions(+), 48 deletions(-)
> > 
> > So, if this goes into Linus's tree, can you let stable@vger.kernel.org
> > know about it so we can add it to the 4.7-stable tree?  Otherwise
> > there's not much I can do here now, right?
> 
> My plan would be actually to not push this to Linus because we have a
> proper fix for Linus tree. It is just that the fix is quite large and I
> felt like the stable should get the most simple fix possible, which is
> this partial revert. So, what I am trying to tell is to push a non-linus
> patch to stable as it is simpler.

I _REALLY_ hate taking any patches that are not in Linus's tree as 90%
of the time (well, almost always), it ends up being wrong and hurting us
in the end.

What exactly are the commits that are in Linus's tree that resolve this
issue?

thanks,

greg k-h

[toc] | [prev] | [next] | [standalone]

#1467632

On Mon 22-08-16 09:31:14, Greg KH wrote:
> On Mon, Aug 22, 2016 at 12:54:41PM +0200, Michal Hocko wrote:
> > On Mon 22-08-16 06:05:28, Greg KH wrote:
> > > On Mon, Aug 22, 2016 at 11:37:07AM +0200, Michal Hocko wrote:
> > [...]
> > > > > From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> > > > > From: Michal Hocko <mhocko@suse.com>
> > > > > Date: Mon, 22 Aug 2016 10:52:06 +0200
> > > > > Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
> > > > >  order request
> > > > > 
> > > > > There have been several reports about pre-mature OOM killer invocation
> > > > > in 4.7 kernel when order-2 allocation request (for the kernel stack)
> > > > > invoked OOM killer even during basic workloads (light IO or even kernel
> > > > > compile on some filesystems). In all reported cases the memory is
> > > > > fragmented and there are no order-2+ pages available. There is usually
> > > > > a large amount of slab memory (usually dentries/inodes) and further
> > > > > debugging has shown that there are way too many unmovable blocks which
> > > > > are skipped during the compaction. Multiple reporters have confirmed that
> > > > > the current linux-next which includes [1] and [2] helped and OOMs are
> > > > > not reproducible anymore. A simpler fix for the stable is to simply
> > > > > ignore the compaction feedback and retry as long as there is a reclaim
> > > > > progress for high order requests which we used to do before. We already
> > > > > do that for CONFING_COMPACTION=n so let's reuse the same code when
> > > > > compaction is enabled as well.
> > > > > 
> > > > > [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > > > > [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > > > > 
> > > > > Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> > > > > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > > > > ---
> > > > >  mm/page_alloc.c | 50 ++------------------------------------------------
> > > > >  1 file changed, 2 insertions(+), 48 deletions(-)
> > > 
> > > So, if this goes into Linus's tree, can you let stable@vger.kernel.org
> > > know about it so we can add it to the 4.7-stable tree?  Otherwise
> > > there's not much I can do here now, right?
> > 
> > My plan would be actually to not push this to Linus because we have a
> > proper fix for Linus tree. It is just that the fix is quite large and I
> > felt like the stable should get the most simple fix possible, which is
> > this partial revert. So, what I am trying to tell is to push a non-linus
> > patch to stable as it is simpler.
> 
> I _REALLY_ hate taking any patches that are not in Linus's tree as 90%
> of the time (well, almost always), it ends up being wrong and hurting us
> in the end.

I do not like it either but if there is a simple and straightforward
workaround for stable while the upstream can go with the _proper_ fix
from the longer POV then I think this is perfectly justified. Stable
should be always about the simplest fix for the problem IMHO.

Of course, if Linus/Andrew doesn't like to take those compaction
improvements this late then I will ask to merge the partial revert to
Linus tree as well and then there is not much to discuss.

> What exactly are the commits that are in Linus's tree that resolve this
> issue?

The initial email in this thread has pointed to those patches. Please
note that some of its dependeces (mostly code cleanups) are already
merged and that backporting without them would make the backport harder
and more risky.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1467652

On Mon, Aug 22, 2016 at 03:42:28PM +0200, Michal Hocko wrote:
> On Mon 22-08-16 09:31:14, Greg KH wrote:
> > On Mon, Aug 22, 2016 at 12:54:41PM +0200, Michal Hocko wrote:
> > > On Mon 22-08-16 06:05:28, Greg KH wrote:
> > > > On Mon, Aug 22, 2016 at 11:37:07AM +0200, Michal Hocko wrote:
> > > [...]
> > > > > > From 899b738538de41295839dca2090a774bdd17acd2 Mon Sep 17 00:00:00 2001
> > > > > > From: Michal Hocko <mhocko@suse.com>
> > > > > > Date: Mon, 22 Aug 2016 10:52:06 +0200
> > > > > > Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
> > > > > >  order request
> > > > > > 
> > > > > > There have been several reports about pre-mature OOM killer invocation
> > > > > > in 4.7 kernel when order-2 allocation request (for the kernel stack)
> > > > > > invoked OOM killer even during basic workloads (light IO or even kernel
> > > > > > compile on some filesystems). In all reported cases the memory is
> > > > > > fragmented and there are no order-2+ pages available. There is usually
> > > > > > a large amount of slab memory (usually dentries/inodes) and further
> > > > > > debugging has shown that there are way too many unmovable blocks which
> > > > > > are skipped during the compaction. Multiple reporters have confirmed that
> > > > > > the current linux-next which includes [1] and [2] helped and OOMs are
> > > > > > not reproducible anymore. A simpler fix for the stable is to simply
> > > > > > ignore the compaction feedback and retry as long as there is a reclaim
> > > > > > progress for high order requests which we used to do before. We already
> > > > > > do that for CONFING_COMPACTION=n so let's reuse the same code when
> > > > > > compaction is enabled as well.
> > > > > > 
> > > > > > [1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
> > > > > > [2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz
> > > > > > 
> > > > > > Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
> > > > > > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > > > > > ---
> > > > > >  mm/page_alloc.c | 50 ++------------------------------------------------
> > > > > >  1 file changed, 2 insertions(+), 48 deletions(-)
> > > > 
> > > > So, if this goes into Linus's tree, can you let stable@vger.kernel.org
> > > > know about it so we can add it to the 4.7-stable tree?  Otherwise
> > > > there's not much I can do here now, right?
> > > 
> > > My plan would be actually to not push this to Linus because we have a
> > > proper fix for Linus tree. It is just that the fix is quite large and I
> > > felt like the stable should get the most simple fix possible, which is
> > > this partial revert. So, what I am trying to tell is to push a non-linus
> > > patch to stable as it is simpler.
> > 
> > I _REALLY_ hate taking any patches that are not in Linus's tree as 90%
> > of the time (well, almost always), it ends up being wrong and hurting us
> > in the end.
> 
> I do not like it either but if there is a simple and straightforward
> workaround for stable while the upstream can go with the _proper_ fix
> from the longer POV then I think this is perfectly justified. Stable
> should be always about the simplest fix for the problem IMHO.

No, stable should always be "what is in Linus's tree to get it fixed."

Again, almost every time we try to "just do this simple thing instead"
in a stable tree, it ends up being broken somehow.  We have the history
to back this up, look at our archives.

I'll gladly take 10+ patches to resolve something, _if_ it actually
resolves something.

But, if we argue about it for a month or so, then we don't have to worry
about it as everyone will be using 4.8 :)

> Of course, if Linus/Andrew doesn't like to take those compaction
> improvements this late then I will ask to merge the partial revert to
> Linus tree as well and then there is not much to discuss.

Ok, let me know how it goes and we can see what to do.

thanks.

greg k-h

[toc] | [prev] | [next] | [standalone]

#1468089

On Mon, 22 Aug 2016 15:42:28 +0200 Michal Hocko <mhocko@kernel.org> wrote:

> Of course, if Linus/Andrew doesn't like to take those compaction
> improvements this late then I will ask to merge the partial revert to
> Linus tree as well and then there is not much to discuss.

This sounds like the prudent option.  Can we get 4.8 working
well-enough, backport that into 4.7.x and worry about the fancier stuff
for 4.9?

[toc] | [prev] | [next] | [standalone]

#1468407

On Mon 22-08-16 15:05:17, Andrew Morton wrote:
> On Mon, 22 Aug 2016 15:42:28 +0200 Michal Hocko <mhocko@kernel.org> wrote:
> 
> > Of course, if Linus/Andrew doesn't like to take those compaction
> > improvements this late then I will ask to merge the partial revert to
> > Linus tree as well and then there is not much to discuss.
> 
> This sounds like the prudent option.  Can we get 4.8 working
> well-enough, backport that into 4.7.x and worry about the fancier stuff
> for 4.9?

OK, fair enough.

I would really appreciate if the original reporters could retest with
this patch on top of the current Linus tree. The stable backport posted
earlier doesn't apply on the current master cleanly but the change is
essentially same. mmotm tree then can revert this patch before Vlastimil
series is applied because that code is touching the currently removed
code.
---
From 90b6b282bede7966fb6c830a6d012d2239ac40e4 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.com>
Date: Mon, 22 Aug 2016 10:52:06 +0200
Subject: [PATCH] mm, oom: prevent pre-mature OOM killer invocation for high
 order request

There have been several reports about pre-mature OOM killer invocation
in 4.7 kernel when order-2 allocation request (for the kernel stack)
invoked OOM killer even during basic workloads (light IO or even kernel
compile on some filesystems). In all reported cases the memory is
fragmented and there are no order-2+ pages available. There is usually
a large amount of slab memory (usually dentries/inodes) and further
debugging has shown that there are way too many unmovable blocks which
are skipped during the compaction. Multiple reporters have confirmed that
the current linux-next which includes [1] and [2] helped and OOMs are
not reproducible anymore.

A simpler fix for the late rc and stable is to simply ignore the
compaction feedback and retry as long as there is a reclaim progress
and we are not getting OOM for order-0 pages. We already do that for
CONFING_COMPACTION=n so let's reuse the same code when compaction is
enabled as well.

[1] http://lkml.kernel.org/r/20160810091226.6709-1-vbabka@suse.cz
[2] http://lkml.kernel.org/r/f7a9ea9d-bb88-bfd6-e340-3a933559305a@suse.cz

Fixes: 0a0337e0d1d1 ("mm, oom: rework oom detection")
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/page_alloc.c | 51 ++-------------------------------------------------
 1 file changed, 2 insertions(+), 49 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3fbe73a6fe4b..7791a03f8deb 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3137,54 +3137,6 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 	return NULL;
 }
 
-static inline bool
-should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
-		     enum compact_result compact_result,
-		     enum compact_priority *compact_priority,
-		     int compaction_retries)
-{
-	int max_retries = MAX_COMPACT_RETRIES;
-
-	if (!order)
-		return false;
-
-	/*
-	 * compaction considers all the zone as desperately out of memory
-	 * so it doesn't really make much sense to retry except when the
-	 * failure could be caused by insufficient priority
-	 */
-	if (compaction_failed(compact_result)) {
-		if (*compact_priority > MIN_COMPACT_PRIORITY) {
-			(*compact_priority)--;
-			return true;
-		}
-		return false;
-	}
-
-	/*
-	 * make sure the compaction wasn't deferred or didn't bail out early
-	 * due to locks contention before we declare that we should give up.
-	 * But do not retry if the given zonelist is not suitable for
-	 * compaction.
-	 */
-	if (compaction_withdrawn(compact_result))
-		return compaction_zonelist_suitable(ac, order, alloc_flags);
-
-	/*
-	 * !costly requests are much more important than __GFP_REPEAT
-	 * costly ones because they are de facto nofail and invoke OOM
-	 * killer to move on while costly can fail and users are ready
-	 * to cope with that. 1/4 retries is rather arbitrary but we
-	 * would need much more detailed feedback from compaction to
-	 * make a better decision.
-	 */
-	if (order > PAGE_ALLOC_COSTLY_ORDER)
-		max_retries /= 4;
-	if (compaction_retries <= max_retries)
-		return true;
-
-	return false;
-}
 #else
 static inline struct page *
 __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
@@ -3195,6 +3147,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 	return NULL;
 }
 
+#endif /* CONFIG_COMPACTION */
+
 static inline bool
 should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_flags,
 		     enum compact_result compact_result,
@@ -3221,7 +3175,6 @@ should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_fla
 	}
 	return false;
 }
-#endif /* CONFIG_COMPACTION */
 
 /* Perform direct synchronous page reclaim */
 static int
-- 
2.8.1


-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1469907

[Multipart message — attachments visible in raw view] — view raw

On Thu, Aug 25, Michal Hocko wrote:

> Any luck with the testing of this patch?

Not this week, sorry.

Olaf

[toc] | [prev] | [next] | [standalone]

#1469918

On Tue 23-08-16 09:43:39, Michal Hocko wrote:
> On Mon 22-08-16 15:05:17, Andrew Morton wrote:
> > On Mon, 22 Aug 2016 15:42:28 +0200 Michal Hocko <mhocko@kernel.org> wrote:
> > 
> > > Of course, if Linus/Andrew doesn't like to take those compaction
> > > improvements this late then I will ask to merge the partial revert to
> > > Linus tree as well and then there is not much to discuss.
> > 
> > This sounds like the prudent option.  Can we get 4.8 working
> > well-enough, backport that into 4.7.x and worry about the fancier stuff
> > for 4.9?
> 
> OK, fair enough.
> 
> I would really appreciate if the original reporters could retest with
> this patch on top of the current Linus tree.

Any luck with the testing of this patch?
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web