Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1376066

Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to helpers

Show all headers | View raw

On Mon 11-04-16 16:39:21, Vlastimil Babka wrote:
> On 04/05/2016 01:25 PM, Michal Hocko wrote:
[...]
> >+/* Compaction has failed and it doesn't make much sense to keep retrying. */
> >+static inline bool compaction_failed(enum compact_result result)
> >+{
> >+	/* All zones where scanned completely and still not result. */
> 
> Hmm given that try_to_compact_pages() uses a max() on results, then in fact
> it takes only one zone to get this. Others could have been also SKIPPED or
> DEFERRED. Is that what you want?

In short I didn't find any better way and still guarantee a some
guarantee of convergence. COMPACT_COMPLETE means that at least one zone
was completely scanned and led to no result. That zone would be
compact_suitable by definition. If I made DEFERRED or SKIPPED more
priorite (aka higher in the enum) then I could easily end up in a state
where all zones would return COMPACT_COMPLETE and few remaining would
just alternate returning their DEFFERED resp. SKIPPED. So while this
might sound like giving up too early I couldn't come up with anything
more specific that would lead to reliable results.

I am open to any suggestions of course.

[...]
> >--- a/mm/page_alloc.c
> >+++ b/mm/page_alloc.c
> >@@ -3362,25 +3362,12 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
> >  	if (page)
> >  		goto got_pg;
> >
> >-	/* Checks for THP-specific high-order allocations */
> >-	if (is_thp_gfp_mask(gfp_mask)) {
> >-		/*
> >-		 * If compaction is deferred for high-order allocations, it is
> >-		 * because sync compaction recently failed. If this is the case
> >-		 * and the caller requested a THP allocation, we do not want
> >-		 * to heavily disrupt the system, so we fail the allocation
> >-		 * instead of entering direct reclaim.
> >-		 */
> >-		if (compact_result == COMPACT_DEFERRED)
> >-			goto nopage;
> >-
> >-		/*
> >-		 * Compaction is contended so rather back off than cause
> >-		 * excessive stalls.
> >-		 */
> >-		if(compact_result == COMPACT_CONTENDED)
> >-			goto nopage;
> >-	}
> >+	/*
> >+	 * Checks for THP-specific high-order allocations and back off
> >+	 * if the the compaction backed off
> >+	 */
> >+	if (is_thp_gfp_mask(gfp_mask) && compaction_withdrawn(compact_result))
> >+		goto nopage;
> 
> The change of semantics for THP is not trivial here and should at least be
> discussed in changelog. CONTENDED and DEFERRED is only subset of
> compaction_withdrawn() as seen above.

True. My main motivation was to get rid of the compaction specific code
from the allocator path as much as possible. I can drop the above hunk
of course but I think we should get rid of these checks and make the
code simpler. To be honest I am not even sure those changes are really
measurable.

> Why is it useful to back off due to
> COMPACT_PARTIAL_SKIPPED (we were just unlucky in our starting position), but
> not due to COMPACT_COMPLETE (we have seen the whole zone but failed anyway)?

OK, that is a good remark. I could change that to:
	if (is_thp_gfp_mask(gfp_mask) &&
		(compaction_withdrawn(compact_result) || compaction_failed(compact_result))

> Why back off due to COMPACT_SKIPPED (not enough order-0 pages) without
> trying reclaim at least once, and then another async compaction, like
> before?

The idea was that COMPACT_SKIPPED wouldn't change after a single reclaim
round most of the time because a zone would have to get above
low_wmark + 1<<9 pages.  So the only situation where it would matter would be if
we had some order-9 pages available hidden by the min wmark and we would
reclaim enough to get above the above gap. I am not sure this is what we
really want in the first place. Increase the reclaim stalls when we are
getting under memory pressure.

Thanks!
-- 
Michal Hocko
SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 00/11] oom detection rework v5 Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
  [PATCH 10/11] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 10/11] mm, oom: protect !costly allocations some more Andrew Morton <akpm@linux-foundation.org> - 2016-04-06 02:10 +0200
      Re: [PATCH 10/11] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-04-06 11:30 +0200
    Re: [PATCH 10/11] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 16:50 +0200
  [PATCH 08/11] mm, compaction: Simplify __alloc_pages_direct_compact feedback interface Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 08/11] mm, compaction: Simplify  __alloc_pages_direct_compact feedback interface Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 16:00 +0200
  [PATCH 04/11] mm, compaction: cover all compaction mode in compact_zone Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
  [PATCH 05/11] mm, compaction: distinguish COMPACT_DEFERRED from COMPACT_SKIPPED Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 05/11] mm, compaction: distinguish COMPACT_DEFERRED from  COMPACT_SKIPPED Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 13:10 +0200
      Re: [PATCH 05/11] mm, compaction: distinguish COMPACT_DEFERRED from  COMPACT_SKIPPED Michal Hocko <mhocko@kernel.org> - 2016-04-11 13:30 +0200
  [PATCH 07/11] mm, compaction: Update compaction_result ordering Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 07/11] mm, compaction: Update compaction_result ordering Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 14:20 +0200
  [PATCH 02/11] mm: throttle on IO only when there are too many dirty and writeback pages Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
  [PATCH 01/11] mm, oom: rework oom detection Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
  [PATCH 03/11] mm, compaction: change COMPACT_ constants into enum Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
  [PATCH 06/11] mm, compaction: distinguish between full and partial COMPACT_COMPLETE Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 14:20 +0200
      Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Michal Hocko <mhocko@kernel.org> - 2016-04-11 14:50 +0200
        Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 15:00 +0200
          Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Michal Hocko <mhocko@kernel.org> - 2016-04-11 15:30 +0200
            Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 15:50 +0200
              Re: [PATCH 06/11] mm, compaction: distinguish between full and  partial COMPACT_COMPLETE Michal Hocko <mhocko@kernel.org> - 2016-04-11 15:50 +0200
  [PATCH 09/11] mm, compaction: Abstract compaction feedback to helpers Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Andrew Morton <akpm@linux-foundation.org> - 2016-04-06 02:00 +0200
      Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Hugh Dickins <hughd@google.com> - 2016-04-06 03:00 +0200
        Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Michal Hocko <mhocko@kernel.org> - 2016-04-06 11:30 +0200
        Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Andrew Morton <akpm@linux-foundation.org> - 2016-04-06 19:50 +0200
    Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 16:40 +0200
      Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Michal Hocko <mhocko@kernel.org> - 2016-04-11 17:20 +0200
        Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Michal Hocko <mhocko@kernel.org> - 2016-04-11 17:40 +0200
        Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Vlastimil Babka <vbabka@suse.cz> - 2016-04-12 14:00 +0200
          Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Michal Hocko <mhocko@kernel.org> - 2016-04-12 14:30 +0200
    Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Michal Hocko <mhocko@kernel.org> - 2016-04-11 18:00 +0200
      [RFC PATCH] mm: use compaction feedback for thp backoff conditions Michal Hocko <mhocko@kernel.org> - 2016-04-11 18:10 +0200
      Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to  helpers Vlastimil Babka <vbabka@suse.cz> - 2016-04-12 14:00 +0200
  [PATCH 11/11] mm: consider compaction feedback also for costly allocation Michal Hocko <mhocko@kernel.org> - 2016-04-05 13:30 +0200
    Re: [PATCH 11/11] mm: consider compaction feedback also for costly  allocation Michal Hocko <mhocko@kernel.org> - 2016-04-05 14:50 +0200
    Re: [PATCH 11/11] mm: consider compaction feedback also for costly  allocation Vlastimil Babka <vbabka@suse.cz> - 2016-04-11 17:10 +0200
  Re: [PATCH 00/11] oom detection rework v5 Michal Hocko <mhocko@kernel.org> - 2016-04-05 14:50 +0200

csiph-web