Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1352900

Re: [PATCH] mm, oom: protect !costly allocations some more

Show all headers | View raw

On 03/08/2016 11:10 AM, Michal Hocko wrote:
> On Tue 08-03-16 10:52:15, Vlastimil Babka wrote:
>> On 03/08/2016 10:46 AM, Michal Hocko wrote:
> [...]
>>>>> @@ -3294,6 +3289,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>>>>>  				 did_some_progress > 0, no_progress_loops))
>>>>>  		goto retry;
>>>>>  
>>>>> +	/*
>>>>> +	 * !costly allocations are really important and we have to make sure
>>>>> +	 * the compaction wasn't deferred or didn't bail out early due to locks
>>>>> +	 * contention before we go OOM.
>>>>> +	 */
>>>>> +	if (order && order <= PAGE_ALLOC_COSTLY_ORDER) {
>>>>> +		if (compact_result <= COMPACT_CONTINUE)
>>>>
>>>> Same here.
>>>> I was going to say that this didn't have effect on Sergey's test, but
>>>> turns out it did :)
>>>
>>> This should work as expected because compact_result is unsigned long
>>> and so this is the unsigned arithmetic. I can make
>>> #define COMPACT_NONE            -1UL
>>>
>>> to make the intention more obvious if you prefer, though.
>>
>> Well, what wasn't obvious to me is actually that here (unlike in the
>> test above) it was actually intended that COMPACT_NONE doesn't result in
>> a retry. But it makes sense, otherwise we would retry endlessly if
>> reclaim couldn't form a higher-order page, right.
> 
> Yeah, that was the whole point. An alternative would be moving the test
> into should_compact_retry(order, compact_result, contended_compaction)
> which would be CONFIG_COMPACTION specific so we can get rid of the
> COMPACT_NONE altogether. Something like the following. We would lose the
> always initialized compact_result but this would matter only for
> order==0 and we check for that. Even gcc doesn't complain.

Yeah I like this version better, you can add my Acked-By.

Thanks.

> A more important question is whether the criteria I have chosen are
> reasonable and reasonably independent on the particular implementation
> of the compaction. I still cannot convince myself about the convergence
> here. Is it possible that the compaction would keep returning 
> compact_result <= COMPACT_CONTINUE while not making any progress at all?

Theoretically, if reclaim/compaction suitability decisions and
allocation attempts didn't match the watermark checks, including the
alloc_flags and classzone_idx parameters. Possible scenarios:

- reclaim thinks compaction has enough to proceed, but compaction thinks
otherwise and returns COMPACT_SKIPPED
- compaction thinks it succeeded and returns COMPACT_PARTIAL, but
allocation attempt fails
- and perhaps some other combinations

> Sure we can see a case where somebody is stealing the compacted blocks
> but that is very same with the order-0 where parallel mem eaters will
> piggy back on the reclaimer and there is no upper boundary as well well.

Yep.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-07 17:10 +0100
  Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-03-08 04:50 +0100
    Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-08 10:10 +0100
      Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-03-08 10:30 +0100
  Re: [PATCH] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 10:30 +0100
    Re: [PATCH] mm, oom: protect !costly allocations some more Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-03-08 10:40 +0100
    Re: [PATCH] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-08 10:50 +0100
      Re: [PATCH] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 11:00 +0100
        Re: [PATCH] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-08 11:20 +0100
          Re: [PATCH] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 12:20 +0100
            Re: [PATCH] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-08 13:30 +0100
              Re: [PATCH] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 13:40 +0100
  Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-03-08 11:00 +0100
    Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-08 15:00 +0100
  Re: [PATCH] mm, oom: protect !costly allocations some more (was:  Re: [PATCH 0/3] OOM detection rework v4) Hugh Dickins <hughd@google.com> - 2016-03-08 11:40 +0100
  [PATCH 1/3] mm, compaction: change COMPACT_ constants into enum Michal Hocko <mhocko@kernel.org> - 2016-03-08 14:50 +0100
    Re: [PATCH 1/3] mm, compaction: change COMPACT_ constants into enum Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 15:20 +0100
    Re: [PATCH 1/3] mm, compaction: change COMPACT_ constants into enum "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2016-03-09 05:00 +0100
  [PATCH 3/3] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-08 14:50 +0100
    Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 15:40 +0100
      Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-08 15:50 +0100
        Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 16:10 +0100
    Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-09 12:20 +0100
      Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Vlastimil Babka <vbabka@suse.cz> - 2016-03-09 15:10 +0100
      Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Hugh Dickins <hughd@google.com> - 2016-03-11 13:20 +0100
        Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-11 14:10 +0100
          Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Hugh Dickins <hughd@google.com> - 2016-03-11 20:10 +0100
            Re: [PATCH 3/3] mm, oom: protect !costly allocations some more Michal Hocko <mhocko@kernel.org> - 2016-03-14 17:30 +0100
  [PATCH 0/2] oom rework: high order enahncements Michal Hocko <mhocko@kernel.org> - 2016-03-08 14:50 +0100
    [PATCH 2/3] mm, compaction: cover all compaction mode in compact_zone Michal Hocko <mhocko@kernel.org> - 2016-03-08 14:50 +0100
      Re: [PATCH 2/3] mm, compaction: cover all compaction mode in  compact_zone Vlastimil Babka <vbabka@suse.cz> - 2016-03-08 15:30 +0100
      Re: [PATCH 2/3] mm, compaction: cover all compaction mode in compact_zone "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2016-03-09 05:00 +0100
  Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Joonsoo Kim <js1304@gmail.com> - 2016-03-08 16:20 +0100
    Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-08 17:10 +0100
      Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Joonsoo Kim <js1304@gmail.com> - 2016-03-08 18:10 +0100
        Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-09 11:50 +0100
          Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Joonsoo Kim <js1304@gmail.com> - 2016-03-11 16:00 +0100
            Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re:  [PATCH 0/3] OOM detection rework v4) Michal Hocko <mhocko@kernel.org> - 2016-03-11 16:30 +0100

csiph-web