Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically

From	Johannes Weiner <hannes@cmpxchg.org>
Newsgroups	linux.kernel
Subject	Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically
Date	2016-12-16 18:40 +0100
Message-ID	<sP4E9-6Gh-1@gated-at.bofh.it> (permalink)
References	<sOVrc-uT-5@gated-at.bofh.it> <sP35n-5AA-5@gated-at.bofh.it> <sP35n-5AA-23@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Fri, Dec 16, 2016 at 04:58:08PM +0100, Michal Hocko wrote:
> @@ -1013,7 +1013,7 @@ bool out_of_memory(struct oom_control *oc)
>  	 * make sure exclude 0 mask - all other users should have at least
>  	 * ___GFP_DIRECT_RECLAIM to get here.
>  	 */
> -	if (oc->gfp_mask && !(oc->gfp_mask & (__GFP_FS|__GFP_NOFAIL)))
> +	if (oc->gfp_mask && !(oc->gfp_mask & __GFP_FS))
>  		return true;

This makes sense, we should go back to what we had here. Because it's
not that the reported OOMs are premature - there is genuinely no more
memory reclaimable from the allocating context - but that this class
of allocations should never invoke the OOM killer in the first place.

> @@ -3737,6 +3752,16 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  		 */
>  		WARN_ON_ONCE(order > PAGE_ALLOC_COSTLY_ORDER);
>  
> +		/*
> +		 * Help non-failing allocations by giving them access to memory
> +		 * reserves but do not use ALLOC_NO_WATERMARKS because this
> +		 * could deplete whole memory reserves which would just make
> +		 * the situation worse
> +		 */
> +		page = __alloc_pages_cpuset_fallback(gfp_mask, order, ALLOC_HARDER, ac);
> +		if (page)
> +			goto got_pg;
> +

But this should be a separate patch, IMO.

Do we observe GFP_NOFS lockups when we don't do this? Don't we risk
premature exhaustion of the memory reserves, and it's better to wait
for other reclaimers to make some progress instead? Should we give
reserve access to all GFP_NOFS allocations, or just the ones from a
reclaim/cleaning context? All that should go into the changelog of a
separate allocation booster patch, I think.

Thread

OOM: Better, but still there on 4.9 Nils Holland <nholland@tisys.org> - 2016-12-16 00:10 +0100
  Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-16 08:50 +0100
    Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
      [PATCH 1/2] mm: consolidate GFP_NOFAIL checks in the allocator slowpath Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
      [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
        Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Johannes Weiner <hannes@cmpxchg.org> - 2016-12-16 18:40 +0100
          Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Michal Hocko <mhocko@kernel.org> - 2016-12-16 23:20 +0100
            Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-17 12:20 +0100
              Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Michal Hocko <mhocko@kernel.org> - 2016-12-18 17:40 +0100
      Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-17 01:10 +0100
        Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 14:10 +0100
          Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-17 15:50 +0100
            Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 18:20 +0100
            Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 22:10 +0100
              Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-18 06:20 +0100
              Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-19 14:50 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-20 03:10 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-21 08:40 +0100
                Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-21 12:10 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-21 12:20 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 11:20 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-22 11:30 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 11:40 +0100
                Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-22 11:50 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-22 20:20 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 22:50 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-23 12:00 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-23 13:20 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-23 14:00 +0100
                [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-23 15:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-23 23:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-26 13:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-26 20:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 09:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 12:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-27 12:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 17:00 +0100
                Re: [PATCH] mm, vmscan: consider eligible zones in get_scan_count kbuild test robot <lkp@intel.com> - 2016-12-27 17:30 +0100
                Re: [PATCH] mm, vmscan: consider eligible zones in get_scan_count Michal Hocko <mhocko@kernel.org> - 2016-12-28 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-27 20:40 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-28 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 02:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-29 10:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-30 03:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-30 11:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 01:40 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 02:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-29 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Mel Gorman <mgorman@suse.de> - 2016-12-30 11:20 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-30 12:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Mel Gorman <mgorman@suse.de> - 2016-12-30 13:50 +0100
                Re: [lkp-developer] [mm, memcg]  d18e2b2aca:  WARNING:at_mm/memcontrol.c:#mem_cgroup_update_lru_size Michal Hocko <mhocko@kernel.org> - 2016-12-26 13:30 +0100
                Re: [lkp-developer] [mm, memcg]  d18e2b2aca:  WARNING:at_mm/memcontrol.c:#mem_cgroup_update_lru_size Michal Hocko <mhocko@kernel.org> - 2016-12-26 14:00 +0100
            Re: OOM: Better, but still there on "Xin Zhou" <xin.zhou@gmx.com> - 2016-12-18 01:40 +0100
    Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-16 23:20 +0100
      Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-17 00:40 +0100

csiph-web