Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically

From	Michal Hocko <mhocko@kernel.org>
Newsgroups	linux.kernel
Subject	Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically
Date	2016-12-18 17:40 +0100
Message-ID	<sPMFb-2zL-3@gated-at.bofh.it> (permalink)
References	(1 earlier) <sP35n-5AA-5@gated-at.bofh.it> <sP35n-5AA-23@gated-at.bofh.it> <sP4E9-6Gh-1@gated-at.bofh.it> <sP919-11E-43@gated-at.bofh.it> <sPlbX-tx-1@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Sat 17-12-16 20:17:07, Tetsuo Handa wrote:
[...]
> I feel that allowing access to memory reserves based on __GFP_NOFAIL might not
> make sense. My understanding is that actual I/O operation triggered by I/O
> requests by filesystem code are processed by other threads. Even if we grant
> access to memory reserves to GFP_NOFS | __GFP_NOFAIL allocations by fs code,
> I think that it is possible that memory allocations by underlying bio code
> fails to make a further progress unless memory reserves are granted as well.

IO layer should rely on mempools to guarantee a forward progress.

> Below is a typical trace which I observe under OOM lockuped situation (though
> this trace is from an OOM stress test using XFS).
> 
> ----------------------------------------
> [ 1845.187246] MemAlloc: kworker/2:1(14498) flags=0x4208060 switches=323636 seq=48 gfp=0x2400000(GFP_NOIO) order=0 delay=430400 uninterruptible
> [ 1845.187248] kworker/2:1     D12712 14498      2 0x00000080
> [ 1845.187251] Workqueue: events_freezable_power_ disk_events_workfn
> [ 1845.187252] Call Trace:
> [ 1845.187253]  ? __schedule+0x23f/0xba0
> [ 1845.187254]  schedule+0x38/0x90
> [ 1845.187255]  schedule_timeout+0x205/0x4a0
> [ 1845.187256]  ? del_timer_sync+0xd0/0xd0
> [ 1845.187257]  schedule_timeout_uninterruptible+0x25/0x30
> [ 1845.187258]  __alloc_pages_nodemask+0x1035/0x10e0
> [ 1845.187259]  ? alloc_request_struct+0x14/0x20
> [ 1845.187261]  alloc_pages_current+0x96/0x1b0
> [ 1845.187262]  ? bio_alloc_bioset+0x20f/0x2e0
> [ 1845.187264]  bio_copy_kern+0xc4/0x180
> [ 1845.187265]  blk_rq_map_kern+0x6f/0x120
> [ 1845.187268]  __scsi_execute.isra.23+0x12f/0x160
> [ 1845.187270]  scsi_execute_req_flags+0x8f/0x100
> [ 1845.187271]  sr_check_events+0xba/0x2b0 [sr_mod]
> [ 1845.187274]  cdrom_check_events+0x13/0x30 [cdrom]
> [ 1845.187275]  sr_block_check_events+0x25/0x30 [sr_mod]
> [ 1845.187276]  disk_check_events+0x5b/0x150
> [ 1845.187277]  disk_events_workfn+0x17/0x20
> [ 1845.187278]  process_one_work+0x1fc/0x750
> [ 1845.187279]  ? process_one_work+0x167/0x750
> [ 1845.187279]  worker_thread+0x126/0x4a0
> [ 1845.187280]  kthread+0x10a/0x140
> [ 1845.187281]  ? process_one_work+0x750/0x750
> [ 1845.187282]  ? kthread_create_on_node+0x60/0x60
> [ 1845.187283]  ret_from_fork+0x2a/0x40
> ----------------------------------------
> 
> I think that this GFP_NOIO allocation request needs to consume more memory reserves
> than GFP_NOFS allocation request to make progress. 

AFAIU, this is an allocation path which doesn't block a forward progress
on a regular IO. It is merely a check whether there is a new medium in
the CDROM (aka regular polling of the device). I really fail to see any
reason why this one should get any access to memory reserves at all.

I actually do not see any reason why it should be NOIO in the first
place but I am not familiar with this code much so there might be some
reasons for that. The fact that it might stall under a heavy memory
pressure is sad but who actually cares?

> Do we want to add __GFP_NOFAIL to this GFP_NOIO allocation request
> in order to allow access to memory reserves as well as GFP_NOFS |
> __GFP_NOFAIL allocation request?

Why?

-- 
Michal Hocko
SUSE Labs

Thread

OOM: Better, but still there on 4.9 Nils Holland <nholland@tisys.org> - 2016-12-16 00:10 +0100
  Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-16 08:50 +0100
    Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
      [PATCH 1/2] mm: consolidate GFP_NOFAIL checks in the allocator slowpath Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
      [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically Michal Hocko <mhocko@kernel.org> - 2016-12-16 17:00 +0100
        Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Johannes Weiner <hannes@cmpxchg.org> - 2016-12-16 18:40 +0100
          Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Michal Hocko <mhocko@kernel.org> - 2016-12-16 23:20 +0100
            Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-17 12:20 +0100
              Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL  automatically Michal Hocko <mhocko@kernel.org> - 2016-12-18 17:40 +0100
      Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-17 01:10 +0100
        Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 14:10 +0100
          Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-17 15:50 +0100
            Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 18:20 +0100
            Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-17 22:10 +0100
              Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-18 06:20 +0100
              Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-19 14:50 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-20 03:10 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-21 08:40 +0100
                Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-21 12:10 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-21 12:20 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 11:20 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-22 11:30 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 11:40 +0100
                Re: OOM: Better, but still there on Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-12-22 11:50 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-22 20:20 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-22 22:50 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-23 12:00 +0100
                Re: OOM: Better, but still there on Nils Holland <nholland@tisys.org> - 2016-12-23 13:20 +0100
                Re: OOM: Better, but still there on Michal Hocko <mhocko@kernel.org> - 2016-12-23 14:00 +0100
                [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-23 15:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-23 23:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-26 13:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-26 20:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 09:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 12:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-27 12:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-27 17:00 +0100
                Re: [PATCH] mm, vmscan: consider eligible zones in get_scan_count kbuild test robot <lkp@intel.com> - 2016-12-27 17:30 +0100
                Re: [PATCH] mm, vmscan: consider eligible zones in get_scan_count Michal Hocko <mhocko@kernel.org> - 2016-12-28 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland <nholland@tisys.org> - 2016-12-27 20:40 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-28 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 02:30 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-29 10:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-30 03:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-30 11:50 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 01:40 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Minchan Kim <minchan@kernel.org> - 2016-12-29 02:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-29 10:00 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Mel Gorman <mgorman@suse.de> - 2016-12-30 11:20 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko <mhocko@kernel.org> - 2016-12-30 12:10 +0100
                Re: [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Mel Gorman <mgorman@suse.de> - 2016-12-30 13:50 +0100
                Re: [lkp-developer] [mm, memcg]  d18e2b2aca:  WARNING:at_mm/memcontrol.c:#mem_cgroup_update_lru_size Michal Hocko <mhocko@kernel.org> - 2016-12-26 13:30 +0100
                Re: [lkp-developer] [mm, memcg]  d18e2b2aca:  WARNING:at_mm/memcontrol.c:#mem_cgroup_update_lru_size Michal Hocko <mhocko@kernel.org> - 2016-12-26 14:00 +0100
            Re: OOM: Better, but still there on "Xin Zhou" <xin.zhou@gmx.com> - 2016-12-18 01:40 +0100
    Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-16 23:20 +0100
      Re: OOM: Better, but still there on 4.9 Michal Hocko <mhocko@kernel.org> - 2016-12-17 00:40 +0100

csiph-web