Path: csiph.com!feeder.erje.net!2.us.feeder.erje.net!border1.nntp.ams1.giganews.com!nntp.giganews.com!news.panservice.it!bofh.it!news.nic.it!robomod From: Tetsuo Handa Newsgroups: linux.kernel Subject: Re: [PATCH 0/3] OOM detection rework v4 Date: Sun, 13 Mar 2016 15:50:01 +0100 Message-ID: References: X-Virus-Status: clean(F-Secure/fsigk_smtp/522/fsav305.sakura.ne.jp) X-Mailer: Winbiff [Version 2.51 PL2] X-Accept-Language: ja,en,zh MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: robomod@news.nic.it List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Approved: robomod@news.nic.it Lines: 44 Organization: linux.* mail to news gateway X-Original-Cc: akpm@linux-foundation.org, torvalds@linux-foundation.org, hannes@cmpxchg.org, mgorman@suse.de, rientjes@google.com, hillf.zj@alibaba-inc.com, kamezawa.hiroyu@jp.fujitsu.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Original-Date: Sun, 13 Mar 2016 23:41:43 +0900 X-Original-Message-ID: <201603132341.JCI39084.OFOLFVHMOSJFtQ@I-love.SAKURA.ne.jp> X-Original-References: <20160311152851.GU27701@dhcp22.suse.cz> <201603120149.JEI86913.JVtSOOFHMFFQOL@I-love.SAKURA.ne.jp> <20160311170022.GX27701@dhcp22.suse.cz> <201603120220.GFJ00000.QOLVOtJOMFFSHF@I-love.SAKURA.ne.jp> <201603121308.FEH04174.OFHLOFMSJOtQVF@I-love.SAKURA.ne.jp> X-Original-Sender: linux-kernel-owner@vger.kernel.org Xref: csiph.com linux.kernel:1356687 Tetsuo Handa wrote: > Michal Hocko wrote: > > OK, that would suggest that the oom rework patches are not really > > related. They just moved from the livelock to a sleep which is good in > > general IMHO. We even know that it is most probably the IO that is the > > problem because we know that more than half of the reclaimable memory is > > either dirty or under writeback. That is where you should be looking. > > Why the IO is not making progress or such a slow progress. > > > > A footnote. Regarding this reproducer, the problem was "anybody can declare > OOM and call out_of_memory(). But out_of_memory() does nothing because there > is a thread which has TIF_MEMDIE." before the OOM detection rework patches, > and the problem is "nobody can declare OOM and call out_of_memory(). Although > out_of_memory() will do nothing because there is a thread which has > TIF_MEMDIE." after the OOM detection rework patches. According to kmallocwd, allocating tasks are very slowly able to call out_of_memory() ( http://I-love.SAKURA.ne.jp/tmp/serial-20160313.txt.xz ). It seems that the oom detection rework patches are not really related. > > Dave Chinner wrote at http://lkml.kernel.org/r/20160211225929.GU14668@dastard : > > > Although there are memory allocating tasks passing gfp flags with > > > __GFP_KSWAPD_RECLAIM, kswapd is unable to make forward progress because > > > it is blocked at down() called from memory reclaim path. And since it is > > > legal to block kswapd from memory reclaim path (am I correct?), I think > > > we must not assume that current_is_kswapd() check will break the infinite > > > loop condition. > > > > Right, the threads that are blocked in writeback waiting on memory > > reclaim will be using GFP_NOFS to prevent recursion deadlocks, but > > that does not avoid the problem that kswapd can then get stuck > > on those locks, too. Hence there is no guarantee that kswapd can > > make reclaim progress if it does dirty page writeback... > > Unless we address the issue Dave commented, the OOM detection rework patches > add a new location of livelock (which is demonstrated by this reproducer) in > the memory allocator. It is an unfortunate change that we add a new location > of livelock when we are trying to solve thrashing problem. > The oom detection rework patches did not add a new location of livelock. They just did not address the problem that I/O cannot make progress.