[PATCH 0/9] mm: kswapd spinning on unreclaimable nodes - fixes and cleanups

From	Johannes Weiner <hannes@cmpxchg.org>
Newsgroups	linux.kernel
Subject	[PATCH 0/9] mm: kswapd spinning on unreclaimable nodes - fixes and cleanups
Date	2017-03-01 01:00 +0100
Message-ID	<tfZQt-2Vc-9@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi,

Jia reported a scenario in which the kswapd of a node indefinitely
spins at 100% CPU usage. We have seen similar cases at Facebook.

The kernel's current method of judging its ability to reclaim a node
(or whether to back off and sleep) is based on the amount of scanned
pages in proportion to the amount of reclaimable pages. In Jia's and
our scenarios, there are no reclaimable pages in the node, however,
and the condition for backing off is never met. Kswapd busyloops in an
attempt to restore the watermarks while having nothing to work with.

This series reworks the definition of an unreclaimable node based not
on scanning but on whether kswapd is able to actually reclaim pages in
MAX_RECLAIM_RETRIES (16) consecutive runs. This is the same criteria
the page allocator uses for giving up on direct reclaim and invoking
the OOM killer. If it cannot free any pages, kswapd will go to sleep
and leave further attempts to direct reclaim invocations, which will
either make progress and re-enable kswapd, or invoke the OOM killer.

Patch #1 fixes the immediate problem Jia reported, the remainder are
smaller fixlets, cleanups, and overall phasing out of the old method.

Patch #6 is the odd one out. It's a nice cleanup to get_scan_count(),
and directly related to #5, but in itself not relevant to the series.

If the whole series is too ambitious for 4.11, I would consider the
first three patches fixes, the rest cleanups.

Thanks

 include/linux/mmzone.h |   3 +-
 mm/internal.h          |   7 +-
 mm/migrate.c           |   3 -
 mm/page_alloc.c        |  39 +++--------
 mm/vmscan.c            | 169 ++++++++++++++++++-----------------------------
 mm/vmstat.c            |  24 ++-----
 6 files changed, 89 insertions(+), 156 deletions(-)

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[PATCH 0/9] mm: kswapd spinning on unreclaimable nodes - fixes and cleanups Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 01:00 +0100
  [PATCH 4/9] mm: remove unnecessary reclaimability check from NUMA balancing target Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 01:10 +0100
    Re: [PATCH 4/9] mm: remove unnecessary reclaimability check from  NUMA balancing target Michal Hocko <mhocko@kernel.org> - 2017-03-01 20:30 +0100
    Re: [PATCH 4/9] mm: remove unnecessary reclaimability check from NUMA balancing target "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-02 04:40 +0100
  [PATCH 8/9] Revert "mm, vmscan: account for skipped pages as a partial scan" Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 01:10 +0100
    Re: [PATCH 8/9] Revert "mm, vmscan: account for skipped pages as a  partial scan" Michal Hocko <mhocko@kernel.org> - 2017-03-01 17:00 +0100
  [PATCH 3/9] mm: remove seemingly spurious reclaimability check from laptop_mode gating Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 09:00 +0100
    Re: [PATCH 3/9] mm: remove seemingly spurious reclaimability check  from laptop_mode gating Mel Gorman <mgorman@suse.de> - 2017-03-01 16:20 +0100
    Re: [PATCH 3/9] mm: remove seemingly spurious reclaimability check  from laptop_mode gating Michal Hocko <mhocko@kernel.org> - 2017-03-01 16:50 +0100
    Re: [PATCH 3/9] mm: remove seemingly spurious reclaimability check from laptop_mode gating "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-02 04:30 +0100
  [PATCH 6/9] mm: don't avoid high-priority reclaim on memcg limit reclaim Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 09:00 +0100
    Re: [PATCH 6/9] mm: don't avoid high-priority reclaim on memcg limit  reclaim Michal Hocko <mhocko@kernel.org> - 2017-03-01 17:00 +0100
      Re: [PATCH 6/9] mm: don't avoid high-priority reclaim on memcg limit  reclaim Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 19:30 +0100
        Re: [PATCH 6/9] mm: don't avoid high-priority reclaim on memcg limit  reclaim Michal Hocko <mhocko@kernel.org> - 2017-03-01 20:20 +0100
    Re: [PATCH 6/9] mm: don't avoid high-priority reclaim on memcg limit reclaim "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-02 04:40 +0100
  [PATCH 2/9] mm: fix check for reclaimable pages in PF_MEMALLOC reclaim throttling Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 09:00 +0100
    Re: [PATCH 2/9] mm: fix check for reclaimable pages in PF_MEMALLOC  reclaim throttling Michal Hocko <mhocko@kernel.org> - 2017-03-01 16:10 +0100
    Re: [PATCH 2/9] mm: fix check for reclaimable pages in PF_MEMALLOC reclaim throttling "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-02 04:30 +0100
  [PATCH 7/9] mm: delete NR_PAGES_SCANNED and pgdat_reclaimable() Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 09:00 +0100
    Re: [PATCH 7/9] mm: delete NR_PAGES_SCANNED and pgdat_reclaimable() Michal Hocko <mhocko@kernel.org> - 2017-03-01 16:50 +0100
  [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable nodes Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 09:00 +0100
    Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable nodes "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-02 04:30 +0100
    Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable nodes Shakeel Butt <shakeelb@google.com> - 2017-03-03 02:40 +0100
    Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Minchan Kim <minchan@kernel.org> - 2017-03-03 02:50 +0100
      Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Michal Hocko <mhocko@kernel.org> - 2017-03-03 09:10 +0100
        Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Minchan Kim <minchan@kernel.org> - 2017-03-06 02:40 +0100
          Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Johannes Weiner <hannes@cmpxchg.org> - 2017-03-06 17:40 +0100
            Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable nodes "Hillf Danton" <hillf.zj@alibaba-inc.com> - 2017-03-07 02:10 +0100
            Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Minchan Kim <minchan@kernel.org> - 2017-03-07 08:50 +0100
            Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Michal Hocko <mhocko@kernel.org> - 2017-03-07 11:30 +0100
              Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Johannes Weiner <hannes@cmpxchg.org> - 2017-03-07 20:00 +0100
                Re: [PATCH 1/9] mm: fix 100% CPU kswapd busyloop on unreclaimable  nodes Mel Gorman <mgorman@suse.de> - 2017-03-09 15:40 +0100
  [PATCH 9/9] mm: remove unnecessary back-off function when retrying page reclaim Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 15:00 +0100
    Re: [PATCH 9/9] mm: remove unnecessary back-off function when  retrying page reclaim Michal Hocko <mhocko@kernel.org> - 2017-03-01 16:50 +0100
  [PATCH 5/9] mm: don't avoid high-priority reclaim on unreclaimable nodes Johannes Weiner <hannes@cmpxchg.org> - 2017-03-01 15:00 +0100
    Re: [PATCH 5/9] mm: don't avoid high-priority reclaim on  unreclaimable nodes Michal Hocko <mhocko@kernel.org> - 2017-03-01 17:00 +0100

csiph-web