Re: computing drop-able caches

From	Johannes Weiner <hannes@cmpxchg.org>
Newsgroups	linux.kernel
Subject	Re: computing drop-able caches
Date	2016-02-12 23:20 +0100
Message-ID	<r1ued-4Bn-7@gated-at.bofh.it> (permalink)
References	(5 earlier) <r0HwS-5Rl-21@gated-at.bofh.it> <r0IsV-6rW-1@gated-at.bofh.it> <r17KG-6xV-1@gated-at.bofh.it> <r1qki-24n-11@gated-at.bofh.it> <r1tLe-49e-23@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Fri, Feb 12, 2016 at 01:46:54PM -0800, Dave Hansen wrote:
> On 02/12/2016 10:01 AM, Khalid Mughal (khalidm) wrote:
> > If you look at the attached pdf, you will notice that OOM messages start
> > to appear when memAvailable is showing 253MB (259228 KB) Free, memFree is
> > 13.5MB (14008 KB) Free, and dropcache based calculation ³Available memory²
> > is showing 21MB (21720 KB) Free.
> > 
> > So, it appears that memAvailable is not as accurate, especially if data is
> > used to warn user about system running low on memory.
> 
> Yep, that's true.
> 
> But, MemAvailable is calculated from some very cheap counters.  The
> "dropcache-based-calculation" requires iterating over every 4k page
> cache page in the system.

It's also completely off when the dominating cache consumer uses
mmap() instead of buffered IO.

> We track dirty and writebackw with counters, so we should theoretically
> be able to pull those out of MemAvailable fairly cheaply.

Yeah, that sounds like a good idea.

But the fact remains that this will never be fully accurate, and there
will always be scenarios where the system will thrash and trigger OOM
before this counter depletes, simply because maintaining uptodate heat
information of the page cache would be crazy expensive.

IMO, the best way to use the MemAvailable counter is to calculate a
utilization percentage against MemTotal, and then maintaining a
healthy number like 80-90% - depending on expected runtime variance
and an educated guess of how hot the page cache is.

Back to linux.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Re: computing drop-able caches Daniel Walker <danielwa@cisco.com> - 2016-02-10 19:10 +0100
  Re: computing drop-able caches Dave Hansen <dave.hansen@intel.com> - 2016-02-10 19:20 +0100
    Re: computing drop-able caches Daniel Walker <danielwa@cisco.com> - 2016-02-10 20:20 +0100
      Re: computing drop-able caches Rik van Riel <riel@redhat.com> - 2016-02-11 23:20 +0100
        Re: computing drop-able caches Dave Hansen <dave.hansen@intel.com> - 2016-02-12 22:50 +0100
          Re: computing drop-able caches Johannes Weiner <hannes@cmpxchg.org> - 2016-02-12 23:20 +0100

csiph-web