Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1415406 > unrolled thread

[PATCH 01/10] mm: allow swappiness that prefers anon over file

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 01/10] mm: allow swappiness that prefers anon over file Johannes Weiner <hannes@cmpxchg.org> - 2016-06-06 22:00 +0200
    Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Minchan Kim <minchan@kernel.org> - 2016-06-07 02:30 +0200
      Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Johannes Weiner <hannes@cmpxchg.org> - 2016-06-07 16:20 +0200
        Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Minchan Kim <minchan@kernel.org> - 2016-06-08 02:10 +0200
          Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Johannes Weiner <hannes@cmpxchg.org> - 2016-06-08 18:00 +0200
            Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Minchan Kim <minchan@kernel.org> - 2016-06-09 03:10 +0200
              Re: [PATCH 01/10] mm: allow swappiness that prefers anon over file Johannes Weiner <hannes@cmpxchg.org> - 2016-06-09 15:40 +0200

#1415406 — [PATCH 01/10] mm: allow swappiness that prefers anon over file

With the advent of fast random IO devices (SSDs, PMEM) and in-memory
swap devices such as zswap, it's possible for swap to be much faster
than filesystems, and for swapping to be preferable over thrashing
filesystem caches.

Allow setting swappiness - which defines the relative IO cost of cache
misses between page cache and swap-backed pages - to reflect such
situations by making the swap-preferred range configurable.

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt | 16 +++++++++++-----
 kernel/sysctl.c             |  3 ++-
 mm/vmscan.c                 |  2 +-
 3 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index 720355cbdf45..54030750cd31 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
 
 swappiness
 
-This control is used to define how aggressive the kernel will swap
-memory pages.  Higher values will increase agressiveness, lower values
-decrease the amount of swap.  A value of 0 instructs the kernel not to
-initiate swap until the amount of free and file-backed pages is less
-than the high water mark in a zone.
+This control is used to define the relative IO cost of cache misses
+between the swap device and the filesystem as a value between 0 and
+200. At 100, the VM assumes equal IO cost and will thus apply memory
+pressure to the page cache and swap-backed pages equally. At 0, the
+kernel will not initiate swap until the amount of free and file-backed
+pages is less than the high watermark in a zone.
 
 The default value is 60.
 
+On non-rotational swap devices, a value of 100 (or higher, depending
+on what's backing the filesystem) is recommended.
+
+For in-memory swap, like zswap, values closer to 200 are recommended.
+
 ==============================================================
 
 - user_reserve_kbytes
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 2effd84d83e3..56a9243eb171 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -126,6 +126,7 @@ static int __maybe_unused two = 2;
 static int __maybe_unused four = 4;
 static unsigned long one_ul = 1;
 static int one_hundred = 100;
+static int two_hundred = 200;
 static int one_thousand = 1000;
 #ifdef CONFIG_PRINTK
 static int ten_thousand = 10000;
@@ -1323,7 +1324,7 @@ static struct ctl_table vm_table[] = {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec_minmax,
 		.extra1		= &zero,
-		.extra2		= &one_hundred,
+		.extra2		= &two_hundred,
 	},
 #ifdef CONFIG_HUGETLB_PAGE
 	{
diff --git a/mm/vmscan.c b/mm/vmscan.c
index c4a2f4512fca..f79010bbcdd4 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -136,7 +136,7 @@ struct scan_control {
 #endif
 
 /*
- * From 0 .. 100.  Higher means more swappy.
+ * From 0 .. 200.  Higher means more swappy.
  */
 int vm_swappiness = 60;
 /*
-- 
2.8.3

[toc] | [next] | [standalone]

#1415597

Hi Johannes,

Thanks for the nice work. I didn't read all patchset yet but the design
makes sense to me so it would be better for zram-based on workload
compared to as is.

On Mon, Jun 06, 2016 at 03:48:27PM -0400, Johannes Weiner wrote:
> With the advent of fast random IO devices (SSDs, PMEM) and in-memory
> swap devices such as zswap, it's possible for swap to be much faster
> than filesystems, and for swapping to be preferable over thrashing
> filesystem caches.
> 
> Allow setting swappiness - which defines the relative IO cost of cache
> misses between page cache and swap-backed pages - to reflect such
> situations by making the swap-preferred range configurable.
> 
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> ---
>  Documentation/sysctl/vm.txt | 16 +++++++++++-----
>  kernel/sysctl.c             |  3 ++-
>  mm/vmscan.c                 |  2 +-
>  3 files changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
> index 720355cbdf45..54030750cd31 100644
> --- a/Documentation/sysctl/vm.txt
> +++ b/Documentation/sysctl/vm.txt
> @@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
>  
>  swappiness
>  
> -This control is used to define how aggressive the kernel will swap
> -memory pages.  Higher values will increase agressiveness, lower values
> -decrease the amount of swap.  A value of 0 instructs the kernel not to
> -initiate swap until the amount of free and file-backed pages is less
> -than the high water mark in a zone.
> +This control is used to define the relative IO cost of cache misses
> +between the swap device and the filesystem as a value between 0 and
> +200. At 100, the VM assumes equal IO cost and will thus apply memory
> +pressure to the page cache and swap-backed pages equally. At 0, the
> +kernel will not initiate swap until the amount of free and file-backed
> +pages is less than the high watermark in a zone.

Generally, I agree extending swappiness value good but not sure 200 is
enough to represent speed gap between file and swap sotrage in every
cases. - Just nitpick.

Some years ago, I extended it to 200 like your patch and experimented it
based on zram in our platform workload. At that time, it was terribly
slow in app switching workload if swappiness is higher than 150.
Although it was highly dependent on the workload, it's dangerous to
recommend it before fixing balacing between file and anon, I think.
IOW, I think this patch should be last one in this patchset.

>  
>  The default value is 60.
>  
> +On non-rotational swap devices, a value of 100 (or higher, depending
> +on what's backing the filesystem) is recommended.
> +
> +For in-memory swap, like zswap, values closer to 200 are recommended.

                maybe, like zram

I'm not sure it would be good suggestion for zswap because it ends up
writing cached pages to swap device once it reaches threshold.
Then, the cost is compression + decompression + write I/O which is
heavier than normal swap device(i.e., write I/O). OTOH, zram have no
(writeback I/O+ decompression) cost.

> +
>  ==============================================================
>  
>  - user_reserve_kbytes
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 2effd84d83e3..56a9243eb171 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -126,6 +126,7 @@ static int __maybe_unused two = 2;
>  static int __maybe_unused four = 4;
>  static unsigned long one_ul = 1;
>  static int one_hundred = 100;
> +static int two_hundred = 200;
>  static int one_thousand = 1000;
>  #ifdef CONFIG_PRINTK
>  static int ten_thousand = 10000;
> @@ -1323,7 +1324,7 @@ static struct ctl_table vm_table[] = {
>  		.mode		= 0644,
>  		.proc_handler	= proc_dointvec_minmax,
>  		.extra1		= &zero,
> -		.extra2		= &one_hundred,
> +		.extra2		= &two_hundred,
>  	},
>  #ifdef CONFIG_HUGETLB_PAGE
>  	{
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index c4a2f4512fca..f79010bbcdd4 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -136,7 +136,7 @@ struct scan_control {
>  #endif
>  
>  /*
> - * From 0 .. 100.  Higher means more swappy.
> + * From 0 .. 200.  Higher means more swappy.
>   */
>  int vm_swappiness = 60;
>  /*
> -- 
> 2.8.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[toc] | [prev] | [next] | [standalone]

#1416256

On Tue, Jun 07, 2016 at 09:25:50AM +0900, Minchan Kim wrote:
> Hi Johannes,
> 
> Thanks for the nice work. I didn't read all patchset yet but the design
> makes sense to me so it would be better for zram-based on workload
> compared to as is.

Thanks!

> On Mon, Jun 06, 2016 at 03:48:27PM -0400, Johannes Weiner wrote:
> > --- a/Documentation/sysctl/vm.txt
> > +++ b/Documentation/sysctl/vm.txt
> > @@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
> >  
> >  swappiness
> >  
> > -This control is used to define how aggressive the kernel will swap
> > -memory pages.  Higher values will increase agressiveness, lower values
> > -decrease the amount of swap.  A value of 0 instructs the kernel not to
> > -initiate swap until the amount of free and file-backed pages is less
> > -than the high water mark in a zone.
> > +This control is used to define the relative IO cost of cache misses
> > +between the swap device and the filesystem as a value between 0 and
> > +200. At 100, the VM assumes equal IO cost and will thus apply memory
> > +pressure to the page cache and swap-backed pages equally. At 0, the
> > +kernel will not initiate swap until the amount of free and file-backed
> > +pages is less than the high watermark in a zone.
> 
> Generally, I agree extending swappiness value good but not sure 200 is
> enough to represent speed gap between file and swap sotrage in every
> cases. - Just nitpick.

How so? You can't give swap more weight than 100%. 200 is the maximum
possible value.

> Some years ago, I extended it to 200 like your patch and experimented it
> based on zram in our platform workload. At that time, it was terribly
> slow in app switching workload if swappiness is higher than 150.
> Although it was highly dependent on the workload, it's dangerous to
> recommend it before fixing balacing between file and anon, I think.
> IOW, I think this patch should be last one in this patchset.

Good point. I'll tone down the recommendations. But OTOH it's a fairly
trivial patch, so I wouldn't want it to close after the current 10/10.

> >  The default value is 60.
> >  
> > +On non-rotational swap devices, a value of 100 (or higher, depending
> > +on what's backing the filesystem) is recommended.
> > +
> > +For in-memory swap, like zswap, values closer to 200 are recommended.
> 
>                 maybe, like zram
> 
> I'm not sure it would be good suggestion for zswap because it ends up
> writing cached pages to swap device once it reaches threshold.
> Then, the cost is compression + decompression + write I/O which is
> heavier than normal swap device(i.e., write I/O). OTOH, zram have no
> (writeback I/O+ decompression) cost.

Oh, good catch. Yeah, I'll change that for v2.

Thanks for your input, Minchan

[toc] | [prev] | [next] | [standalone]

#1416766

On Tue, Jun 07, 2016 at 10:18:18AM -0400, Johannes Weiner wrote:
> On Tue, Jun 07, 2016 at 09:25:50AM +0900, Minchan Kim wrote:
> > Hi Johannes,
> > 
> > Thanks for the nice work. I didn't read all patchset yet but the design
> > makes sense to me so it would be better for zram-based on workload
> > compared to as is.
> 
> Thanks!
> 
> > On Mon, Jun 06, 2016 at 03:48:27PM -0400, Johannes Weiner wrote:
> > > --- a/Documentation/sysctl/vm.txt
> > > +++ b/Documentation/sysctl/vm.txt
> > > @@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
> > >  
> > >  swappiness
> > >  
> > > -This control is used to define how aggressive the kernel will swap
> > > -memory pages.  Higher values will increase agressiveness, lower values
> > > -decrease the amount of swap.  A value of 0 instructs the kernel not to
> > > -initiate swap until the amount of free and file-backed pages is less
> > > -than the high water mark in a zone.
> > > +This control is used to define the relative IO cost of cache misses
> > > +between the swap device and the filesystem as a value between 0 and
> > > +200. At 100, the VM assumes equal IO cost and will thus apply memory
> > > +pressure to the page cache and swap-backed pages equally. At 0, the
> > > +kernel will not initiate swap until the amount of free and file-backed
> > > +pages is less than the high watermark in a zone.
> > 
> > Generally, I agree extending swappiness value good but not sure 200 is
> > enough to represent speed gap between file and swap sotrage in every
> > cases. - Just nitpick.
> 
> How so? You can't give swap more weight than 100%. 200 is the maximum
> possible value.

In old, swappiness is how agressively reclaim anonymous pages in favour
of page cache. But when I read your description and changes about
swappiness in vm.txt, esp, *relative IO cost*, I feel you change swappiness
define to represent relative IO cost between swap storage and file storage.
Then, with that, we could balance anonymous and file LRU with the weight.

For example, let's assume that in-memory swap storage is 10x times faster
than slow thumb drive. In that case, IO cost of 5 anonymous pages
swapping-in/out is equal to 1 file-backed page-discard/read.

I thought it does make sense because that measuring the speed gab between
those storages is easier than selecting vague swappiness tendency.

In terms of such approach, I thought 200 is not enough to show the gab
because the gap is started from 100.
Isn't it your intention? If so, to me, the description was rather
misleading. :(

[toc] | [prev] | [next] | [standalone]

#1417623

On Wed, Jun 08, 2016 at 09:06:32AM +0900, Minchan Kim wrote:
> On Tue, Jun 07, 2016 at 10:18:18AM -0400, Johannes Weiner wrote:
> > On Tue, Jun 07, 2016 at 09:25:50AM +0900, Minchan Kim wrote:
> > > On Mon, Jun 06, 2016 at 03:48:27PM -0400, Johannes Weiner wrote:
> > > > --- a/Documentation/sysctl/vm.txt
> > > > +++ b/Documentation/sysctl/vm.txt
> > > > @@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
> > > >  
> > > >  swappiness
> > > >  
> > > > -This control is used to define how aggressive the kernel will swap
> > > > -memory pages.  Higher values will increase agressiveness, lower values
> > > > -decrease the amount of swap.  A value of 0 instructs the kernel not to
> > > > -initiate swap until the amount of free and file-backed pages is less
> > > > -than the high water mark in a zone.
> > > > +This control is used to define the relative IO cost of cache misses
> > > > +between the swap device and the filesystem as a value between 0 and
> > > > +200. At 100, the VM assumes equal IO cost and will thus apply memory
> > > > +pressure to the page cache and swap-backed pages equally. At 0, the
> > > > +kernel will not initiate swap until the amount of free and file-backed
> > > > +pages is less than the high watermark in a zone.
> > > 
> > > Generally, I agree extending swappiness value good but not sure 200 is
> > > enough to represent speed gap between file and swap sotrage in every
> > > cases. - Just nitpick.
> > 
> > How so? You can't give swap more weight than 100%. 200 is the maximum
> > possible value.
> 
> In old, swappiness is how agressively reclaim anonymous pages in favour
> of page cache. But when I read your description and changes about
> swappiness in vm.txt, esp, *relative IO cost*, I feel you change swappiness
> define to represent relative IO cost between swap storage and file storage.
> Then, with that, we could balance anonymous and file LRU with the weight.
> 
> For example, let's assume that in-memory swap storage is 10x times faster
> than slow thumb drive. In that case, IO cost of 5 anonymous pages
> swapping-in/out is equal to 1 file-backed page-discard/read.
> 
> I thought it does make sense because that measuring the speed gab between
> those storages is easier than selecting vague swappiness tendency.
> 
> In terms of such approach, I thought 200 is not enough to show the gab
> because the gap is started from 100.
> Isn't it your intention? If so, to me, the description was rather
> misleading. :(

The way swappiness works never actually changed.

The only thing that changed is that we used to look at referenced
pages (recent_rotated) and *assumed* they would likely cause IO when
reclaimed, whereas with my patches we actually know whether they are.
But swappiness has always been about relative IO cost of the LRUs.

Swappiness defines relative IO cost between file and swap on a scale
from 0 to 200, where 100 is the point of equality. The scale factors
are calculated in get_scan_count() like this:

  anon_prio = swappiness
  file_prio = 200 - swappiness

and those are applied to the recorded cost/value ratios like this:

  ap = anon_prio * scanned / rotated
  fp = file_prio * scanned / rotated

That means if your swap device is 10 times faster than your filesystem
device, and you thus want anon to receive 10x the refaults when the
anon and file pages are used equally, you do this:

  x + 10x = 200
        x = 18 (ish)

So your file priority is ~18 and your swap priority is the remainder
of the range, 200 - 18. You set swappiness to 182.

Now fill in the numbers while assuming all pages on both lists have
been referenced before and will likely refault (or in the new model,
all pages are refaulting):

  fraction[anon] = ap      = 182 * 1 / 1 = 182
  fraction[file] = fp      =  18 * 1 / 1 =  18
     denominator = ap + fp =    182 + 18 = 200

and then calculate the scan target like this:

  scan[type] = (lru_size() >> priority) * fraction[type] / denominator

This will scan and reclaim 9% of the file pages and 90% of the anon
pages. On refault, 9% of the IO will be from the filesystem and 90%
from the swap device.

[toc] | [prev] | [next] | [standalone]

#1418000

On Wed, Jun 08, 2016 at 11:58:12AM -0400, Johannes Weiner wrote:
> On Wed, Jun 08, 2016 at 09:06:32AM +0900, Minchan Kim wrote:
> > On Tue, Jun 07, 2016 at 10:18:18AM -0400, Johannes Weiner wrote:
> > > On Tue, Jun 07, 2016 at 09:25:50AM +0900, Minchan Kim wrote:
> > > > On Mon, Jun 06, 2016 at 03:48:27PM -0400, Johannes Weiner wrote:
> > > > > --- a/Documentation/sysctl/vm.txt
> > > > > +++ b/Documentation/sysctl/vm.txt
> > > > > @@ -771,14 +771,20 @@ with no ill effects: errors and warnings on these stats are suppressed.)
> > > > >  
> > > > >  swappiness
> > > > >  
> > > > > -This control is used to define how aggressive the kernel will swap
> > > > > -memory pages.  Higher values will increase agressiveness, lower values
> > > > > -decrease the amount of swap.  A value of 0 instructs the kernel not to
> > > > > -initiate swap until the amount of free and file-backed pages is less
> > > > > -than the high water mark in a zone.
> > > > > +This control is used to define the relative IO cost of cache misses
> > > > > +between the swap device and the filesystem as a value between 0 and
> > > > > +200. At 100, the VM assumes equal IO cost and will thus apply memory
> > > > > +pressure to the page cache and swap-backed pages equally. At 0, the
> > > > > +kernel will not initiate swap until the amount of free and file-backed
> > > > > +pages is less than the high watermark in a zone.
> > > > 
> > > > Generally, I agree extending swappiness value good but not sure 200 is
> > > > enough to represent speed gap between file and swap sotrage in every
> > > > cases. - Just nitpick.
> > > 
> > > How so? You can't give swap more weight than 100%. 200 is the maximum
> > > possible value.
> > 
> > In old, swappiness is how agressively reclaim anonymous pages in favour
> > of page cache. But when I read your description and changes about
> > swappiness in vm.txt, esp, *relative IO cost*, I feel you change swappiness
> > define to represent relative IO cost between swap storage and file storage.
> > Then, with that, we could balance anonymous and file LRU with the weight.
> > 
> > For example, let's assume that in-memory swap storage is 10x times faster
> > than slow thumb drive. In that case, IO cost of 5 anonymous pages
> > swapping-in/out is equal to 1 file-backed page-discard/read.
> > 
> > I thought it does make sense because that measuring the speed gab between
> > those storages is easier than selecting vague swappiness tendency.
> > 
> > In terms of such approach, I thought 200 is not enough to show the gab
> > because the gap is started from 100.
> > Isn't it your intention? If so, to me, the description was rather
> > misleading. :(
> 
> The way swappiness works never actually changed.
> 
> The only thing that changed is that we used to look at referenced
> pages (recent_rotated) and *assumed* they would likely cause IO when
> reclaimed, whereas with my patches we actually know whether they are.
> But swappiness has always been about relative IO cost of the LRUs.
> 
> Swappiness defines relative IO cost between file and swap on a scale
> from 0 to 200, where 100 is the point of equality. The scale factors
> are calculated in get_scan_count() like this:
> 
>   anon_prio = swappiness
>   file_prio = 200 - swappiness
> 
> and those are applied to the recorded cost/value ratios like this:
> 
>   ap = anon_prio * scanned / rotated
>   fp = file_prio * scanned / rotated
> 
> That means if your swap device is 10 times faster than your filesystem
> device, and you thus want anon to receive 10x the refaults when the
> anon and file pages are used equally, you do this:
> 
>   x + 10x = 200
>         x = 18 (ish)
> 
> So your file priority is ~18 and your swap priority is the remainder
> of the range, 200 - 18. You set swappiness to 182.
> 
> Now fill in the numbers while assuming all pages on both lists have
> been referenced before and will likely refault (or in the new model,
> all pages are refaulting):
> 
>   fraction[anon] = ap      = 182 * 1 / 1 = 182
>   fraction[file] = fp      =  18 * 1 / 1 =  18
>      denominator = ap + fp =    182 + 18 = 200
> 
> and then calculate the scan target like this:
> 
>   scan[type] = (lru_size() >> priority) * fraction[type] / denominator
> 
> This will scan and reclaim 9% of the file pages and 90% of the anon
> pages. On refault, 9% of the IO will be from the filesystem and 90%
> from the swap device.

Thanks for the detail example. Then, let's change the example a little bit.

A system has big HDD storage and SSD swap.

HDD:    200 IOPS
SSD: 100000 IOPS
From https://en.wikipedia.org/wiki/IOPS

So, speed gap is 500x.
x + 500x = 200
If we use PCIe-SSD, the gap will be larger.
That's why I said 200 is enough to represent speed gap.
Such system configuration is already non-sense so it is okay to ignore such
usecases?

[toc] | [prev] | [next] | [standalone]

#1418333

On Thu, Jun 09, 2016 at 10:01:07AM +0900, Minchan Kim wrote:
> A system has big HDD storage and SSD swap.
> 
> HDD:    200 IOPS
> SSD: 100000 IOPS
> From https://en.wikipedia.org/wiki/IOPS
> 
> So, speed gap is 500x.
> x + 500x = 200
> If we use PCIe-SSD, the gap will be larger.
> That's why I said 200 is enough to represent speed gap.

Ah, I see what you're saying.

Yeah, that's unfortunately a limitation in the current ABI. Extending
the range to previously unavailable settings is doable; changing the
meaning of existing values is not. We'd have to add another interface.

> Such system configuration is already non-sense so it is okay to ignore such
> usecases?

I'm not sure we have to be proactive about it, but we can always add a
more fine-grained knob to override swappiness when somebody wants to
use such a setup in practice.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web