Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1718853 > unrolled thread
| Started by | js1304@gmail.com |
|---|---|
| First post | 2017-08-24 07:50 +0200 |
| Last post | 2017-08-25 10:00 +0200 |
| Articles | 8 — 4 participants |
Back to article view | Back to linux.kernel
[PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request js1304@gmail.com - 2017-08-24 07:50 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Michal Hocko <mhocko@kernel.org> - 2017-08-24 11:40 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Joonsoo Kim <iamjoonsoo.kim@lge.com> - 2017-08-25 02:20 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Michal Hocko <mhocko@kernel.org> - 2017-08-25 09:40 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Vlastimil Babka <vbabka@suse.cz> - 2017-08-24 11:50 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Joonsoo Kim <iamjoonsoo.kim@lge.com> - 2017-08-25 02:30 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Michal Hocko <mhocko@kernel.org> - 2017-08-25 09:40 +0200
Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request Vlastimil Babka <vbabka@suse.cz> - 2017-08-25 10:00 +0200
| From | js1304@gmail.com |
|---|---|
| Date | 2017-08-24 07:50 +0200 |
| Subject | [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uhTbI-2lw-17@gated-at.bofh.it> |
From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that
important to reserve. When ZONE_MOVABLE is used, this problem would
theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE
allocation request which is mainly used for page cache and anon page
allocation. So, fix it.
And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size
makes code complex. For example, if there is highmem system, following
reserve ratio is activated for *NORMAL ZONE* which would be easyily
misleading people.
#ifdef CONFIG_HIGHMEM
32
#endif
This patch also fix this situation by defining sysctl_lowmem_reserve_ratio
array by MAX_NR_ZONES and place "#ifdef" to right place.
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
---
include/linux/mmzone.h | 2 +-
mm/page_alloc.c | 11 ++++++-----
2 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index e7e92c8..e5f134b 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -882,7 +882,7 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
int watermark_scale_factor_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
-extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1];
+extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES];
int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int,
void __user *, size_t *, loff_t *);
int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *, int,
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 90b1996..6faa53d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -202,17 +202,18 @@ static void __free_pages_ok(struct page *page, unsigned int order);
* TBD: should special case ZONE_DMA32 machines here - in those we normally
* don't need any ZONE_NORMAL reservation
*/
-int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = {
+int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES] = {
#ifdef CONFIG_ZONE_DMA
- 256,
+ [ZONE_DMA] = 256,
#endif
#ifdef CONFIG_ZONE_DMA32
- 256,
+ [ZONE_DMA32] = 256,
#endif
+ [ZONE_NORMAL] = 32,
#ifdef CONFIG_HIGHMEM
- 32,
+ [ZONE_HIGHMEM] = INT_MAX,
#endif
- 32,
+ [ZONE_MOVABLE] = INT_MAX,
};
EXPORT_SYMBOL(totalram_pages);
--
2.7.4
[toc] | [next] | [standalone]
| From | Michal Hocko <mhocko@kernel.org> |
|---|---|
| Date | 2017-08-24 11:40 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uhWMi-4E5-11@gated-at.bofh.it> |
| In reply to | #1718853 |
On Thu 24-08-17 14:45:46, Joonsoo Kim wrote:
> From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that
> important to reserve. When ZONE_MOVABLE is used, this problem would
> theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE
> allocation request which is mainly used for page cache and anon page
> allocation. So, fix it.
I do not really understand what is the problem you are trying to fix.
Yes the memory is reserved for a higher priority consumer and that is
deliberate AFAICT. Just consider that an OOM victim wants to make
further progress and rely on memory reserve while doing
GFP_HIGHUSER_MOVABLE request.
So what is the real problem you are trying to address here?
> And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size
> makes code complex. For example, if there is highmem system, following
> reserve ratio is activated for *NORMAL ZONE* which would be easyily
> misleading people.
>
> #ifdef CONFIG_HIGHMEM
> 32
> #endif
>
> This patch also fix this situation by defining sysctl_lowmem_reserve_ratio
> array by MAX_NR_ZONES and place "#ifdef" to right place.
>
> Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> ---
> include/linux/mmzone.h | 2 +-
> mm/page_alloc.c | 11 ++++++-----
> 2 files changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index e7e92c8..e5f134b 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -882,7 +882,7 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> int watermark_scale_factor_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> -extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1];
> +extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES];
> int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *, int,
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 90b1996..6faa53d 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -202,17 +202,18 @@ static void __free_pages_ok(struct page *page, unsigned int order);
> * TBD: should special case ZONE_DMA32 machines here - in those we normally
> * don't need any ZONE_NORMAL reservation
> */
> -int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = {
> +int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES] = {
> #ifdef CONFIG_ZONE_DMA
> - 256,
> + [ZONE_DMA] = 256,
> #endif
> #ifdef CONFIG_ZONE_DMA32
> - 256,
> + [ZONE_DMA32] = 256,
> #endif
> + [ZONE_NORMAL] = 32,
> #ifdef CONFIG_HIGHMEM
> - 32,
> + [ZONE_HIGHMEM] = INT_MAX,
> #endif
> - 32,
> + [ZONE_MOVABLE] = INT_MAX,
> };
>
> EXPORT_SYMBOL(totalram_pages);
> --
> 2.7.4
>
--
Michal Hocko
SUSE Labs
[toc] | [prev] | [next] | [standalone]
| From | Joonsoo Kim <iamjoonsoo.kim@lge.com> |
|---|---|
| Date | 2017-08-25 02:20 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uiavU-5eJ-17@gated-at.bofh.it> |
| In reply to | #1719065 |
On Thu, Aug 24, 2017 at 11:30:50AM +0200, Michal Hocko wrote: > On Thu 24-08-17 14:45:46, Joonsoo Kim wrote: > > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > > Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that > > important to reserve. When ZONE_MOVABLE is used, this problem would > > theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE > > allocation request which is mainly used for page cache and anon page > > allocation. So, fix it. > > I do not really understand what is the problem you are trying to fix. > Yes the memory is reserved for a higher priority consumer and that is > deliberate AFAICT. Just consider that an OOM victim wants to make > further progress and rely on memory reserve while doing > GFP_HIGHUSER_MOVABLE request. > > So what is the real problem you are trying to address here? If the system has the both, ZONE_HIGHMEM and ZONE_MOVABLE, ZONE_HIGHMEM will reserve the memory for ZONE_MOVABLE request. However, they are consumed by nearly equivalent priority consumer who uses GFP_HIGHMEM + GFP_MOVABLE. In that case, reserved memory in ZONE_HIGHMEM would not be used and it means just waste of the memory. This patch try to fix it to nullify reserving memory in ZONE_HIGHMEM. And, I think that all this problem is caused by the complex code in lowmem reserve calculation. So, did some clean-up. Thanks.
[toc] | [prev] | [next] | [standalone]
| From | Michal Hocko <mhocko@kernel.org> |
|---|---|
| Date | 2017-08-25 09:40 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uihnH-17Y-3@gated-at.bofh.it> |
| In reply to | #1719664 |
On Fri 25-08-17 09:15:43, Joonsoo Kim wrote: > On Thu, Aug 24, 2017 at 11:30:50AM +0200, Michal Hocko wrote: > > On Thu 24-08-17 14:45:46, Joonsoo Kim wrote: > > > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > > > > Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that > > > important to reserve. When ZONE_MOVABLE is used, this problem would > > > theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE > > > allocation request which is mainly used for page cache and anon page > > > allocation. So, fix it. > > > > I do not really understand what is the problem you are trying to fix. > > Yes the memory is reserved for a higher priority consumer and that is > > deliberate AFAICT. Just consider that an OOM victim wants to make > > further progress and rely on memory reserve while doing > > GFP_HIGHUSER_MOVABLE request. > > > > So what is the real problem you are trying to address here? > > If the system has the both, ZONE_HIGHMEM and ZONE_MOVABLE, > ZONE_HIGHMEM will reserve the memory for ZONE_MOVABLE request. Ohh, right. I forgot that __GFP_MOVABLE doesn't really enforce the movable zone. It does so only if __GFP_HIGHMEM is specified as well when ZONE_HIGHMEM is enabled. So indeed reserving memory in both is somehow awkward. So why don't we simply remove reserves from the movable zone when the highmem zone is enabled? -- Michal Hocko SUSE Labs
[toc] | [prev] | [next] | [standalone]
| From | Vlastimil Babka <vbabka@suse.cz> |
|---|---|
| Date | 2017-08-24 11:50 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uhWVY-4Hm-29@gated-at.bofh.it> |
| In reply to | #1718853 |
On 08/24/2017 07:45 AM, js1304@gmail.com wrote:
> From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that
> important to reserve. When ZONE_MOVABLE is used, this problem would
> theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE
> allocation request which is mainly used for page cache and anon page
> allocation. So, fix it.
>
> And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size
> makes code complex. For example, if there is highmem system, following
> reserve ratio is activated for *NORMAL ZONE* which would be easyily
> misleading people.
>
> #ifdef CONFIG_HIGHMEM
> 32
> #endif
>
> This patch also fix this situation by defining sysctl_lowmem_reserve_ratio
> array by MAX_NR_ZONES and place "#ifdef" to right place.
>
> Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
Looks like I did that almost year ago, so definitely had to refresh my
memory now :)
Anyway now I looked more thoroughly and noticed that this change leaks
into the reported sysctl. On a 64bit system with ZONE_MOVABLE:
before the patch:
vm.lowmem_reserve_ratio = 256 256 32
after the patch:
vm.lowmem_reserve_ratio = 256 256 32 2147483647
So if we indeed remove HIGHMEM from protection (c.f. Michal's mail), we
should do that differently than with the INT_MAX trick, IMHO.
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> ---
> include/linux/mmzone.h | 2 +-
> mm/page_alloc.c | 11 ++++++-----
> 2 files changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index e7e92c8..e5f134b 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -882,7 +882,7 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> int watermark_scale_factor_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> -extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1];
> +extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES];
> int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int,
> void __user *, size_t *, loff_t *);
> int percpu_pagelist_fraction_sysctl_handler(struct ctl_table *, int,
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 90b1996..6faa53d 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -202,17 +202,18 @@ static void __free_pages_ok(struct page *page, unsigned int order);
> * TBD: should special case ZONE_DMA32 machines here - in those we normally
> * don't need any ZONE_NORMAL reservation
> */
> -int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES-1] = {
> +int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES] = {
> #ifdef CONFIG_ZONE_DMA
> - 256,
> + [ZONE_DMA] = 256,
> #endif
> #ifdef CONFIG_ZONE_DMA32
> - 256,
> + [ZONE_DMA32] = 256,
> #endif
> + [ZONE_NORMAL] = 32,
> #ifdef CONFIG_HIGHMEM
> - 32,
> + [ZONE_HIGHMEM] = INT_MAX,
> #endif
> - 32,
> + [ZONE_MOVABLE] = INT_MAX,
> };
>
> EXPORT_SYMBOL(totalram_pages);
>
[toc] | [prev] | [next] | [standalone]
| From | Joonsoo Kim <iamjoonsoo.kim@lge.com> |
|---|---|
| Date | 2017-08-25 02:30 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uiaFA-5hT-9@gated-at.bofh.it> |
| In reply to | #1719083 |
On Thu, Aug 24, 2017 at 11:41:58AM +0200, Vlastimil Babka wrote: > On 08/24/2017 07:45 AM, js1304@gmail.com wrote: > > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > > Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that > > important to reserve. When ZONE_MOVABLE is used, this problem would > > theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE > > allocation request which is mainly used for page cache and anon page > > allocation. So, fix it. > > > > And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size > > makes code complex. For example, if there is highmem system, following > > reserve ratio is activated for *NORMAL ZONE* which would be easyily > > misleading people. > > > > #ifdef CONFIG_HIGHMEM > > 32 > > #endif > > > > This patch also fix this situation by defining sysctl_lowmem_reserve_ratio > > array by MAX_NR_ZONES and place "#ifdef" to right place. > > > > Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> > > Acked-by: Vlastimil Babka <vbabka@suse.cz> > > Looks like I did that almost year ago, so definitely had to refresh my > memory now :) > > Anyway now I looked more thoroughly and noticed that this change leaks > into the reported sysctl. On a 64bit system with ZONE_MOVABLE: > > before the patch: > vm.lowmem_reserve_ratio = 256 256 32 > > after the patch: > vm.lowmem_reserve_ratio = 256 256 32 2147483647 > > So if we indeed remove HIGHMEM from protection (c.f. Michal's mail), we > should do that differently than with the INT_MAX trick, IMHO. Hmm, this is already pointed by Minchan and I have answered that. lkml.kernel.org/r/<20170421013243.GA13966@js1304-desktop> If you have a better idea, please let me know. Thanks.
[toc] | [prev] | [next] | [standalone]
| From | Michal Hocko <mhocko@kernel.org> |
|---|---|
| Date | 2017-08-25 09:40 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uihnI-17Y-7@gated-at.bofh.it> |
| In reply to | #1719668 |
On Fri 25-08-17 09:20:31, Joonsoo Kim wrote: > On Thu, Aug 24, 2017 at 11:41:58AM +0200, Vlastimil Babka wrote: > > On 08/24/2017 07:45 AM, js1304@gmail.com wrote: > > > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > > > > Freepage on ZONE_HIGHMEM doesn't work for kernel memory so it's not that > > > important to reserve. When ZONE_MOVABLE is used, this problem would > > > theorectically cause to decrease usable memory for GFP_HIGHUSER_MOVABLE > > > allocation request which is mainly used for page cache and anon page > > > allocation. So, fix it. > > > > > > And, defining sysctl_lowmem_reserve_ratio array by MAX_NR_ZONES - 1 size > > > makes code complex. For example, if there is highmem system, following > > > reserve ratio is activated for *NORMAL ZONE* which would be easyily > > > misleading people. > > > > > > #ifdef CONFIG_HIGHMEM > > > 32 > > > #endif > > > > > > This patch also fix this situation by defining sysctl_lowmem_reserve_ratio > > > array by MAX_NR_ZONES and place "#ifdef" to right place. > > > > > > Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> > > > Acked-by: Vlastimil Babka <vbabka@suse.cz> > > > > Looks like I did that almost year ago, so definitely had to refresh my > > memory now :) > > > > Anyway now I looked more thoroughly and noticed that this change leaks > > into the reported sysctl. On a 64bit system with ZONE_MOVABLE: > > > > before the patch: > > vm.lowmem_reserve_ratio = 256 256 32 > > > > after the patch: > > vm.lowmem_reserve_ratio = 256 256 32 2147483647 > > > > So if we indeed remove HIGHMEM from protection (c.f. Michal's mail), we > > should do that differently than with the INT_MAX trick, IMHO. > > Hmm, this is already pointed by Minchan and I have answered that. > > lkml.kernel.org/r/<20170421013243.GA13966@js1304-desktop> > > If you have a better idea, please let me know. Why don't we just use 0. In fact we are reserving 0 pages... Using INT_MAX is just wrong. -- Michal Hocko SUSE Labs
[toc] | [prev] | [next] | [standalone]
| From | Vlastimil Babka <vbabka@suse.cz> |
|---|---|
| Date | 2017-08-25 10:00 +0200 |
| Subject | Re: [PATCH] mm/page_alloc: don't reserve ZONE_HIGHMEM for ZONE_MOVABLE request |
| Message-ID | <uihH4-1eB-9@gated-at.bofh.it> |
| In reply to | #1719668 |
On 08/25/2017 02:20 AM, Joonsoo Kim wrote: > On Thu, Aug 24, 2017 at 11:41:58AM +0200, Vlastimil Babka wrote: > > Hmm, this is already pointed by Minchan and I have answered that. > > lkml.kernel.org/r/<20170421013243.GA13966@js1304-desktop> > > If you have a better idea, please let me know. My idea is that size of sysctl_lowmem_reserve_ratio is ZONE_NORMAL+1 and it has no entries for zones > NORMAL. The setup_per_zone_lowmem_reserve() is adjusted to only set lower_zone->lowmem_reserve[j] for idx <= ZONE_NORMAL. I can't imagine somebody would want override the ratio for HIGHMEM or MOVABLE (where it has no effect anyway) so the simplest thing is not to expose it at all. > Thanks. >
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web