Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1456266 > unrolled thread
| Started by | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| First post | 2016-08-04 11:30 +0200 |
| Last post | 2016-08-10 20:40 +0200 |
| Articles | 4 — 3 participants |
Back to article view | Back to linux.kernel
mm: Initialise per_cpu_nodestats for all online pgdats at boot Mel Gorman <mgorman@techsingularity.net> - 2016-08-04 11:30 +0200
Re: mm: Initialise per_cpu_nodestats for all online pgdats at boot Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-08-04 17:50 +0200
Re: mm: Initialise per_cpu_nodestats for all online pgdats at boot Paul Mackerras <paulus@ozlabs.org> - 2016-08-05 05:30 +0200
Re: mm: Initialise per_cpu_nodestats for all online pgdats at boot Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-08-10 20:40 +0200
| From | Mel Gorman <mgorman@techsingularity.net> |
|---|---|
| Date | 2016-08-04 11:30 +0200 |
| Subject | mm: Initialise per_cpu_nodestats for all online pgdats at boot |
| Message-ID | <s2n8u-6ZA-41@gated-at.bofh.it> |
Paul Mackerras and Reza Arbab reported that machines with memoryless nodes
fails when vmstats are refreshed. Paul reported an oops as follows
[ 1.713998] Unable to handle kernel paging request for data at address 0xff7a10000
[ 1.714164] Faulting instruction address: 0xc000000000270cd0
[ 1.714304] Oops: Kernel access of bad area, sig: 11 [#1]
[ 1.714414] SMP NR_CPUS=2048 NUMA PowerNV
[ 1.714530] Modules linked in:
[ 1.714647] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0-kvm+ #118
[ 1.714786] task: c000000ff0680010 task.stack: c000000ff0704000
[ 1.714926] NIP: c000000000270cd0 LR: c000000000270ce8 CTR: 0000000000000000
[ 1.715093] REGS: c000000ff0707900 TRAP: 0300 Not tainted (4.7.0-kvm+)
[ 1.715232] MSR: 9000000102009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE,TM[E]> CR: 846b6824 XER: 20000000
[ 1.715748] CFAR: c000000000008768 DAR: 0000000ff7a10000 DSISR: 42000000 SOFTE: 1
GPR00: c000000000270d08 c000000ff0707b80 c0000000011fb200 0000000000000000
GPR04: 0000000000000800 0000000000000000 0000000000000000 0000000000000000
GPR08: ffffffffffffffff 0000000000000000 0000000ff7a10000 c00000000122aae0
GPR12: c000000000a1e440 c00000000fb80000 c00000000000c188 0000000000000000
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 c000000000cecad0
GPR24: c000000000d035b8 c000000000d6cd18 c000000000d6cd18 c000001fffa86300
GPR28: 0000000000000000 c000001fffa96300 c000000001230034 c00000000122eb18
[ 1.717484] NIP [c000000000270cd0] refresh_zone_stat_thresholds+0x80/0x240
[ 1.717568] LR [c000000000270ce8] refresh_zone_stat_thresholds+0x98/0x240
[ 1.717648] Call Trace:
[ 1.717687] [c000000ff0707b80] [c000000000270d08] refresh_zone_stat_thresholds+0xb8/0x240 (unreliable)
Both supplied potential fixes but one potentially misses checks and another
had redundant initialisations. This version initialises per_cpu_nodestats
on a per-pgdat basis instead of on a per-zone basis.
Reported-by: Paul Mackerras <paulus@ozlabs.org>
Reported-by: Reza Arbab <arbab@linux.vnet.ibm.com>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
This has been compile-tested and boot-tested on a 32-bit KVM only. A
memoryless system was not available to test the patch with. A confirmation
from Paul and Reza that it resolves their problem is welcome.
mm/page_alloc.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 39a372a2a1d6..fb975cec3518 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5257,11 +5257,6 @@ static void __meminit setup_zone_pageset(struct zone *zone)
zone->pageset = alloc_percpu(struct per_cpu_pageset);
for_each_possible_cpu(cpu)
zone_pageset_init(zone, cpu);
-
- if (!zone->zone_pgdat->per_cpu_nodestats) {
- zone->zone_pgdat->per_cpu_nodestats =
- alloc_percpu(struct per_cpu_nodestat);
- }
}
/*
@@ -5270,10 +5265,15 @@ static void __meminit setup_zone_pageset(struct zone *zone)
*/
void __init setup_per_cpu_pageset(void)
{
+ struct pglist_data *pgdat;
struct zone *zone;
for_each_populated_zone(zone)
setup_zone_pageset(zone);
+
+ for_each_online_pgdat(pgdat)
+ pgdat->per_cpu_nodestats =
+ alloc_percpu(struct per_cpu_nodestat);
}
static noinline __ref
[toc] | [next] | [standalone]
| From | Reza Arbab <arbab@linux.vnet.ibm.com> |
|---|---|
| Date | 2016-08-04 17:50 +0200 |
| Message-ID | <s2t4d-2PF-5@gated-at.bofh.it> |
| In reply to | #1456266 |
On Thu, Aug 04, 2016 at 10:24:04AM +0100, Mel Gorman wrote: >This has been compile-tested and boot-tested on a 32-bit KVM only. A >memoryless system was not available to test the patch with. A confirmation >from Paul and Reza that it resolves their problem is welcome. Works for me. Thanks, Mel! Tested-by: Reza Arbab <arbab@linux.vnet.ibm.com> -- Reza Arbab
[toc] | [prev] | [next] | [standalone]
| From | Paul Mackerras <paulus@ozlabs.org> |
|---|---|
| Date | 2016-08-05 05:30 +0200 |
| Message-ID | <s2DZD-26z-3@gated-at.bofh.it> |
| In reply to | #1456266 |
On Thu, Aug 04, 2016 at 10:24:04AM +0100, Mel Gorman wrote: > Paul Mackerras and Reza Arbab reported that machines with memoryless nodes > fails when vmstats are refreshed. Paul reported an oops as follows > > [ 1.713998] Unable to handle kernel paging request for data at address 0xff7a10000 > [ 1.714164] Faulting instruction address: 0xc000000000270cd0 > [ 1.714304] Oops: Kernel access of bad area, sig: 11 [#1] > [ 1.714414] SMP NR_CPUS=2048 NUMA PowerNV > [ 1.714530] Modules linked in: > [ 1.714647] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0-kvm+ #118 > [ 1.714786] task: c000000ff0680010 task.stack: c000000ff0704000 > [ 1.714926] NIP: c000000000270cd0 LR: c000000000270ce8 CTR: 0000000000000000 > [ 1.715093] REGS: c000000ff0707900 TRAP: 0300 Not tainted (4.7.0-kvm+) > [ 1.715232] MSR: 9000000102009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE,TM[E]> CR: 846b6824 XER: 20000000 > [ 1.715748] CFAR: c000000000008768 DAR: 0000000ff7a10000 DSISR: 42000000 SOFTE: 1 > GPR00: c000000000270d08 c000000ff0707b80 c0000000011fb200 0000000000000000 > GPR04: 0000000000000800 0000000000000000 0000000000000000 0000000000000000 > GPR08: ffffffffffffffff 0000000000000000 0000000ff7a10000 c00000000122aae0 > GPR12: c000000000a1e440 c00000000fb80000 c00000000000c188 0000000000000000 > GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > GPR20: 0000000000000000 0000000000000000 0000000000000000 c000000000cecad0 > GPR24: c000000000d035b8 c000000000d6cd18 c000000000d6cd18 c000001fffa86300 > GPR28: 0000000000000000 c000001fffa96300 c000000001230034 c00000000122eb18 > [ 1.717484] NIP [c000000000270cd0] refresh_zone_stat_thresholds+0x80/0x240 > [ 1.717568] LR [c000000000270ce8] refresh_zone_stat_thresholds+0x98/0x240 > [ 1.717648] Call Trace: > [ 1.717687] [c000000ff0707b80] [c000000000270d08] refresh_zone_stat_thresholds+0xb8/0x240 (unreliable) > > Both supplied potential fixes but one potentially misses checks and another > had redundant initialisations. This version initialises per_cpu_nodestats > on a per-pgdat basis instead of on a per-zone basis. > > Reported-by: Paul Mackerras <paulus@ozlabs.org> > Reported-by: Reza Arbab <arbab@linux.vnet.ibm.com> > Signed-off-by: Mel Gorman <mgorman@techsingularity.net> That works, thanks. Tested-by: Paul Mackerras <paulus@ozlabs.org>
[toc] | [prev] | [next] | [standalone]
| From | Reza Arbab <arbab@linux.vnet.ibm.com> |
|---|---|
| Date | 2016-08-10 20:40 +0200 |
| Message-ID | <s4GA3-f8-53@gated-at.bofh.it> |
| In reply to | #1456266 |
On Thu, Aug 04, 2016 at 10:24:04AM +0100, Mel Gorman wrote:
>[ 1.713998] Unable to handle kernel paging request for data at address 0xff7a10000
>[ 1.714164] Faulting instruction address: 0xc000000000270cd0
>[ 1.714304] Oops: Kernel access of bad area, sig: 11 [#1]
>[ 1.714414] SMP NR_CPUS=2048 NUMA PowerNV
>[ 1.714530] Modules linked in:
>[ 1.714647] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0-kvm+ #118
>[ 1.714786] task: c000000ff0680010 task.stack: c000000ff0704000
>[ 1.714926] NIP: c000000000270cd0 LR: c000000000270ce8 CTR: 0000000000000000
>[ 1.715093] REGS: c000000ff0707900 TRAP: 0300 Not tainted (4.7.0-kvm+)
>[ 1.715232] MSR: 9000000102009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE,TM[E]> CR: 846b6824 XER: 20000000
>[ 1.715748] CFAR: c000000000008768 DAR: 0000000ff7a10000 DSISR: 42000000 SOFTE: 1
>GPR00: c000000000270d08 c000000ff0707b80 c0000000011fb200 0000000000000000
>GPR04: 0000000000000800 0000000000000000 0000000000000000 0000000000000000
>GPR08: ffffffffffffffff 0000000000000000 0000000ff7a10000 c00000000122aae0
>GPR12: c000000000a1e440 c00000000fb80000 c00000000000c188 0000000000000000
>GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>GPR20: 0000000000000000 0000000000000000 0000000000000000 c000000000cecad0
>GPR24: c000000000d035b8 c000000000d6cd18 c000000000d6cd18 c000001fffa86300
>GPR28: 0000000000000000 c000001fffa96300 c000000001230034 c00000000122eb18
>[ 1.717484] NIP [c000000000270cd0] refresh_zone_stat_thresholds+0x80/0x240
>[ 1.717568] LR [c000000000270ce8] refresh_zone_stat_thresholds+0x98/0x240
>[ 1.717648] Call Trace:
>[ 1.717687] [c000000ff0707b80] [c000000000270d08] refresh_zone_stat_thresholds+0xb8/0x240 (unreliable)
I've been investigating node hotplug. That path is also going to require
initialization of per_cpu_nodestats. This worked for me:
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 3894b65..41266dc 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1219,6 +1219,7 @@ static pg_data_t __ref *hotadd_new_pgdat(int nid, u64 start)
/* init node's zones as empty zones, we don't have any present pages.*/
free_area_init_node(nid, zones_size, start_pfn, zholes_size);
+ pgdat->per_cpu_nodestats = alloc_percpu(struct per_cpu_nodestat);
/*
* The node we allocated has no zone fallback lists. For avoiding
@@ -1249,6 +1250,7 @@ static pg_data_t __ref *hotadd_new_pgdat(int nid, u64 start)
static void rollback_node_hotadd(int nid, pg_data_t *pgdat)
{
arch_refresh_nodedata(nid, NULL);
+ free_percpu(pgdat->per_cpu_nodestats);
arch_free_nodedata(pgdat);
return;
}
--
Reza Arbab
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web