Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1359088 > unrolled thread

Re: 4.5.0+ panic when setup loop device

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: 4.5.0+ panic when setup loop device Thomas Gleixner <tglx@linutronix.de> - 2016-03-16 16:30 +0100
    Re: 4.5.0+ panic when setup loop device Peter Zijlstra <peterz@infradead.org> - 2016-03-17 11:00 +0100
    Re: 4.5.0+ panic when setup loop device Peter Zijlstra <peterz@infradead.org> - 2016-03-17 11:00 +0100
      Re: 4.5.0+ panic when setup loop device Thomas Gleixner <tglx@linutronix.de> - 2016-03-17 11:30 +0100
        Re: 4.5.0+ panic when setup loop device Peter Zijlstra <peterz@infradead.org> - 2016-03-17 11:30 +0100
          Re: 4.5.0+ panic when setup loop device Thomas Gleixner <tglx@linutronix.de> - 2016-03-17 12:50 +0100
            Re: 4.5.0+ panic when setup loop device Peter Zijlstra <peterz@infradead.org> - 2016-03-17 13:00 +0100
              Re: 4.5.0+ panic when setup loop device Borislav Petkov <bp@alien8.de> - 2016-03-17 13:00 +0100
              Re: 4.5.0+ panic when setup loop device Thomas Gleixner <tglx@linutronix.de> - 2016-03-17 13:10 +0100

#1359088 — Re: 4.5.0+ panic when setup loop device

On Wed, 16 Mar 2016, Xiong Zhou wrote:
> full log , bisect log and config are attached.

Can you please provide a full boot log and the output of 'cat /proc/cpuinfo' ?

Thanks,

	tglx

[toc] | [next] | [standalone]

#1359686

On Thu, Mar 17, 2016 at 10:52:20AM +0100, Peter Zijlstra wrote:
> Mar 17 17:34:30 myhost kernel: smpboot: CPU0: AMD Opteron(TM) Processor 6274 (family: 0x15, model: 0x1, stepping: 0x2)
> Mar 17 17:34:30 myhost kernel: Performance Events: Fam15h core perfctr, Broken BIOS detected, complain to your hardware vendor.
> Mar 17 17:34:30 myhost kernel: [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR c0010200 is 430076)

FWIW, you might want to talk to HP about that; some machines have a
magic key-combo in their BIOS screen with extra options, allowing you to
fix this.

[toc] | [prev] | [next] | [standalone]

#1359689

On Thu, Mar 17, 2016 at 09:56:05AM +0800, Xiong Zhou wrote:
> On Wed, Mar 16, 2016 at 11:26 PM, Thomas Gleixner <tglx@linutronix.de> wrote:

> > Can you please provide a full boot log and the output of 'cat /proc/cpuinfo' ?

Mar 17 17:34:30 myhost kernel: smpboot: Max logical packages: 1
Mar 17 17:34:30 myhost kernel: smpboot: APIC(20) Converting physical 1 to logical package 0
Mar 17 17:34:30 myhost kernel: smpboot: APIC(40) Package 2 exceeds logical package map

So that is busted.. it turns out AMD gets x86_max_cores wrong when there
are compute units.

Mar 17 17:34:30 myhost kernel: smpboot: CPU 1 APICId 40 disabled
Mar 17 17:34:30 myhost kernel: Switched APIC routing to physical flat.
Mar 17 17:34:30 myhost kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Mar 17 17:34:30 myhost kernel: smpboot: CPU0: AMD Opteron(TM) Processor 6274 (family: 0x15, model: 0x1, stepping: 0x2)
Mar 17 17:34:30 myhost kernel: Performance Events: Fam15h core perfctr, Broken BIOS detected, complain to your hardware vendor.
Mar 17 17:34:30 myhost kernel: [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR c0010200 is 430076)
Mar 17 17:34:30 myhost kernel: AMD PMU driver.
Mar 17 17:34:30 myhost kernel: ... version:                0
Mar 17 17:34:30 myhost kernel: ... bit width:              48
Mar 17 17:34:30 myhost kernel: ... generic registers:      6
Mar 17 17:34:30 myhost kernel: ... value mask:             0000ffffffffffff
Mar 17 17:34:30 myhost kernel: ... max period:             00007fffffffffff
Mar 17 17:34:30 myhost kernel: ... fixed-purpose events:   0
Mar 17 17:34:30 myhost kernel: ... event mask:             000000000000003f
Mar 17 17:34:30 myhost kernel: NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:    #2  #3  #4  #5  #6  #7  #8  #9 #10 #11 #12 #13 #14 #15 #16
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #17
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #18
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #19
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #20
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #21
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #22
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #23
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #24
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #25
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #26
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #27
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #28
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #29
Mar 17 17:34:30 myhost kernel: .... node  #0, CPUs:   #30
Mar 17 17:34:30 myhost kernel: .... node  #3, CPUs:   #31
Mar 17 17:34:30 myhost kernel: x86: Booted up 2 nodes, 31 CPUs

And that is one weird node mapping..


I have a similar system, which after the below patch says:

[    0.182174] max_cores: 8, cpu_ids: 32, num_siblings: 2, coreid_bits: 5
[    0.188712] smpboot: Max logical packages: 2
[    0.192988] smpboot: APIC(20) Converting physical 1 to logical package 0
[    0.199689] smpboot: APIC(40) Converting physical 2 to logical package 1
[    0.206405] Switched APIC routing to physical flat.
[    0.211851] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.329578] smpboot: CPU0: AMD Opteron(tm) Processor 6278 (family: 0x15, model: 0x1, stepping: 0x2)
[    0.338705] Performance Events: Fam15h core perfctr, AMD PMU driver.
[    0.345134] ... version:                0
[    0.349147] ... bit width:              48
[    0.353262] ... generic registers:      6
[    0.357274] ... value mask:             0000ffffffffffff
[    0.362586] ... max period:             00007fffffffffff
[    0.367900] ... fixed-purpose events:   0
[    0.371911] ... event mask:             000000000000003f
[    0.378664] MCE: In-kernel MCE decoding enabled.
[    0.383965] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
[    0.393079] x86: Booting SMP configuration:
[    0.397262] .... node  #0, CPUs:        #1  #2  #3  #4  #5  #6  #7
[    0.848764] .... node  #1, CPUs:    #8  #9 #10 #11 #12 #13 #14 #15
[    1.364701] .... node  #2, CPUs:   #16 #17 #18 #19 #20 #21 #22 #23
[    1.898586] .... node  #3, CPUs:   #24 #25 #26 #27 #28 #29 #30 #31
[    2.413417] x86: Booted up 4 nodes, 32 CPUs

Could you please try? I'm not sure how this would explain your loop
device bug fail, but it certainly pointed towards broken.


Andreas; Borislav said to Cc you since you wrote all this.
The issue is that Linux assumes:

	nr_logical_cpus = nr_cores * nr_siblings

But AMD reports its CU unit as 2 cores, but then sets num_smp_siblings
to 2 as well.

Thomas; I removed that first branch testing pkg against
__max_logical_packages because if the first pkg id is larger, then the
find_first_zero will find us logical package id 0. However, if the
second pkg id is indeed 0, we'll again claim it without testing if it
was already taken. Also, it fails to print the mapping.


---
 arch/x86/kernel/cpu/amd.c |    8 ++++----
 arch/x86/kernel/smpboot.c |   11 ++++++-----
 2 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 97c59fd..6216e80 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -310,9 +310,9 @@ static void amd_get_topology(struct cpuinfo_x86 *c)
 		node_id = ecx & 7;
 
 		/* get compute unit information */
-		smp_num_siblings = ((ebx >> 8) & 3) + 1;
+		cores_per_cu = smp_num_siblings = ((ebx >> 8) & 3) + 1;
+		c->x86_max_cores /= smp_num_siblings;
 		c->compute_unit_id = ebx & 0xff;
-		cores_per_cu += ((ebx >> 8) & 3);
 	} else if (cpu_has(c, X86_FEATURE_NODEID_MSR)) {
 		u64 value;
 
@@ -328,8 +328,8 @@ static void amd_get_topology(struct cpuinfo_x86 *c)
 		u32 cus_per_node;
 
 		set_cpu_cap(c, X86_FEATURE_AMD_DCM);
-		cores_per_node = c->x86_max_cores / nodes_per_socket;
-		cus_per_node = cores_per_node / cores_per_cu;
+		cus_per_node = c->x86_max_cores / nodes_per_socket;
+		cores_per_node = cus_per_node * cores_per_cu;
 
 		/* store NodeID, use llc_shared_map to store sibling info */
 		per_cpu(cpu_llc_id, cpu) = node_id;
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 643dbdc..15c5fda 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -274,11 +274,6 @@ int topology_update_package_map(unsigned int apicid, unsigned int cpu)
 	if (test_and_set_bit(pkg, physical_package_map))
 		goto found;
 
-	if (pkg < __max_logical_packages) {
-		set_bit(pkg, logical_package_map);
-		physical_to_logical_pkg[pkg] = pkg;
-		goto found;
-	}
 	new = find_first_zero_bit(logical_package_map, __max_logical_packages);
 	if (new >= __max_logical_packages) {
 		physical_to_logical_pkg[pkg] = -1;
@@ -314,6 +309,12 @@ static void __init smp_init_package_map(void)
 	unsigned int ncpus, cpu;
 	size_t size;
 
+	printk("max_cores: %d, cpu_ids: %d, num_siblings: %d, coreid_bits: %d\n",
+			boot_cpu_data.x86_max_cores,
+			nr_cpu_ids,
+			smp_num_siblings,
+			boot_cpu_data.x86_coreid_bits);
+
 	/*
 	 * Today neither Intel nor AMD support heterogenous systems. That
 	 * might change in the future....

[toc] | [prev] | [next] | [standalone]

#1359711

On Thu, 17 Mar 2016, Peter Zijlstra wrote:

> Could you please try? I'm not sure how this would explain your loop
> device bug fail, but it certainly pointed towards broken.

It definitely does not explain it. The wreckage that topo stuff causes is that
it disables a cpu, but that really is not a reason for block/loop to explode.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1359714

On Thu, Mar 17, 2016 at 11:21:24AM +0100, Thomas Gleixner wrote:
> On Thu, 17 Mar 2016, Peter Zijlstra wrote:
> 
> > Could you please try? I'm not sure how this would explain your loop
> > device bug fail, but it certainly pointed towards broken.
> 
> It definitely does not explain it. The wreckage that topo stuff causes is that
> it disables a cpu, but that really is not a reason for block/loop to explode.

Right. Sadly I could not reproduce that error on my machine. But we can
at least start by fixing the 'obvious' problems and then maybe we get
more clues ;-)

[toc] | [prev] | [next] | [standalone]

#1359767

B1;2802;0cOn Thu, 17 Mar 2016, Peter Zijlstra wrote:

> On Thu, Mar 17, 2016 at 11:21:24AM +0100, Thomas Gleixner wrote:
> > On Thu, 17 Mar 2016, Peter Zijlstra wrote:
> > 
> > > Could you please try? I'm not sure how this would explain your loop
> > > device bug fail, but it certainly pointed towards broken.
> > 
> > It definitely does not explain it. The wreckage that topo stuff causes is that
> > it disables a cpu, but that really is not a reason for block/loop to explode.
> 
> Right. Sadly I could not reproduce that error on my machine. But we can
> at least start by fixing the 'obvious' problems and then maybe we get
> more clues ;-)

I'm able to reproduce by rejecting a cpu in that topology map function
forcefully.

That stuff explodes, because the block-mq code assumes that cpu_possible_mask
has no holes.

#define queue_for_each_ctx(q, ctx, i)                                   \
        for ((i) = 0; (i) < (q)->nr_queues &&                           \
             ({ ctx = per_cpu_ptr((q)->queue_ctx, (i)); 1; }); (i)++)

is what makes that assumption about a consecutive possible mask.

The cure for now is the patch below on top of PeterZ's patch.

But we have to clarify and document whether holes in cpu_possible_mask are not
allowed at all or if code like the above is simply broken.

Thanks,

	tglx
---
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 643dbdccf4bc..f2ed8a01f870 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -345,7 +345,6 @@ static void __init smp_init_package_map(void)
 			continue;
 		pr_warn("CPU %u APICId %x disabled\n", cpu, apicid);
 		per_cpu(x86_bios_cpu_apicid, cpu) = BAD_APICID;
-		set_cpu_possible(cpu, false);
 		set_cpu_present(cpu, false);
 	}
 }







 

[toc] | [prev] | [next] | [standalone]

#1359775

On Thu, Mar 17, 2016 at 12:39:46PM +0100, Thomas Gleixner wrote:
> But we have to clarify and document whether holes in cpu_possible_mask are not
> allowed at all or if code like the above is simply broken.

So the general rule is that cpumasks can have holes, and exempting one
just muddles the water.

Therefore I'd call the code just plain broken.

[toc] | [prev] | [next] | [standalone]

#1359779

On Thu, Mar 17, 2016 at 12:51:20PM +0100, Peter Zijlstra wrote:
> On Thu, Mar 17, 2016 at 12:39:46PM +0100, Thomas Gleixner wrote:
> > But we have to clarify and document whether holes in cpu_possible_mask are not
> > allowed at all or if code like the above is simply broken.
> 
> So the general rule is that cpumasks can have holes, and exempting one
> just muddles the water.
> 
> Therefore I'd call the code just plain broken.

I'll say.

Can't the code simply do:

	if (!cpu_possible(i))
		continue;

?

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

[toc] | [prev] | [next] | [standalone]

#1359787

On Thu, 17 Mar 2016, Peter Zijlstra wrote:
> On Thu, Mar 17, 2016 at 12:39:46PM +0100, Thomas Gleixner wrote:
> > But we have to clarify and document whether holes in cpu_possible_mask are not
> > allowed at all or if code like the above is simply broken.
> 
> So the general rule is that cpumasks can have holes, and exempting one
> just muddles the water.
> 
> Therefore I'd call the code just plain broken.

Agreed. 

That macro is not really helping the readability of the code at all. So a
simple for_each_possible_cpu() loop would have avoided that wreckage.

Thanks,

	tglx

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web