Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1167140 > unrolled thread

Re: [PATCH V1] x86, espfix: postpone the initialization of espfix stack for AP

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH V1] x86, espfix: postpone the initialization of espfix  stack for AP Borislav Petkov <bp@alien8.de> - 2015-06-17 23:10 +0200
    Re: [PATCH V1] x86, espfix: postpone the initialization of espfix  stack for AP "H. Peter Anvin" <hpa@zytor.com> - 2015-06-17 23:20 +0200
      Re: [PATCH V1] x86, espfix: postpone the initialization of espfix  stack for AP Borislav Petkov <bp@alien8.de> - 2015-06-18 00:00 +0200

#1167140 — Re: [PATCH V1] x86, espfix: postpone the initialization of espfix stack for AP

On Wed, Jun 17, 2015 at 12:27:05AM -0700, H. Peter Anvin wrote:
> On 06/04/2015 02:45 AM, Gu Zheng wrote:
> > The following lockdep warning occurrs when running with latest kernel:
> > [    3.178000] ------------[ cut here ]------------
> > [    3.183000] WARNING: CPU: 128 PID: 0 at kernel/locking/lockdep.c:2755 lockdep_trace_alloc+0xdd/0xe0()
> > [    3.193000] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
> > [    3.199000] Modules linked in:
> > 
> > [    3.203000] CPU: 128 PID: 0 Comm: swapper/128 Not tainted 4.1.0-rc3 #70
> > [    3.221000]  0000000000000000 2d6601fb3e6d4e4c ffff88086fd5fc38 ffffffff81773f0a
> > [    3.230000]  0000000000000000 ffff88086fd5fc90 ffff88086fd5fc78 ffffffff8108c85a
> > [    3.238000]  ffff88086fd60000 0000000000000092 ffff88086fd60000 00000000000000d0
> > [    3.246000] Call Trace:
> > [    3.249000]  [<ffffffff81773f0a>] dump_stack+0x4c/0x65
> > [    3.255000]  [<ffffffff8108c85a>] warn_slowpath_common+0x8a/0xc0
> > [    3.261000]  [<ffffffff8108c8e5>] warn_slowpath_fmt+0x55/0x70
> > [    3.268000]  [<ffffffff810ee24d>] lockdep_trace_alloc+0xdd/0xe0
> > [    3.274000]  [<ffffffff811cda0d>] __alloc_pages_nodemask+0xad/0xca0
> > [    3.281000]  [<ffffffff810ec7ad>] ? __lock_acquire+0xf6d/0x1560
> > [    3.288000]  [<ffffffff81219c8a>] alloc_page_interleave+0x3a/0x90
> > [    3.295000]  [<ffffffff8121b32d>] alloc_pages_current+0x17d/0x1a0
> > [    3.301000]  [<ffffffff811c869e>] ? __get_free_pages+0xe/0x50
> > [    3.308000]  [<ffffffff811c869e>] __get_free_pages+0xe/0x50
> > [    3.314000]  [<ffffffff8102640b>] init_espfix_ap+0x17b/0x320
> > [    3.320000]  [<ffffffff8105c691>] start_secondary+0xf1/0x1f0
> > [    3.327000] ---[ end trace 1b3327d9d6a1d62c ]---
> > 
> > As we alloc pages with GFP_KERNEL in init_espfix_ap() which is called
> > before enabled local irq, and the lockdep sub-system considers this
> > behaviour as allocating memory with GFP_FS with local irq disabled,
> > then trigger the warning as mentioned about.
> > 
> > Though we could allocate them on the boot CPU side and hand them over to
> > the secondary CPU, but it seemes a bit waste if some of cpus are offline.
> > As thers is no need to these pages(espfix stack) until we try to run user
> > code, so we postpone the initialization of espfix stack, and let the boot
> > up routine init the espfix stack for the target cpu after it booted to
> > avoid the noise.
> > 
> 
> It isn't *at all* obvious to me at least that if the GFP_KERNEL
> allocation fails we may not get rescheduled on another CPU and/or get stuck.
> 
> I'm starting to think that the right thing to do is to allocate these on
> the CPU that is bringing up the other CPU, at the same time we allocate
> the percpu area.  This won't affect offline CPUs.

Btw, as part of experimenting for something else, I was able to trigger
this even on a guest here. It is an insane guest though: 16 NUMA nodes,
with 8 cores each:

[    0.032000] ------------[ cut here ]------------
[    0.032000] WARNING: CPU: 64 PID: 0 at kernel/locking/lockdep.c:2755 lockdep_trace_alloc+0x10c/0x120()
[    0.032000] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
[    0.032000] Modules linked in:
[    0.032000] CPU: 64 PID: 0 Comm: swapper/64 Not tainted 4.1.0-rc3+ #4
[    0.032000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[    0.032000]  ffffffff818dd1a1 ffff880006a1fca8 ffffffff816a9685 0000000000000000
[    0.032000]  ffff880006a1fcf8 ffff880006a1fce8 ffffffff81058585 00000000001d74c0
[    0.032000]  0000000000000080 0000000000000046 ffff880047ffcd00 ffff88003e804058
[    0.032000] Call Trace:
[    0.032000]  [<ffffffff816a9685>] dump_stack+0x4f/0x7b
[    0.032000]  [<ffffffff81058585>] warn_slowpath_common+0x95/0xe0
[    0.032000]  [<ffffffff81058616>] warn_slowpath_fmt+0x46/0x50
[    0.032000]  [<ffffffff810a662c>] lockdep_trace_alloc+0x10c/0x120
[    0.032000]  [<ffffffff8113d6ed>] __alloc_pages_nodemask+0xad/0xab0
[    0.032000]  [<ffffffff813442d7>] ? debug_smp_processor_id+0x17/0x20
[    0.032000]  [<ffffffff810a370e>] ? put_lock_stats.isra.19+0xe/0x30
[    0.032000]  [<ffffffff816ae288>] ? mutex_lock_nested+0x2e8/0x420
[    0.032000]  [<ffffffff8117e0cc>] alloc_page_interleave+0x3c/0x90
[    0.032000]  [<ffffffff8117e995>] alloc_pages_current+0xc5/0xd0
[    0.032000]  [<ffffffff81138734>] __get_free_pages+0x14/0x50
[    0.032000]  [<ffffffff8100a484>] init_espfix_ap.part.5+0x164/0x270
[    0.032000]  [<ffffffff8100a5b1>] init_espfix_ap+0x21/0x30
[    0.032000]  [<ffffffff8103cd28>] start_secondary+0xe8/0x180
[    0.032000] ---[ end trace 6a7abdb28fbb7667 ]---

Now I can test the future fix too. :)

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1167145

On 06/17/2015 02:04 PM, Borislav Petkov wrote:
>>
>> It isn't *at all* obvious to me at least that if the GFP_KERNEL
>> allocation fails we may not get rescheduled on another CPU and/or get stuck.
>>
>> I'm starting to think that the right thing to do is to allocate these on
>> the CPU that is bringing up the other CPU, at the same time we allocate
>> the percpu area.  This won't affect offline CPUs.
> 
> Btw, as part of experimenting for something else, I was able to trigger
> this even on a guest here. It is an insane guest though: 16 NUMA nodes,
> with 8 cores each:
> 

Is this reliable?

	-hpa

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1167176

On Wed, Jun 17, 2015 at 02:11:27PM -0700, H. Peter Anvin wrote:
> Is this reliable?

10 out of 10 :)

I'm booting with:

...
-smp 128
-numa node,nodeid=0,cpus=0-7
-numa node,nodeid=1,cpus=8-15
-numa node,nodeid=2,cpus=16-23
-numa node,nodeid=3,cpus=24-31
-numa node,nodeid=4,cpus=32-39
-numa node,nodeid=5,cpus=40-47
-numa node,nodeid=6,cpus=48-55
-numa node,nodeid=7,cpus=56-63
-numa node,nodeid=8,cpus=64-71
-numa node,nodeid=9,cpus=72-79
-numa node,nodeid=10,cpus=80-87
-numa node,nodeid=11,cpus=88-95
-numa node,nodeid=12,cpus=96-103
-numa node,nodeid=13,cpus=104-111
-numa node,nodeid=14,cpus=112-119
-numa node,nodeid=15,cpus=120-127
...

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web