Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1671360
| From | Andy Lutomirski <luto@kernel.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() |
| Date | 2017-06-21 07:30 +0200 |
| Message-ID | <tUGng-1Tn-29@gated-at.bofh.it> (permalink) |
| References | <tUGnf-1Tn-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
It was historically possible to have two concurrent TLB flushes
targetting the same CPU: one initiated locally and one initiated
remotely. This can now cause an OOPS in leave_mm() at
arch/x86/mm/tlb.c:47:
if (this_cpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
BUG();
with this call trace:
flush_tlb_func_local arch/x86/mm/tlb.c:239 [inline]
flush_tlb_mm_range+0x26d/0x370 arch/x86/mm/tlb.c:317
Without reentrancy, this OOPS is impossible: leave_mm() is only
called if we're not in TLBSTATE_OK, but then we're unexpectedly
in TLBSTATE_OK in leave_mm().
This can be caused by flush_tlb_func_remote() happening between
the two checks and calling leave_mm(), resulting in two consecutive
leave_mm() calls on the same CPU with no intervening switch_mm()
calls.
We never saw this OOPS before because the old leave_mm()
implementation didn't put us back in TLBSTATE_OK, so the assertion
didn't fire.
Nadav noticed the reentrancy issue in a different context, but
neither of us realized that it caused a problem yet.
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Reported-by: "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>
Fixes: 3d28ebceaffa ("x86/mm: Rework lazy TLB to track the actual loaded mm")
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/mm/tlb.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 2a5e851f2035..f06239c6919f 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -208,6 +208,9 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
static void flush_tlb_func_common(const struct flush_tlb_info *f,
bool local, enum tlb_flush_reason reason)
{
+ /* This code cannot presently handle being reentered. */
+ VM_WARN_ON(!irqs_disabled());
+
if (this_cpu_read(cpu_tlbstate.state) != TLBSTATE_OK) {
leave_mm(smp_processor_id());
return;
@@ -313,8 +316,12 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
info.end = TLB_FLUSH_ALL;
}
- if (mm == this_cpu_read(cpu_tlbstate.loaded_mm))
+ if (mm == this_cpu_read(cpu_tlbstate.loaded_mm)) {
+ local_irq_disable();
flush_tlb_func_local(&info, TLB_LOCAL_MM_SHOOTDOWN);
+ local_irq_enable();
+ }
+
if (cpumask_any_but(mm_cpumask(mm), cpu) < nr_cpu_ids)
flush_tlb_others(mm_cpumask(mm), &info);
put_cpu();
@@ -370,8 +377,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
int cpu = get_cpu();
- if (cpumask_test_cpu(cpu, &batch->cpumask))
+ if (cpumask_test_cpu(cpu, &batch->cpumask)) {
+ local_irq_disable();
flush_tlb_func_local(&info, TLB_LOCAL_SHOOTDOWN);
+ local_irq_enable();
+ }
+
if (cpumask_any_but(&batch->cpumask, cpu) < nr_cpu_ids)
flush_tlb_others(&batch->cpumask, &info);
cpumask_clear(&batch->cpumask);
--
2.9.4
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH v3 00/11] PCID and improved laziness Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
[PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:40 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:50 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 05:00 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-22 14:30 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 20:20 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-22 23:30 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-23 05:10 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-23 09:30 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Nadav Amit <nadav.amit@gmail.com> - 2017-06-22 18:20 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 20:20 +0200
Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Borislav Petkov <bp@alien8.de> - 2017-06-26 18:10 +0200
[PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:30 +0200
Re: [PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Borislav Petkov <bp@alien8.de> - 2017-06-23 11:30 +0200
[PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
Re: [PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Borislav Petkov <bp@alien8.de> - 2017-06-21 11:50 +0200
[tip:x86/mm] x86/ldt: Simplify the LDT switching logic tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2017-06-22 13:20 +0200
[PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Borislav Petkov <bp@alien8.de> - 2017-06-21 10:50 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-21 17:20 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Nadav Amit <nadav.amit@gmail.com> - 2017-06-22 01:30 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-22 04:30 +0200
Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Ingo Molnar <mingo@kernel.org> - 2017-06-22 09:40 +0200
[PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:10 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:30 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Borislav Petkov <bp@alien8.de> - 2017-06-21 19:30 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Borislav Petkov <bp@alien8.de> - 2017-06-22 17:00 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-22 19:50 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Borislav Petkov <bp@alien8.de> - 2017-06-22 21:10 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-06-23 15:40 +0200
Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-23 17:30 +0200
[PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 12:40 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:40 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 19:10 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 19:50 +0200
Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-22 04:40 +0200
[PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:40 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:50 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-21 22:40 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Borislav Petkov <bp@alien8.de> - 2017-06-23 14:00 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-23 17:30 +0200
Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-06-23 15:40 +0200
[PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:30 +0200
Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:30 +0200
Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Borislav Petkov <bp@alien8.de> - 2017-06-23 11:10 +0200
Re: [PATCH v3 00/11] PCID and improved laziness Linus Torvalds <torvalds@linux-foundation.org> - 2017-06-21 20:30 +0200
Re: [PATCH v3 00/11] PCID and improved laziness Andy Lutomirski <luto@kernel.org> - 2017-06-22 07:30 +0200
csiph-web