Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1390042

[tip:sched/core] x86/mm, sched/core: Turn off IRQs in switch_mm()

Show all headers | View raw

Commit-ID:  078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a
Gitweb:     http://git.kernel.org/tip/078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Tue, 26 Apr 2016 09:39:09 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 28 Apr 2016 11:44:20 +0200

x86/mm, sched/core: Turn off IRQs in switch_mm()

Potential races between switch_mm() and TLB-flush or LDT-flush IPIs
could be very messy.  AFAICT the code is currently okay, whether by
accident or by careful design, but enabling PCID will make it
considerably more complicated and will no longer be obviously safe.

Fix it with a big hammer: run switch_mm() with IRQs off.

To avoid a performance hit in the scheduler, we take advantage of
our knowledge that the scheduler already has IRQs disabled when it
calls switch_mm().

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/include/asm/mmu_context.h |  3 +++
 arch/x86/mm/tlb.c                  | 10 ++++++++++
 2 files changed, 13 insertions(+)

diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index bb911dd..39634819 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -118,6 +118,9 @@ static inline void destroy_context(struct mm_struct *mm)
 extern void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 		      struct task_struct *tsk);
 
+extern void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
+			       struct task_struct *tsk);
+#define switch_mm_irqs_off switch_mm_irqs_off
 
 #define activate_mm(prev, next)			\
 do {						\
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index ce7a0c9..5643fd0 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -64,6 +64,16 @@ EXPORT_SYMBOL_GPL(leave_mm);
 void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 	       struct task_struct *tsk)
 {
+	unsigned long flags;
+
+	local_irq_save(flags);
+	switch_mm_irqs_off(prev, next, tsk);
+	local_irq_restore(flags);
+}
+
+void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
+			struct task_struct *tsk)
+{
 	unsigned cpu = smp_processor_id();
 
 	if (likely(prev != next)) {

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v2 0/5] x86 switch_mm uninlining and IRQ improvements Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:40 +0200
  [PATCH v2 3/5] x86/mm: Build arch/x86/mm/tlb.c even on !SMP Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:40 +0200
    [tip:sched/core] x86/mm: Build arch/x86/mm/tlb.c even on !SMP tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2016-04-28 12:50 +0200
  [PATCH v2 4/5] x86/mm: Uninline switch_mm Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:40 +0200
    [tip:sched/core] x86/mm, sched/core: Uninline switch_mm() tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2016-04-28 12:50 +0200
  [PATCH v2 5/5] x86/mm: Turn off IRQs in switch_mm Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:40 +0200
    [tip:sched/core] x86/mm, sched/core: Turn off IRQs in switch_mm() tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2016-04-28 12:50 +0200
  [PATCH v2 1/5] arm: Include linux/preempt.h from asm/mmu_context.h Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:40 +0200
    [tip:sched/core] sched/core, ARM: Include linux/preempt.h from  asm/mmu_context.h tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2016-04-28 12:40 +0200
      Re: [tip:sched/core] sched/core, ARM: Include linux/preempt.h from  asm/mmu_context.h Russell King - ARM Linux <linux@arm.linux.org.uk> - 2016-04-28 12:50 +0200
    Re: [PATCH v2 1/5] arm: Include linux/preempt.h from  asm/mmu_context.h Russell King - ARM Linux <linux@arm.linux.org.uk> - 2016-04-28 15:30 +0200
  [PATCH v2 2/5] sched: Add switch_mm_irqs_off and use it in the scheduler Andy Lutomirski <luto@kernel.org> - 2016-04-26 18:50 +0200
    [tip:sched/core] sched/core: Add switch_mm_irqs_off() and use it in  the scheduler tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2016-04-28 12:50 +0200
  Re: [PATCH v2 0/5] x86 switch_mm uninlining and IRQ improvements Borislav Petkov <bp@alien8.de> - 2016-04-27 20:10 +0200

csiph-web