[PATCH v3 00/11] PCID and improved laziness

From	Andy Lutomirski <luto@kernel.org>
Newsgroups	linux.kernel
Subject	[PATCH v3 00/11] PCID and improved laziness
Date	2017-06-21 07:30 +0200
Message-ID	<tUGnf-1Tn-3@gated-at.bofh.it> (permalink)
Organization	linux.* mail to news gateway

Show all headers | View raw

There are three performance benefits here:

1. TLB flushing is slow.  (I.e. the flush itself takes a while.)
   This avoids many of them when switching tasks by using PCID.  In
   a stupid little benchmark I did, it saves about 100ns on my laptop
   per context switch.  I'll try to improve that benchmark.

2. Mms that have been used recently on a given CPU might get to keep
   their TLB entries alive across process switches with this patch
   set.  TLB fills are pretty fast on modern CPUs, but they're even
   faster when they don't happen.

3. Lazy TLB is way better.  We used to do two stupid things when we
   ran kernel threads: we'd send IPIs to flush user contexts on their
   CPUs and then we'd write to CR3 for no particular reason as an excuse
   to stop further IPIs.  With this patch, we do neither.

This will, in general, perform suboptimally if paravirt TLB flushing
is in use (currently just Xen, I think, but Hyper-V is in the works).
The code is structured so we could fix it in one of two ways: we
could take a spinlock when touching the percpu state so we can update
it remotely after a paravirt flush, or we could be more careful about
our exactly how we access the state and use cmpxchg16b to do atomic
remote updates.  (On SMP systems without cmpxchg16b, we'd just skip
the optimization entirely.)

This is based on tip:x86/mm.  The branch is here if you want to play:
https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/log/?h=x86/pcid

Changes from v2:
 - Add some Acks
 - Move the reentrancy issue to the beginning.
   (I also sent the same patch as a standalone fix -- it's just in here
    so that this series applies to x86/mm.)
 - Fix some comments.

Changes from RFC:
 - flush_tlb_func_common() no longer gets reentered (Nadav)
 - Fix ASID corruption on unlazying (kbuild bot)
 - Move Xen init to the right place
 - Misc cleanups

Andy Lutomirski (11):
  x86/mm: Don't reenter flush_tlb_func_common()
  x86/ldt: Simplify LDT switching logic
  x86/mm: Remove reset_lazy_tlbstate()
  x86/mm: Give each mm TLB flush generation a unique ID
  x86/mm: Track the TLB's tlb_gen and update the flushing algorithm
  x86/mm: Rework lazy TLB mode and TLB freshness tracking
  x86/mm: Stop calling leave_mm() in idle code
  x86/mm: Disable PCID on 32-bit kernels
  x86/mm: Add nopcid to turn off PCID
  x86/mm: Enable CR4.PCIDE on supported systems
  x86/mm: Try to preserve old TLB entries using PCID

 Documentation/admin-guide/kernel-parameters.txt |   2 +
 arch/ia64/include/asm/acpi.h                    |   2 -
 arch/x86/include/asm/acpi.h                     |   2 -
 arch/x86/include/asm/disabled-features.h        |   4 +-
 arch/x86/include/asm/mmu.h                      |  25 +-
 arch/x86/include/asm/mmu_context.h              |  40 ++-
 arch/x86/include/asm/processor-flags.h          |   2 +
 arch/x86/include/asm/tlbflush.h                 |  89 +++++-
 arch/x86/kernel/cpu/bugs.c                      |   8 +
 arch/x86/kernel/cpu/common.c                    |  33 +++
 arch/x86/kernel/smpboot.c                       |   1 -
 arch/x86/mm/init.c                              |   2 +-
 arch/x86/mm/tlb.c                               | 368 +++++++++++++++---------
 arch/x86/xen/enlighten_pv.c                     |   6 +
 arch/x86/xen/mmu_pv.c                           |   3 +-
 drivers/acpi/processor_idle.c                   |   2 -
 drivers/idle/intel_idle.c                       |   9 +-
 17 files changed, 430 insertions(+), 168 deletions(-)

-- 
2.9.4

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[PATCH v3 00/11] PCID and improved laziness Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
  [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:40 +0200
      Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:50 +0200
      Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 05:00 +0200
        Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-22 14:30 +0200
          Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 20:20 +0200
            Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-22 23:30 +0200
              Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-23 05:10 +0200
                Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Thomas Gleixner <tglx@linutronix.de> - 2017-06-23 09:30 +0200
    Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Nadav Amit <nadav.amit@gmail.com> - 2017-06-22 18:20 +0200
      Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using PCID Andy Lutomirski <luto@kernel.org> - 2017-06-22 20:20 +0200
    Re: [PATCH v3 11/11] x86/mm: Try to preserve old TLB entries using  PCID Borislav Petkov <bp@alien8.de> - 2017-06-26 18:10 +0200
  [PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:30 +0200
    Re: [PATCH v3 08/11] x86/mm: Disable PCID on 32-bit kernels Borislav Petkov <bp@alien8.de> - 2017-06-23 11:30 +0200
  [PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
    Re: [PATCH v3 02/11] x86/ldt: Simplify LDT switching logic Borislav Petkov <bp@alien8.de> - 2017-06-21 11:50 +0200
    [tip:x86/mm] x86/ldt: Simplify the LDT switching logic tip-bot for Andy Lutomirski <tipbot@zytor.com> - 2017-06-22 13:20 +0200
  [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
    Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Borislav Petkov <bp@alien8.de> - 2017-06-21 10:50 +0200
      Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-21 17:20 +0200
    Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Nadav Amit <nadav.amit@gmail.com> - 2017-06-22 01:30 +0200
      Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Andy Lutomirski <luto@kernel.org> - 2017-06-22 04:30 +0200
        Re: [PATCH v3 01/11] x86/mm: Don't reenter flush_tlb_func_common() Ingo Molnar <mingo@kernel.org> - 2017-06-22 09:40 +0200
  [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness  tracking Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:10 +0200
      Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:30 +0200
        Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness  tracking Borislav Petkov <bp@alien8.de> - 2017-06-21 19:30 +0200
    Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness  tracking Borislav Petkov <bp@alien8.de> - 2017-06-22 17:00 +0200
      Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-22 19:50 +0200
        Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness  tracking Borislav Petkov <bp@alien8.de> - 2017-06-22 21:10 +0200
    Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness  tracking Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-06-23 15:40 +0200
      Re: [PATCH v3 06/11] x86/mm: Rework lazy TLB mode and TLB freshness tracking Andy Lutomirski <luto@kernel.org> - 2017-06-23 17:30 +0200
  [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 10:10 +0200
    Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 12:40 +0200
      Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:40 +0200
        Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 19:10 +0200
    Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Borislav Petkov <bp@alien8.de> - 2017-06-21 19:50 +0200
      Re: [PATCH v3 04/11] x86/mm: Give each mm TLB flush generation a  unique ID Andy Lutomirski <luto@kernel.org> - 2017-06-22 04:40 +0200
  [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:40 +0200
      Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 15:50 +0200
      Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-21 22:40 +0200
    Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Borislav Petkov <bp@alien8.de> - 2017-06-23 14:00 +0200
      Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Andy Lutomirski <luto@kernel.org> - 2017-06-23 17:30 +0200
    Re: [PATCH v3 10/11] x86/mm: Enable CR4.PCIDE on supported systems Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-06-23 15:40 +0200
  [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Andy Lutomirski <luto@kernel.org> - 2017-06-21 07:30 +0200
    Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Thomas Gleixner <tglx@linutronix.de> - 2017-06-21 11:30 +0200
      Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Andy Lutomirski <luto@kernel.org> - 2017-06-21 18:30 +0200
    Re: [PATCH v3 07/11] x86/mm: Stop calling leave_mm() in idle code Borislav Petkov <bp@alien8.de> - 2017-06-23 11:10 +0200
  Re: [PATCH v3 00/11] PCID and improved laziness Linus Torvalds <torvalds@linux-foundation.org> - 2017-06-21 20:30 +0200
    Re: [PATCH v3 00/11] PCID and improved laziness Andy Lutomirski <luto@kernel.org> - 2017-06-22 07:30 +0200

csiph-web