Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1479655

Re: [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier

Show all headers | View raw

On Thu, Sep 8, 2016 at 5:09 PM, Benjamin Serebrin <serebrin@google.com> wrote:
> Sorry for the delay, I was eaten by a grue.
>
> I found that my initial study did not actually measure the number of
> TLB shootdown IPIs sent per TLB shootdown.  I think the intuition was
> correct but I didn't actually observe what I thought I had; my
> original use of probe points was incorrect.  However, after fixing my
> methodology, I'm having trouble proving that the existing Lazy TLB
> mode is working properly.
>
>
>
> I've spent some time trying to reproduce this in a microbenchmark.
> One thread does mmap, touch page, munmap, while other threads in the
> same process are configured to either busy-spin or busy-spin and
> yield.  All threads set their own affinity to a unique cpu, and the
> system is otherwise idle.  I look at the per-cpu delta of the TLB and
> CAL lines of /proc/interrupts over the run of the microbenchmark.
>
> Let's say I have 4 spin threads that never yield.  The mmap thread
> does N unmaps.  I observe each spin-thread core receives N (+/-  small
> noise) TLB shootdown interrupts, and the total TLB interrupt count is
> 4N (+/- small noise).  This is expected behavior.
>
> Then I add some synchronization:  the unmap thread rendezvouses with
> all the spinners, and when they are all ready, the spinners busy-spin
> for D milliseconds and then yield (pthread_yield, sched_yield produce
> identical results, though I'm not confident here that this is the
> right yield).  Meanwhile, the unmap thread busy-spins for D+E
> milliseconds and then does M map/touch/unmaps.  (D, E are single-digit
> milliseconds).  The idea here is that the unmap happens a little while
> after the spinners yielded; the kernel should be in the user process'
> mm but lazy TLB mode should defer TLB flushes.  It seems that lazy
> mode on each CPU should take 1 interrupt and then suppress subsequent
> interrupts.

If they're busy threads, shouldn't the yield return immediately
because the threads are still ready to run?  Lazy TLB won't do much
unless you get the kernel in some state where it's running in the
context of a different kernel thread and hasn't switched to
swapper_pg_dir.  IIRC idle works like that, but you'd need to actually
sleep to go idle.

--Andy

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-25 21:10 +0200
  Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-25 22:00 +0200
  [PATCH RFC v2] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-25 22:20 +0200
  [PATCH RFC v3] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-25 23:10 +0200
    Re: [PATCH RFC v3] x86,mm,sched: make lazy TLB mode even lazier Ingo Molnar <mingo@kernel.org> - 2016-08-27 10:10 +0200
      Re: [PATCH RFC v3] x86,mm,sched: make lazy TLB mode even lazier Linus Torvalds <torvalds@linux-foundation.org> - 2016-08-28 01:10 +0200
        [PATCH RFC v4] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-30 22:00 +0200
          [PATCH RFC v5] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-30 23:10 +0200
        [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-31 18:40 +0200
          Re: [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier Ingo Molnar <mingo@kernel.org> - 2016-09-08 09:00 +0200
            Re: [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier Benjamin Serebrin <serebrin@google.com> - 2016-09-09 02:20 +0200
              Re: [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier Andy Lutomirski <luto@amacapital.net> - 2016-09-09 06:50 +0200
                Re: [PATCH RFC v6] x86,mm,sched: make lazy TLB mode even lazier Peter Zijlstra <peterz@infradead.org> - 2016-09-09 09:50 +0200
      Re: [PATCH RFC v3] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-29 17:30 +0200
  Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier "H. Peter Anvin" <hpa@zytor.com> - 2016-08-26 00:50 +0200
    Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-29 18:10 +0200
  Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Andy Lutomirski <luto@amacapital.net> - 2016-08-28 10:20 +0200
    Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Rik van Riel <riel@redhat.com> - 2016-08-29 17:10 +0200
      Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Andy Lutomirski <luto@amacapital.net> - 2016-08-30 02:00 +0200
        Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier "H. Peter Anvin" <hpa@zytor.com> - 2016-08-30 03:20 +0200
          Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier Andy Lutomirski <luto@amacapital.net> - 2016-08-30 20:30 +0200

csiph-web