Re: sem_lock() vs qspinlocks

From	Boqun Feng <boqun.feng@gmail.com>
Newsgroups	linux.kernel
Subject	Re: sem_lock() vs qspinlocks
Date	2016-05-25 08:40 +0200
Message-ID	<rCAE2-646-15@gated-at.bofh.it> (permalink)
References	(3 earlier) <rAUxc-7bu-3@gated-at.bofh.it> <rAV9T-7EI-15@gated-at.bofh.it> <rAW5Y-8mK-7@gated-at.bofh.it> <rBX9D-5Jd-13@gated-at.bofh.it> <rC2j0-xs-37@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Mon, May 23, 2016 at 10:52:09AM -0700, Linus Torvalds wrote:
> On Mon, May 23, 2016 at 5:25 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > Paul has smp_mb__after_unlock_lock() for the RCpc 'upgrade'. How about
> > something like:
> >
> >         smp_mb__after_lock()
> 
> I'd much rather make the naming be higher level. It's not necessarily

Speak of higher level, I realize that problem here is similar to the
problem we discussed last year:

http://lkml.kernel.org/r/20151112070915.GC6314@fixme-laptop.cn.ibm.com

the problem here is about synchronization between two spinlocks and that
problem is about synchronization between a spinlock and ordinary
variables.

(One result of this similarity is that qspinlock on x86 may be also
broken in the do_exit() code as spinlocks on AARCH64 and PPC. Because
a variable LOAD inside a qspinlock critical section could be reordered
before the STORE part of a qspinlock acquisition.)

For the problem we found last year, the current solution for AARCH64 and
PPC is to have a little heavy weight spin_unlock_wait() to pair with
spin_lock():

AARCH64: http://lkml.kernel.org/r/1448624646-15863-1-git-send-email-will.deacon@arm.com
PPC: http://lkml.kernel.org/r/1461130033-70898-1-git-send-email-boqun.feng@gmail.com (not merged yet)

Another solution works on PPC is what Paul Mckenney suggested, using
smp_mb__after_unlock_lock():

http://lkml.kernel.org/r/20151112144004.GU3972@linux.vnet.ibm.com

, which is petty much the same as the spinlock synchronization
primitive we are discussing about here.

So I'm thinking, if we are going to introduce some primitives for
synchronizing two spinlocks (or even a spinlock and a mutex) anyway,
could we be a little more higher level, to reuse/invent primitives to
solve the synchronzing problem we have between
spinlocks(spin_unlock_wait()) and normal variables?

One benefit of this is that we could drop the complex implementations of
spin_unlock_wait() on AARCH64 and PPC.

Thoughts?

Regards,
Boqun

> going to be a "mb", and while the problem is about smp, the primitives
> it is synchronizing aren't actually smp-specific (ie you're
> synchronizing a lock that is relevant on UP too).
> 
> So I'd just call it something like
> 
>         spin_lock_sync_after_lock();
> 
> because different locks might have different levels of serialization
> (ie maybe a spinlock needs one thing, and a mutex needs another - if
> we start worrying about ordering between spin_lock and
> mutex_is_locked(), for example, or between mutex_lock() and
> spin_is_locked()).
> 
> Hmm?
> 
>                      Linus

Thread

sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-20 07:40 +0200
  Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 10:00 +0200
    Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-20 17:10 +0200
      Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 17:10 +0200
        Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-20 17:30 +0200
        Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 17:30 +0200
      Re: sem_lock() vs qspinlocks Waiman Long <waiman.long@hpe.com> - 2016-05-20 22:50 +0200
        Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 23:00 +0200
          Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-21 03:00 +0200
            Re: sem_lock() vs qspinlocks Waiman Long <waiman.long@hpe.com> - 2016-05-21 06:10 +0200
              Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-21 09:50 +0200
  Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 10:00 +0200
  Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 10:20 +0200
    Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 10:20 +0200
      Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 11:40 +0200
  Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 10:40 +0200
    Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 11:10 +0200
      Re: sem_lock() vs qspinlocks Ingo Molnar <mingo@kernel.org> - 2016-05-20 12:10 +0200
        Re: sem_lock() vs qspinlocks Mel Gorman <mgorman@techsingularity.net> - 2016-05-20 12:50 +0200
  Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 14:00 +0200
    Re: sem_lock() vs qspinlocks Boqun Feng <boqun.feng@gmail.com> - 2016-05-20 16:10 +0200
      Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 17:30 +0200
        Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 18:10 +0200
          Re: sem_lock() vs qspinlocks Linus Torvalds <torvalds@linux-foundation.org> - 2016-05-20 19:10 +0200
            Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 23:10 +0200
              Re: sem_lock() vs qspinlocks Linus Torvalds <torvalds@linux-foundation.org> - 2016-05-20 23:50 +0200
                Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-21 02:50 +0200
                Re: sem_lock() vs qspinlocks Linus Torvalds <torvalds@linux-foundation.org> - 2016-05-21 04:40 +0200
                Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-21 09:40 +0200
                Re: sem_lock() vs qspinlocks Manfred Spraul <manfred@colorfullife.com> - 2016-05-21 15:50 +0200
                Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-24 13:00 +0200
                Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-21 19:20 +0200
            Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-23 14:30 +0200
              Re: sem_lock() vs qspinlocks Linus Torvalds <torvalds@linux-foundation.org> - 2016-05-23 20:00 +0200
                Re: sem_lock() vs qspinlocks Boqun Feng <boqun.feng@gmail.com> - 2016-05-25 08:40 +0200
          Re: sem_lock() vs qspinlocks Manfred Spraul <manfred@colorfullife.com> - 2016-05-22 10:50 +0200
            Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-22 11:40 +0200
    Re: sem_lock() vs qspinlocks Davidlohr Bueso <dave@stgolabs.net> - 2016-05-20 18:30 +0200
    Re: sem_lock() vs qspinlocks Waiman Long <waiman.long@hpe.com> - 2016-05-20 22:50 +0200
      Re: sem_lock() vs qspinlocks Peter Zijlstra <peterz@infradead.org> - 2016-05-20 23:00 +0200

csiph-web