Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1500355
| From | Will Deacon <will.deacon@arm.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: [PATCH -v4 8/8] locking/mutex: Enable optimistic spinning of woken waiter |
| Date | 2016-10-13 17:30 +0200 |
| Message-ID | <srQ7f-5ge-13@gated-at.bofh.it> (permalink) |
| References | <spEWB-2dt-5@gated-at.bofh.it> <spFfX-2lv-31@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
On Fri, Oct 07, 2016 at 04:52:51PM +0200, Peter Zijlstra wrote:
> From: Waiman Long <Waiman.Long@hpe.com>
>
> This patch makes the waiter that sets the HANDOFF flag start spinning
> instead of sleeping until the handoff is complete or the owner
> sleeps. Otherwise, the handoff will cause the optimistic spinners to
> abort spinning as the handed-off owner may not be running.
>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Tim Chen <tim.c.chen@linux.intel.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Imre Deak <imre.deak@intel.com>
> Cc: Jason Low <jason.low2@hpe.com>
> Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
> Cc: Ding Tianhong <dingtianhong@huawei.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Will Deacon <Will.Deacon@arm.com>
> Signed-off-by: Waiman Long <Waiman.Long@hpe.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> kernel/locking/mutex.c | 77 ++++++++++++++++++++++++++++++++++---------------
> 1 file changed, 54 insertions(+), 23 deletions(-)
>
> --- a/kernel/locking/mutex.c
> +++ b/kernel/locking/mutex.c
> @@ -416,24 +416,39 @@ static inline int mutex_can_spin_on_owne
> *
> * Returns true when the lock was taken, otherwise false, indicating
> * that we need to jump to the slowpath and sleep.
> + *
> + * The waiter flag is set to true if the spinner is a waiter in the wait
> + * queue. The waiter-spinner will spin on the lock directly and concurrently
> + * with the spinner at the head of the OSQ, if present, until the owner is
> + * changed to itself.
> */
> static bool mutex_optimistic_spin(struct mutex *lock,
> - struct ww_acquire_ctx *ww_ctx, const bool use_ww_ctx)
> + struct ww_acquire_ctx *ww_ctx,
> + const bool use_ww_ctx, const bool waiter)
> {
> struct task_struct *task = current;
>
> - if (!mutex_can_spin_on_owner(lock))
> - goto done;
> + if (!waiter) {
> + /*
> + * The purpose of the mutex_can_spin_on_owner() function is
> + * to eliminate the overhead of osq_lock() and osq_unlock()
> + * in case spinning isn't possible. As a waiter-spinner
> + * is not going to take OSQ lock anyway, there is no need
> + * to call mutex_can_spin_on_owner().
> + */
> + if (!mutex_can_spin_on_owner(lock))
> + goto fail;
>
> - /*
> - * In order to avoid a stampede of mutex spinners trying to
> - * acquire the mutex all at once, the spinners need to take a
> - * MCS (queued) lock first before spinning on the owner field.
> - */
> - if (!osq_lock(&lock->osq))
> - goto done;
> + /*
> + * In order to avoid a stampede of mutex spinners trying to
> + * acquire the mutex all at once, the spinners need to take a
> + * MCS (queued) lock first before spinning on the owner field.
> + */
> + if (!osq_lock(&lock->osq))
> + goto fail;
> + }
>
> - while (true) {
> + for (;;) {
> struct task_struct *owner;
>
> if (use_ww_ctx && ww_ctx->acquired > 0) {
> @@ -449,7 +464,7 @@ static bool mutex_optimistic_spin(struct
> * performed the optimistic spinning cannot be done.
> */
> if (READ_ONCE(ww->ctx))
> - break;
> + goto fail_unlock;
> }
>
> /*
> @@ -457,15 +472,20 @@ static bool mutex_optimistic_spin(struct
> * release the lock or go to sleep.
> */
> owner = __mutex_owner(lock);
> - if (owner && !mutex_spin_on_owner(lock, owner))
> - break;
> + if (owner) {
> + if (waiter && owner == task) {
> + smp_mb(); /* ACQUIRE */
Hmm, is this barrier actually needed? This only happens on the handoff path,
and we take the wait_lock immediately after this succeeds anyway. That
control dependency, coupled with the acquire semantics of the spin_lock,
should be sufficient, no?
Will
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH -v4 0/8] locking/mutex: Rewrite basic mutex Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:10 +0200
[PATCH -v4 4/8] locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:10 +0200
[PATCH -v4 7/8] locking/mutex: Simplify some ww_mutex code in __mutex_lock_common() Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:10 +0200
[PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:30 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Will Deacon <will.deacon@arm.com> - 2016-10-13 17:20 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-17 12:50 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-17 15:30 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Boqun Feng <boqun.feng@gmail.com> - 2016-10-17 15:50 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-17 18:00 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-19 19:40 +0200
ciao set_task_state() (was Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop) Davidlohr Bueso <dave@stgolabs.net> - 2016-10-24 04:00 +0200
Re: ciao set_task_state() (was Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop) Kent Overstreet <kent.overstreet@gmail.com> - 2016-10-24 15:30 +0200
Re: ciao set_task_state() (was Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop) Kent Overstreet <kent.overstreet@gmail.com> - 2016-10-24 16:30 +0200
Re: ciao set_task_state() (was Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop) Eric Wheeler <bcache@lists.ewheeler.net> - 2016-10-25 19:00 +0200
Re: ciao set_task_state() (was Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop) Kent Overstreet <kent.overstreet@gmail.com> - 2016-10-25 19:50 +0200
Re: [PATCH -v4 6/8] locking/mutex: Restructure wait loop Peter Zijlstra <peterz@infradead.org> - 2016-10-18 15:20 +0200
[PATCH -v4 8/8] locking/mutex: Enable optimistic spinning of woken waiter Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:30 +0200
Re: [PATCH -v4 8/8] locking/mutex: Enable optimistic spinning of woken waiter Will Deacon <will.deacon@arm.com> - 2016-10-13 17:30 +0200
Re: [PATCH -v4 8/8] locking/mutex: Enable optimistic spinning of woken waiter Peter Zijlstra <peterz@infradead.org> - 2016-10-17 11:40 +0200
Re: [PATCH -v4 8/8] locking/mutex: Enable optimistic spinning of woken waiter Peter Zijlstra <peterz@infradead.org> - 2016-10-18 14:30 +0200
[PATCH -v4 2/8] locking/mutex: Rework mutex::owner Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:30 +0200
Re: [PATCH -v4 2/8] locking/mutex: Rework mutex::owner Davidlohr Bueso <dave@stgolabs.net> - 2016-10-12 20:40 +0200
Re: [PATCH -v4 2/8] locking/mutex: Rework mutex::owner Jason Low <jason.low2@hpe.com> - 2016-10-12 22:00 +0200
Re: [PATCH -v4 2/8] locking/mutex: Rework mutex::owner Will Deacon <will.deacon@arm.com> - 2016-10-13 17:20 +0200
[PATCH -v4 5/8] locking/mutex: Add lock handoff to avoid starvation Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:40 +0200
Re: [PATCH -v4 5/8] locking/mutex: Add lock handoff to avoid starvation Will Deacon <will.deacon@arm.com> - 2016-10-13 17:20 +0200
Re: [PATCH -v4 5/8] locking/mutex: Add lock handoff to avoid starvation Peter Zijlstra <peterz@infradead.org> - 2016-10-17 11:30 +0200
Re: [PATCH -v4 5/8] locking/mutex: Add lock handoff to avoid starvation Peter Zijlstra <peterz@infradead.org> - 2016-10-18 14:40 +0200
Re: [PATCH -v4 5/8] locking/mutex: Add lock handoff to avoid starvation Peter Zijlstra <peterz@infradead.org> - 2016-10-18 15:10 +0200
[PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:40 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:50 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Linus Torvalds <torvalds@linux-foundation.org> - 2016-10-07 18:00 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-07 18:20 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Thomas Gleixner <tglx@linutronix.de> - 2016-10-08 14:10 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Thomas Gleixner <tglx@linutronix.de> - 2016-10-08 16:20 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-08 18:50 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-08 16:20 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-18 14:50 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Peter Zijlstra <peterz@infradead.org> - 2016-10-18 15:00 +0200
Re: [PATCH -v4 1/8] locking/drm: Kill mutex trickery Chris Wilson <chris@chris-wilson.co.uk> - 2016-10-18 15:00 +0200
[PATCH -v4 3/8] locking/mutex: Kill arch specific code Peter Zijlstra <peterz@infradead.org> - 2016-10-07 17:40 +0200
Re: [PATCH -v4 0/8] locking/mutex: Rewrite basic mutex Linus Torvalds <torvalds@linux-foundation.org> - 2016-10-07 17:50 +0200
Re: [PATCH -v4 0/8] locking/mutex: Rewrite basic mutex Jason Low <jason.low2@hpe.com> - 2016-10-11 21:10 +0200
csiph-web