Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1396586

[PATCH 4/4] locking/rwsem: Rework zeroing reader waiter->task

Show all headers | View raw

Readers that are awoken will expect a nil ->task indicating
that a wakeup has occurred. There is a mismatch between the
smp_mb() and its documentation, in that the serialization is
done between reading the task and the nil store. Furthermore,
in addition to having the overlapping of loads and stores to
waiter->task guaranteed to be ordered within that CPU, both
wake_up_process() originally and now wake_q_add() already
imply barriers upon successful calls, which serves the comment.

Just atomically do a xchg() and simplify the whole thing. We can
use relaxed semantics as before mentioned in addition to the
barrier provided by wake_q_add(), delaying there is no risk in
reordering with the actual wakeup.

Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
 kernel/locking/rwsem-xadd.c | 17 +++++------------
 1 file changed, 5 insertions(+), 12 deletions(-)

diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c
index 1b8c1285a2aa..96e53cb4a4db 100644
--- a/kernel/locking/rwsem-xadd.c
+++ b/kernel/locking/rwsem-xadd.c
@@ -126,7 +126,6 @@ __rwsem_mark_wake(struct rw_semaphore *sem,
 		  enum rwsem_wake_type wake_type, struct wake_q_head *wake_q)
 {
 	struct rwsem_waiter *waiter;
-	struct task_struct *tsk;
 	struct list_head *next;
 	long oldcount, woken, loop, adjustment;
 
@@ -190,24 +189,18 @@ __rwsem_mark_wake(struct rw_semaphore *sem,
 	next = sem->wait_list.next;
 	loop = woken;
 	do {
+		struct task_struct *tsk;
+
 		waiter = list_entry(next, struct rwsem_waiter, list);
 		next = waiter->list.next;
-		tsk = waiter->task;
-		/*
-		 * Make sure we do not wakeup the next reader before
-		 * setting the nil condition to grant the next reader;
-		 * otherwise we could miss the wakeup on the other
-		 * side and end up sleeping again. See the pairing
-		 * in rwsem_down_read_failed().
-		 */
-		smp_mb();
-		waiter->task = NULL;
+
+		tsk = xchg_relaxed(&waiter->task, NULL);
 		wake_q_add(wake_q, tsk);
 	} while (--loop);
 
 	sem->wait_list.next = next;
 	next->prev = &sem->wait_list;
- out:
+out:
 	return sem;
 }
 
-- 
2.8.1

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH -tip 0/4] locking/rwsem (xadd): Reader waiter optimizations Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 07:00 +0200
  [PATCH 4/4] locking/rwsem: Rework zeroing reader waiter->task Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 07:10 +0200
    Re: [PATCH 4/4] locking/rwsem: Rework zeroing reader waiter->task Peter Zijlstra <peterz@infradead.org> - 2016-05-09 10:00 +0200
  [PATCH 3/4] locking/rwsem: Enable lockless waiter wakeup(s) Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 07:10 +0200
    Re: [PATCH 3/4] locking/rwsem: Enable lockless waiter wakeup(s) Peter Zijlstra <peterz@infradead.org> - 2016-05-09 09:50 +0200
  [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 07:10 +0200
    Re: [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Peter Zijlstra <peterz@infradead.org> - 2016-05-09 09:40 +0200
      Re: [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 18:00 +0200
        Re: [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Peter Zijlstra <peterz@infradead.org> - 2016-05-09 18:20 +0200
          Re: [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 21:00 +0200
            Re: [PATCH 2/4] locking/rwsem: Drop superfluous waiter refcount Peter Zijlstra <peterz@infradead.org> - 2016-05-09 21:10 +0200
  [PATCH 1/4] locking/rwsem: Avoid stale ->count for rwsem_down_write_failed() Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 07:10 +0200
    Re: [PATCH 1/4] locking/rwsem: Avoid stale ->count for  rwsem_down_write_failed() Peter Hurley <peter@hurleysoftware.com> - 2016-05-09 07:40 +0200
      Re: [PATCH 1/4] locking/rwsem: Avoid stale ->count for  rwsem_down_write_failed() Peter Zijlstra <peterz@infradead.org> - 2016-05-09 09:20 +0200
      Re: [PATCH 1/4] locking/rwsem: Avoid stale ->count for  rwsem_down_write_failed() Davidlohr Bueso <dave@stgolabs.net> - 2016-05-09 16:40 +0200
  Re: [PATCH -tip 0/4] locking/rwsem (xadd): Reader waiter optimizations Waiman Long <waiman.long@hpe.com> - 2016-05-10 03:40 +0200

csiph-web