Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1240728

[PATCH tip/core/rcu 11/13] locking/percpu-rwsem: Fix the comments outdated by rcu_sync

Show all headers | View raw

From: Oleg Nesterov <oleg@redhat.com>

Update the comments broken by the previous change.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/locking/percpu-rwsem.c | 50 ++++++++++---------------------------------
 1 file changed, 11 insertions(+), 39 deletions(-)

diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c
index 183a71151ac0..02a726dd9adc 100644
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -39,27 +39,12 @@ void percpu_free_rwsem(struct percpu_rw_semaphore *brw)
 }
 
 /*
- * This is the fast-path for down_read/up_read, it only needs to ensure
- * there is no pending writer (atomic_read(write_ctr) == 0) and inc/dec the
- * fast per-cpu counter. The writer uses synchronize_sched_expedited() to
- * serialize with the preempt-disabled section below.
- *
- * The nontrivial part is that we should guarantee acquire/release semantics
- * in case when
- *
- *	R_W: down_write() comes after up_read(), the writer should see all
- *	     changes done by the reader
- * or
- *	W_R: down_read() comes after up_write(), the reader should see all
- *	     changes done by the writer
+ * This is the fast-path for down_read/up_read. If it succeeds we rely
+ * on the barriers provided by rcu_sync_enter/exit; see the comments in
+ * percpu_down_write() and percpu_up_write().
  *
  * If this helper fails the callers rely on the normal rw_semaphore and
  * atomic_dec_and_test(), so in this case we have the necessary barriers.
- *
- * But if it succeeds we do not have any barriers, atomic_read(write_ctr) or
- * __this_cpu_add() below can be reordered with any LOAD/STORE done by the
- * reader inside the critical section. See the comments in down_write and
- * up_write below.
  */
 static bool update_fast_ctr(struct percpu_rw_semaphore *brw, unsigned int val)
 {
@@ -136,29 +121,15 @@ static int clear_fast_ctr(struct percpu_rw_semaphore *brw)
 	return sum;
 }
 
-/*
- * A writer increments ->write_ctr to force the readers to switch to the
- * slow mode, note the atomic_read() check in update_fast_ctr().
- *
- * After that the readers can only inc/dec the slow ->slow_read_ctr counter,
- * ->fast_read_ctr is stable. Once the writer moves its sum into the slow
- * counter it represents the number of active readers.
- *
- * Finally the writer takes ->rw_sem for writing and blocks the new readers,
- * then waits until the slow counter becomes zero.
- */
 void percpu_down_write(struct percpu_rw_semaphore *brw)
 {
 	/*
-	 * 1. Ensures that write_ctr != 0 is visible to any down_read/up_read
-	 *    so that update_fast_ctr() can't succeed.
-	 *
-	 * 2. Ensures we see the result of every previous this_cpu_add() in
-	 *    update_fast_ctr().
+	 * Make rcu_sync_is_idle() == F and thus disable the fast-path in
+	 * percpu_down_read() and percpu_up_read(), and wait for gp pass.
 	 *
-	 * 3. Ensures that if any reader has exited its critical section via
-	 *    fast-path, it executes a full memory barrier before we return.
-	 *    See R_W case in the comment above update_fast_ctr().
+	 * The latter synchronises us with the preceding readers which used
+	 * the fast-past, so we can not miss the result of __this_cpu_add()
+	 * or anything else inside their criticial sections.
 	 */
 	rcu_sync_enter(&brw->rss);
 
@@ -178,8 +149,9 @@ void percpu_up_write(struct percpu_rw_semaphore *brw)
 	/* release the lock, but the readers can't use the fast-path */
 	up_write(&brw->rw_sem);
 	/*
-	 * Insert the barrier before the next fast-path in down_read,
-	 * see W_R case in the comment above update_fast_ctr().
+	 * Enable the fast-path in percpu_down_read() and percpu_up_read()
+	 * but only after another gp pass; this adds the necessary barrier
+	 * to ensure the reader can't miss the changes done by us.
 	 */
 	rcu_sync_exit(&brw->rss);
 }
-- 
2.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH tip/core/rcu 0/13] percpu-rwsem patches for 4.4 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
  [PATCH tip/core/rcu 02/13] locking/percpu-rwsem: Export symbols for locktorture "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
  [PATCH tip/core/rcu 07/13] rcu_sync: Add CONFIG_PROVE_RCU checks "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
  [PATCH tip/core/rcu 05/13] rcu: Create rcu_sync infrastructure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
  [PATCH tip/core/rcu 01/13] locktorture: Support rtmutex torturing "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 09/13] locking/percpu-rwsem: Make percpu_free_rwsem() after kzalloc() safe "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 04/13] torture: Consolidate cond_resched_rcu_qs() into stutter_wait() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 12/13] locking/percpu-rwsem: Clean up the lockdep annotations in percpu_down_read() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 08/13] rcu_sync: Introduce rcu_sync_dtor() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 06/13] rcu_sync: Simplify rcu_sync using new rcu_sync_ops structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 11/13] locking/percpu-rwsem: Fix the comments outdated by rcu_sync "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:50 +0200
    [PATCH tip/core/rcu 13/13] rcu_sync: Cleanup the CONFIG_PROVE_RCU checks "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 19:00 +0200
    [PATCH tip/core/rcu 03/13] locktorture: Add torture tests for percpu_rwsem "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 19:00 +0200
    [PATCH tip/core/rcu 10/13] locking/percpu-rwsem: Make use of the rcu_sync infrastructure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 19:00 +0200
  Re: [PATCH tip/core/rcu 0/13] percpu-rwsem patches for 4.4 Josh Triplett <josh@joshtriplett.org> - 2015-10-06 19:50 +0200
    Re: [PATCH tip/core/rcu 0/13] percpu-rwsem patches for 4.4 Oleg Nesterov <oleg@redhat.com> - 2015-10-06 21:00 +0200
      Re: [PATCH tip/core/rcu 0/13] percpu-rwsem patches for 4.4 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 21:40 +0200
        Re: [PATCH tip/core/rcu 0/13] percpu-rwsem patches for 4.4 Josh Triplett <josh@joshtriplett.org> - 2015-10-06 22:40 +0200

csiph-web