Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1622453 > unrolled thread

[PATCH tip/core/rcu 0/40] SRCU callback parallelization for 4.12

Back to article view | Back to linux.kernel

Contents

  [PATCH tip/core/rcu 0/40] SRCU callback parallelization for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:50 +0200
    [PATCH tip/core/rcu 03/40] srcu: Consolidate batch checking into rcu_all_batches_empty() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 20:00 +0200
    [PATCH tip/core/rcu 05/40] rcu: Semicolon inside RCU_TRACE() for rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 20:00 +0200
    [PATCH tip/core/rcu 11/40] rcu: Pull rcu_sched_qs_mask into rcu_dynticks structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 20:00 +0200
    Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling Peter Zijlstra <peterz@infradead.org> - 2017-04-13 12:00 +0200
      Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 18:40 +0200
        Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 19:50 +0200
    Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling Peter Zijlstra <peterz@infradead.org> - 2017-04-13 12:00 +0200
      Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 19:00 +0200
    [PATCH v2 tip/core/rcu 0/40] SRCU callback parallelization for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 02/39] rcu: Make arch select smp_mb__after_unlock_lock() strength "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
        Re: [PATCH v2 tip/core/rcu 02/39] rcu: Make arch select  smp_mb__after_unlock_lock() strength Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:20 +0200
      [PATCH v2 tip/core/rcu 10/39] rcu: Eliminate flavor scan in rcu_momentary_dyntick_idle() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 24/39] srcu: Move combining-tree definitions for SRCU's benefit "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 31/39] srcu: Allow a second bit in rcu_seq for SRCU state "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 15/39] srcu: Allow early boot use of synchronize_srcu() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 14/39] srcu: Allow SRCU to access rcu_scheduler_active "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 05/39] rcu: Semicolon inside RCU_TRACE() for rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 33/39] srcu: Crude control of expedited grace periods "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 29/39] srcu: Fix bogus try_check_zero() comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 25/39] srcu: Move rcu_init_levelspread() to rcu_tree_node.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 27/39] srcu: Move rcu_node traversal macros to rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 06/39] rcu: Semicolon inside RCU_TRACE() for Tiny RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 39/39] rcu: Make non-preemptive schedule be Tasks RCU quiescent state "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 07/39] rcu: Semicolon inside RCU_TRACE() for tree.c "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 18/39] rcu: Expedited wakeups need to be fully ordered "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 21/39] srcu: Move to state-based grace-period sequencing "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 22/39] srcu: Add grace-period sequence numbers "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 34/39] mm: Use static initialization for "srcu" "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 35/39] srcu: Create a tiny SRCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 16/39] rcu: Add single-element dequeue functions to rcu_segcblist "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:50 +0200
      [PATCH v2 tip/core/rcu 19/39] rcu: Fix warning in rcu_seq_end() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 03/39] srcu: Consolidate batch checking into rcu_all_batches_empty() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
        Re: [PATCH v2 tip/core/rcu 03/39] srcu: Consolidate batch checking  into rcu_all_batches_empty() Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:40 +0200
      [PATCH v2 tip/core/rcu 08/39] rcu: Pull rcu_sched_qs_mask into rcu_dynticks structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 09/39] rcu: Pull rcu_qs_ctr into rcu_dynticks structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 30/39] srcu: Improve rcu_seq grace-period-counter abstraction "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 17/39] srcu: Move rcu_seq_start() and friends to rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 11/39] rcu: Place guard on rcu_all_qs() and rcu_note_context_switch() actions "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 28/39] srcu: Make num_rcu_lvl[] array be external "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 26/39] rcu: Remove redundant levelcnt[] array from rcu_init_one() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 32/39] srcu: Merge ->srcu_state into ->srcu_gp_seq "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 36/39] srcutorture: Print Tiny SRCU reader statistics "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
      [PATCH v2 tip/core/rcu 01/39] rcu: Maintain special bits at bottom of ->dynticks counter "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
        Re: [PATCH v2 tip/core/rcu 01/39] rcu: Maintain special bits at  bottom of ->dynticks counter Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:10 +0200
          Re: [PATCH v2 tip/core/rcu 01/39] rcu: Maintain special bits at  bottom of ->dynticks counter "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 20:30 +0200
      [PATCH v2 tip/core/rcu 04/39] srcu: Check for tardy grace-period activity in cleanup_srcu_struct() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 02:00 +0200
        Re: [PATCH v2 tip/core/rcu 04/39] srcu: Check for tardy grace-period  activity in cleanup_srcu_struct() Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:40 +0200
          Re: [PATCH v2 tip/core/rcu 04/39] srcu: Check for tardy grace-period  activity in cleanup_srcu_struct() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 20:40 +0200
            Re: [PATCH v2 tip/core/rcu 04/39] srcu: Check for tardy grace-period  activity in cleanup_srcu_struct() Josh Triplett <josh@joshtriplett.org> - 2017-04-18 21:50 +0200
        Re: [PATCH v2 tip/core/rcu 04/39] srcu: Check for tardy grace-period  activity in cleanup_srcu_struct() Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:40 +0200
      [PATCH v3 tip/core/rcu 0/40] SRCU callback parallelization for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:00 +0200
        [PATCH v3 tip/core/rcu 35/40] srcu: Create a tiny SRCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:00 +0200
        [PATCH v3 tip/core/rcu 12/40] rcu: Default RCU_FANOUT_LEAF to 16 unless explicitly changed "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:00 +0200
        [PATCH v3 tip/core/rcu 05/40] rcu: Semicolon inside RCU_TRACE() for rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
          Re: [PATCH v3 tip/core/rcu 05/40] rcu: Semicolon inside RCU_TRACE()  for rcu.h Joe Perches <joe@perches.com> - 2017-04-19 19:50 +0200
        [PATCH v3 tip/core/rcu 26/40] rcu: Remove redundant levelcnt[] array from rcu_init_one() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 14/40] srcu: Allow SRCU to access rcu_scheduler_active "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 27/40] srcu: Move rcu_node traversal macros to rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 19/40] rcu: Fix warning in rcu_seq_end() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 29/40] srcu: Fix bogus try_check_zero() comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 28/40] srcu: Make num_rcu_lvl[] array be external "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 22/40] srcu: Add grace-period sequence numbers "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 39/40] srcu: Expedite srcu_schedule_cbs_snp() callback invocation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 30/40] srcu: Improve rcu_seq grace-period-counter abstraction "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 01/40] rcu: Maintain special bits at bottom of ->dynticks counter "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 20/40] srcu: Push srcu_advance_batches() fastpath into common case "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 34/40] mm: Use static initialization for "srcu" "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 17/40] srcu: Move rcu_seq_start() and friends to rcu.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 08/40] rcu: Pull rcu_sched_qs_mask into rcu_dynticks structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 36/40] srcutorture: Print Tiny SRCU reader statistics "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 09/40] rcu: Pull rcu_qs_ctr into rcu_dynticks structure "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 04/40] srcu: Check for tardy grace-period activity in cleanup_srcu_struct() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 07/40] rcu: Semicolon inside RCU_TRACE() for tree.c "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 10/40] rcu: Eliminate flavor scan in rcu_momentary_dyntick_idle() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 02/40] rcu: Make arch select smp_mb__after_unlock_lock() strength "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 24/40] srcu: Move combining-tree definitions for SRCU's benefit "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 15/40] srcu: Allow early boot use of synchronize_srcu() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 33/40] srcu: Crude control of expedited grace periods "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 06/40] rcu: Semicolon inside RCU_TRACE() for Tiny RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 03/40] srcu: Consolidate batch checking into rcu_all_batches_empty() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 32/40] srcu: Merge ->srcu_state into ->srcu_gp_seq "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
        [PATCH v3 tip/core/rcu 25/40] srcu: Move rcu_init_levelspread() to rcu_tree_node.h "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:10 +0200
    powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling) Michael Ellerman <michaele@au1.ibm.com> - 2017-04-20 05:50 +0200
      Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-20 16:30 +0200
      Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) Paolo Bonzini <pbonzini@redhat.com> - 2017-04-20 17:30 +0200
        Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-21 02:40 +0200
          Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling) Michael Ellerman <michaele@au1.ibm.com> - 2017-04-21 03:50 +0200
            Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-21 06:20 +0200
              Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) Paolo Bonzini <pbonzini@redhat.com> - 2017-04-21 09:30 +0200
                Re: powerpc KVM build break in linux-next (was Re: [PATCH  tip/core/rcu 40/40] srcu: Parallelize callback handling) "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-21 15:00 +0200
                  Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling) Michael Ellerman <michaele@au1.ibm.com> - 2017-04-22 08:20 +0200
        Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling) Michael Ellerman <michaele@au1.ibm.com> - 2017-04-21 04:20 +0200

Page 5 of 5 — ← Prev page 1 2 3 4 [5]

#1626602 — [PATCH v3 tip/core/rcu 03/40] srcu: Consolidate batch checking into rcu_all_batches_empty()

The srcu_reschedule() function invokes rcu_batch_empty() on each of
the four rcu_batch structures in the srcu_struct in question twice.
Given that this check will also be needed in cleanup_srcu_struct(), this
commit consolidates these four checks into a new rcu_all_batches_empty()
function.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
---
 kernel/rcu/srcu.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/kernel/rcu/srcu.c b/kernel/rcu/srcu.c
index ef3bcfb15b39..ba41a5d04b49 100644
--- a/kernel/rcu/srcu.c
+++ b/kernel/rcu/srcu.c
@@ -65,6 +65,17 @@ static inline bool rcu_batch_empty(struct rcu_batch *b)
 }
 
 /*
+ * Are all batches empty for the specified srcu_struct?
+ */
+static inline bool rcu_all_batches_empty(struct srcu_struct *sp)
+{
+	return rcu_batch_empty(&sp->batch_done) &&
+	       rcu_batch_empty(&sp->batch_check1) &&
+	       rcu_batch_empty(&sp->batch_check0) &&
+	       rcu_batch_empty(&sp->batch_queue);
+}
+
+/*
  * Remove the callback at the head of the specified rcu_batch structure
  * and return a pointer to it, or return NULL if the structure is empty.
  */
@@ -619,15 +630,9 @@ static void srcu_reschedule(struct srcu_struct *sp)
 {
 	bool pending = true;
 
-	if (rcu_batch_empty(&sp->batch_done) &&
-	    rcu_batch_empty(&sp->batch_check1) &&
-	    rcu_batch_empty(&sp->batch_check0) &&
-	    rcu_batch_empty(&sp->batch_queue)) {
+	if (rcu_all_batches_empty(sp)) {
 		spin_lock_irq(&sp->queue_lock);
-		if (rcu_batch_empty(&sp->batch_done) &&
-		    rcu_batch_empty(&sp->batch_check1) &&
-		    rcu_batch_empty(&sp->batch_check0) &&
-		    rcu_batch_empty(&sp->batch_queue)) {
+		if (rcu_all_batches_empty(sp)) {
 			sp->running = false;
 			pending = false;
 		}
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1626603 — [PATCH v3 tip/core/rcu 32/40] srcu: Merge ->srcu_state into ->srcu_gp_seq

Updating ->srcu_state and ->srcu_gp_seq will lead to extremely complex
race conditions given multiple callback queues, so this commit takes
advantage of the two-bit state now available in rcu_seq counters to
store the state in the bottom two bits of ->srcu_gp_seq.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/srcu.h |  5 +----
 kernel/rcu/rcu.h     | 10 ++++++++++
 kernel/rcu/srcu.c    | 55 +++++++++++++++++++++++++++++++++-------------------
 3 files changed, 46 insertions(+), 24 deletions(-)

diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index ad154a7bc114..e7dbc01b61a1 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -43,8 +43,7 @@ struct srcu_struct {
 	unsigned long completed;
 	unsigned long srcu_gp_seq;
 	struct srcu_array __percpu *per_cpu_ref;
-	spinlock_t queue_lock; /* protect ->srcu_cblist, ->srcu_state */
-	int srcu_state;
+	spinlock_t queue_lock; /* protect ->srcu_cblist */
 	struct rcu_segcblist srcu_cblist;
 	struct delayed_work work;
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
@@ -56,7 +55,6 @@ struct srcu_struct {
 #define SRCU_STATE_IDLE		0
 #define SRCU_STATE_SCAN1	1
 #define SRCU_STATE_SCAN2	2
-#define SRCU_STATE_DONE		3
 
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
 
@@ -85,7 +83,6 @@ void process_srcu(struct work_struct *work);
 		.completed = -300,					\
 		.per_cpu_ref = &name##_srcu_array,			\
 		.queue_lock = __SPIN_LOCK_UNLOCKED(name.queue_lock),	\
-		.srcu_state = SRCU_STATE_IDLE,				\
 		.srcu_cblist = RCU_SEGCBLIST_INITIALIZER(name.srcu_cblist),\
 		.work = __DELAYED_WORK_INITIALIZER(name.work, process_srcu, 0),\
 		__SRCU_DEP_MAP_INIT(name)				\
diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 87a0ac95b551..73e16ec4054b 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -82,6 +82,16 @@ static inline int rcu_seq_state(unsigned long s)
 	return s & RCU_SEQ_STATE_MASK;
 }
 
+/*
+ * Set the state portion of the pointed-to sequence number.
+ * The caller is responsible for preventing conflicting updates.
+ */
+static inline void rcu_seq_set_state(unsigned long *sp, int newstate)
+{
+	WARN_ON_ONCE(newstate & ~RCU_SEQ_STATE_MASK);
+	WRITE_ONCE(*sp, (*sp & ~RCU_SEQ_STATE_MASK) + newstate);
+}
+
 /* Adjust sequence number for start of update-side operation. */
 static inline void rcu_seq_start(unsigned long *sp)
 {
diff --git a/kernel/rcu/srcu.c b/kernel/rcu/srcu.c
index 1a2dc74bb625..90ffea31b188 100644
--- a/kernel/rcu/srcu.c
+++ b/kernel/rcu/srcu.c
@@ -44,7 +44,6 @@ static int init_srcu_struct_fields(struct srcu_struct *sp)
 	sp->completed = 0;
 	sp->srcu_gp_seq = 0;
 	spin_lock_init(&sp->queue_lock);
-	sp->srcu_state = SRCU_STATE_IDLE;
 	rcu_segcblist_init(&sp->srcu_cblist);
 	INIT_DELAYED_WORK(&sp->work, process_srcu);
 	sp->per_cpu_ref = alloc_percpu(struct srcu_array);
@@ -180,6 +179,9 @@ static bool srcu_readers_active(struct srcu_struct *sp)
 	return sum;
 }
 
+#define SRCU_CALLBACK_BATCH	10
+#define SRCU_INTERVAL		1
+
 /**
  * cleanup_srcu_struct - deconstruct a sleep-RCU structure
  * @sp: structure to clean up.
@@ -200,8 +202,10 @@ void cleanup_srcu_struct(struct srcu_struct *sp)
 	if (WARN_ON(!rcu_segcblist_empty(&sp->srcu_cblist)))
 		return; /* Leakage unless caller handles error. */
 	flush_delayed_work(&sp->work);
-	if (WARN_ON(READ_ONCE(sp->srcu_state) != SRCU_STATE_IDLE))
+	if (WARN_ON(rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) != SRCU_STATE_IDLE)) {
+		pr_info("cleanup_srcu_struct: Active srcu_struct %lu CBs %c state: %d\n", rcu_segcblist_n_cbs(&sp->srcu_cblist), ".E"[rcu_segcblist_empty(&sp->srcu_cblist)], rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)));
 		return; /* Caller forgot to stop doing call_srcu()? */
+	}
 	free_percpu(sp->per_cpu_ref);
 	sp->per_cpu_ref = NULL;
 }
@@ -253,10 +257,13 @@ EXPORT_SYMBOL_GPL(__srcu_read_unlock);
  */
 static void srcu_gp_start(struct srcu_struct *sp)
 {
+	int state;
+
 	rcu_segcblist_accelerate(&sp->srcu_cblist,
 				 rcu_seq_snap(&sp->srcu_gp_seq));
-	WRITE_ONCE(sp->srcu_state, SRCU_STATE_SCAN1);
 	rcu_seq_start(&sp->srcu_gp_seq);
+	state = rcu_seq_state(READ_ONCE(sp->srcu_gp_seq));
+	WARN_ON_ONCE(state != SRCU_STATE_SCAN1);
 }
 
 /*
@@ -300,7 +307,6 @@ static void srcu_flip(struct srcu_struct *sp)
 static void srcu_gp_end(struct srcu_struct *sp)
 {
 	rcu_seq_end(&sp->srcu_gp_seq);
-	WRITE_ONCE(sp->srcu_state, SRCU_STATE_DONE);
 
 	spin_lock_irq(&sp->queue_lock);
 	rcu_segcblist_advance(&sp->srcu_cblist,
@@ -345,7 +351,7 @@ void call_srcu(struct srcu_struct *sp, struct rcu_head *head,
 	spin_lock_irqsave(&sp->queue_lock, flags);
 	smp_mb__after_unlock_lock(); /* Caller's prior accesses before GP. */
 	rcu_segcblist_enqueue(&sp->srcu_cblist, head, false);
-	if (READ_ONCE(sp->srcu_state) == SRCU_STATE_IDLE) {
+	if (rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) == SRCU_STATE_IDLE) {
 		srcu_gp_start(sp);
 		queue_delayed_work(system_power_efficient_wq, &sp->work, 0);
 	}
@@ -378,7 +384,7 @@ static void __synchronize_srcu(struct srcu_struct *sp, int trycount)
 	head->func = wakeme_after_rcu;
 	spin_lock_irq(&sp->queue_lock);
 	smp_mb__after_unlock_lock(); /* Caller's prior accesses before GP. */
-	if (READ_ONCE(sp->srcu_state) == SRCU_STATE_IDLE) {
+	if (rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) == SRCU_STATE_IDLE) {
 		/* steal the processing owner */
 		rcu_segcblist_enqueue(&sp->srcu_cblist, head, false);
 		srcu_gp_start(sp);
@@ -480,9 +486,6 @@ unsigned long srcu_batches_completed(struct srcu_struct *sp)
 }
 EXPORT_SYMBOL_GPL(srcu_batches_completed);
 
-#define SRCU_CALLBACK_BATCH	10
-#define SRCU_INTERVAL		1
-
 /*
  * Core SRCU state machine.  Advance callbacks from ->batch_check0 to
  * ->batch_check1 and then to ->batch_done as readers drain.
@@ -491,28 +494,40 @@ static void srcu_advance_batches(struct srcu_struct *sp, int trycount)
 {
 	int idx;
 
-	WARN_ON_ONCE(sp->srcu_state == SRCU_STATE_IDLE);
-
 	/*
 	 * Because readers might be delayed for an extended period after
 	 * fetching ->completed for their index, at any point in time there
 	 * might well be readers using both idx=0 and idx=1.  We therefore
 	 * need to wait for readers to clear from both index values before
 	 * invoking a callback.
+	 *
+	 * The load-acquire ensures that we see the accesses performed
+	 * by the prior grace period.
 	 */
+	idx = rcu_seq_state(smp_load_acquire(&sp->srcu_gp_seq)); /* ^^^ */
+	if (idx == SRCU_STATE_IDLE) {
+		spin_lock_irq(&sp->queue_lock);
+		if (rcu_segcblist_empty(&sp->srcu_cblist)) {
+			spin_unlock_irq(&sp->queue_lock);
+			return;
+		}
+		idx = rcu_seq_state(READ_ONCE(sp->srcu_gp_seq));
+		if (idx == SRCU_STATE_IDLE)
+			srcu_gp_start(sp);
+		spin_unlock_irq(&sp->queue_lock);
+		if (idx != SRCU_STATE_IDLE)
+			return; /* Someone else started the grace period. */
+	}
 
-	if (sp->srcu_state == SRCU_STATE_DONE)
-		srcu_gp_start(sp);
-
-	if (sp->srcu_state == SRCU_STATE_SCAN1) {
+	if (rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) == SRCU_STATE_SCAN1) {
 		idx = 1 ^ (sp->completed & 1);
 		if (!try_check_zero(sp, idx, trycount))
 			return; /* readers present, retry after SRCU_INTERVAL */
 		srcu_flip(sp);
-		WRITE_ONCE(sp->srcu_state, SRCU_STATE_SCAN2);
+		rcu_seq_set_state(&sp->srcu_gp_seq, SRCU_STATE_SCAN2);
 	}
 
-	if (sp->srcu_state == SRCU_STATE_SCAN2) {
+	if (rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) == SRCU_STATE_SCAN2) {
 
 		/*
 		 * SRCU read-side critical sections are normally short,
@@ -563,14 +578,14 @@ static void srcu_invoke_callbacks(struct srcu_struct *sp)
 static void srcu_reschedule(struct srcu_struct *sp, unsigned long delay)
 {
 	bool pending = true;
+	int state;
 
 	if (rcu_segcblist_empty(&sp->srcu_cblist)) {
 		spin_lock_irq(&sp->queue_lock);
+		state = rcu_seq_state(READ_ONCE(sp->srcu_gp_seq));
 		if (rcu_segcblist_empty(&sp->srcu_cblist) &&
-		    READ_ONCE(sp->srcu_state) == SRCU_STATE_DONE) {
-			WRITE_ONCE(sp->srcu_state, SRCU_STATE_IDLE);
+		    state == SRCU_STATE_IDLE)
 			pending = false;
-		}
 		spin_unlock_irq(&sp->queue_lock);
 	}
 
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1626604 — [PATCH v3 tip/core/rcu 25/40] srcu: Move rcu_init_levelspread() to rcu_tree_node.h

This commit moves the rcu_init_levelspread() function from
kernel/rcu/tree.c to kernel/rcu/rcu.h so that SRCU can access it.  This is
another step towards enabling SRCU to create its own combining tree.
This commit is code-movement only, give or take knock-on adjustments.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/rcu_node_tree.h |  3 ---
 kernel/rcu/rcu.h              | 36 ++++++++++++++++++++++++++++++++++++
 kernel/rcu/srcu.c             |  1 +
 kernel/rcu/tree.c             | 25 -------------------------
 kernel/rcu/tree_trace.c       |  1 +
 5 files changed, 38 insertions(+), 28 deletions(-)

diff --git a/include/linux/rcu_node_tree.h b/include/linux/rcu_node_tree.h
index b7eb97096b1c..4b766b61e1a0 100644
--- a/include/linux/rcu_node_tree.h
+++ b/include/linux/rcu_node_tree.h
@@ -96,7 +96,4 @@
 # error "CONFIG_RCU_FANOUT insufficient for NR_CPUS"
 #endif /* #if (NR_CPUS) <= RCU_FANOUT_1 */
 
-extern int rcu_num_lvls;
-extern int rcu_num_nodes;
-
 #endif /* __LINUX_RCU_NODE_TREE_H */
diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index a943b42a9cf7..87326479b39a 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -190,4 +190,40 @@ void rcu_test_sync_prims(void);
  */
 extern void resched_cpu(int cpu);
 
+#if defined(SRCU) || !defined(TINY_RCU)
+
+#include <linux/rcu_node_tree.h>
+
+extern int rcu_num_lvls;
+extern int rcu_num_nodes;
+static bool rcu_fanout_exact;
+static int rcu_fanout_leaf;
+
+/*
+ * Compute the per-level fanout, either using the exact fanout specified
+ * or balancing the tree, depending on the rcu_fanout_exact boot parameter.
+ */
+static inline void rcu_init_levelspread(int *levelspread, const int *levelcnt)
+{
+	int i;
+
+	if (rcu_fanout_exact) {
+		levelspread[rcu_num_lvls - 1] = rcu_fanout_leaf;
+		for (i = rcu_num_lvls - 2; i >= 0; i--)
+			levelspread[i] = RCU_FANOUT;
+	} else {
+		int ccur;
+		int cprv;
+
+		cprv = nr_cpu_ids;
+		for (i = rcu_num_lvls - 1; i >= 0; i--) {
+			ccur = levelcnt[i];
+			levelspread[i] = (cprv + ccur - 1) / ccur;
+			cprv = ccur;
+		}
+	}
+}
+
+#endif /* #if defined(SRCU) || !defined(TINY_RCU) */
+
 #endif /* __LINUX_RCU_H */
diff --git a/kernel/rcu/srcu.c b/kernel/rcu/srcu.c
index 56fd30862122..0b511de7ca4d 100644
--- a/kernel/rcu/srcu.c
+++ b/kernel/rcu/srcu.c
@@ -36,6 +36,7 @@
 #include <linux/delay.h>
 #include <linux/srcu.h>
 
+#include <linux/rcu_node_tree.h>
 #include "rcu.h"
 
 static int init_srcu_struct_fields(struct srcu_struct *sp)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 844a030c1960..df3527744af8 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3952,31 +3952,6 @@ void rcu_scheduler_starting(void)
 }
 
 /*
- * Compute the per-level fanout, either using the exact fanout specified
- * or balancing the tree, depending on the rcu_fanout_exact boot parameter.
- */
-static void __init rcu_init_levelspread(int *levelspread, const int *levelcnt)
-{
-	int i;
-
-	if (rcu_fanout_exact) {
-		levelspread[rcu_num_lvls - 1] = rcu_fanout_leaf;
-		for (i = rcu_num_lvls - 2; i >= 0; i--)
-			levelspread[i] = RCU_FANOUT;
-	} else {
-		int ccur;
-		int cprv;
-
-		cprv = nr_cpu_ids;
-		for (i = rcu_num_lvls - 1; i >= 0; i--) {
-			ccur = levelcnt[i];
-			levelspread[i] = (cprv + ccur - 1) / ccur;
-			cprv = ccur;
-		}
-	}
-}
-
-/*
  * Helper function for rcu_init() that initializes one rcu_state structure.
  */
 static void __init rcu_init_one(struct rcu_state *rsp)
diff --git a/kernel/rcu/tree_trace.c b/kernel/rcu/tree_trace.c
index 066c64071a7b..30c5bf89ee58 100644
--- a/kernel/rcu/tree_trace.c
+++ b/kernel/rcu/tree_trace.c
@@ -45,6 +45,7 @@
 
 #define RCU_TREE_NONCORE
 #include "tree.h"
+#include "rcu.h"
 
 static int r_open(struct inode *inode, struct file *file,
 					const struct seq_operations *op)
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1626958 — powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:

> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h
> index f2b3bd6c6bc2..0400e211aa44 100644
> --- a/include/linux/srcutree.h
> +++ b/include/linux/srcutree.h
> @@ -24,25 +24,75 @@
...
> +/*
> + * Per-SRCU-domain structure, similar in function to rcu_state.
> + */
>  struct srcu_struct {
> -	unsigned long completed;
> -	unsigned long srcu_gp_seq;
> -	atomic_t srcu_exp_cnt;
> -	struct srcu_array __percpu *per_cpu_ref;
> -	spinlock_t queue_lock; /* protect ->srcu_cblist */
> -	struct rcu_segcblist srcu_cblist;
> +	struct srcu_node node[NUM_RCU_NODES];	/* Combining tree. */
> +	struct srcu_node *level[RCU_NUM_LVLS + 1];
> +						/* First node at each level. */
> +	struct mutex srcu_cb_mutex;		/* Serialize CB preparation. */
> +	spinlock_t gp_lock;			/* protect ->srcu_cblist */
> +	struct mutex srcu_gp_mutex;		/* Serialize GP work. */
> +	unsigned int srcu_idx;			/* Current rdr array element. */
> +	unsigned long srcu_gp_seq;		/* Grace-period seq #. */
> +	unsigned long srcu_gp_seq_needed;	/* Latest gp_seq needed. */
> +	atomic_t srcu_exp_cnt;			/* # ongoing expedited GPs. */
> +	struct srcu_data __percpu *sda;		/* Per-CPU srcu_data array. */
> +	unsigned long srcu_barrier_seq;		/* srcu_barrier seq #. */
> +	struct mutex srcu_barrier_mutex;	/* Serialize barrier ops. */
> +	struct completion srcu_barrier_completion;
> +						/* Awaken barrier rq at end. */
> +	atomic_t srcu_barrier_cpu_cnt;		/* # CPUs not yet posting a */
> +						/*  callback for the barrier */
> +						/*  operation. */

This change seems to have had the non-obvious effect of breaking the
powerpc KVM build.

Because struct kvm contains two srcu_structs which are located
before the kvm_arch struct, the increase in size of srcu_struct has
caused the offset from the start of struct kvm to kvm_arch to be too big
for some of our asm.

struct kvm {
	spinlock_t mmu_lock;
	struct mutex slots_lock;
	struct mm_struct *mm; /* userspace tied to this vm */
	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
	struct srcu_struct srcu;
	struct srcu_struct irq_srcu;
        ...
	struct kvm_arch arch;


Example error:
  arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)

Where line 617 is:
  lwz	r7,KVM_LPID(r9)

And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs
new is:

  -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)	 #
  +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)	 #


We can probably just fix it by changing the asm to keep the address of
kvm_arch in a register, and then offset from that.

cheers

[toc] | [prev] | [next] | [standalone]

#1627536 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On Thu, Apr 20, 2017 at 01:40:13PM +1000, Michael Ellerman wrote:
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> 
> > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h
> > index f2b3bd6c6bc2..0400e211aa44 100644
> > --- a/include/linux/srcutree.h
> > +++ b/include/linux/srcutree.h
> > @@ -24,25 +24,75 @@
> ...
> > +/*
> > + * Per-SRCU-domain structure, similar in function to rcu_state.
> > + */
> >  struct srcu_struct {
> > -	unsigned long completed;
> > -	unsigned long srcu_gp_seq;
> > -	atomic_t srcu_exp_cnt;
> > -	struct srcu_array __percpu *per_cpu_ref;
> > -	spinlock_t queue_lock; /* protect ->srcu_cblist */
> > -	struct rcu_segcblist srcu_cblist;
> > +	struct srcu_node node[NUM_RCU_NODES];	/* Combining tree. */
> > +	struct srcu_node *level[RCU_NUM_LVLS + 1];
> > +						/* First node at each level. */
> > +	struct mutex srcu_cb_mutex;		/* Serialize CB preparation. */
> > +	spinlock_t gp_lock;			/* protect ->srcu_cblist */
> > +	struct mutex srcu_gp_mutex;		/* Serialize GP work. */
> > +	unsigned int srcu_idx;			/* Current rdr array element. */
> > +	unsigned long srcu_gp_seq;		/* Grace-period seq #. */
> > +	unsigned long srcu_gp_seq_needed;	/* Latest gp_seq needed. */
> > +	atomic_t srcu_exp_cnt;			/* # ongoing expedited GPs. */
> > +	struct srcu_data __percpu *sda;		/* Per-CPU srcu_data array. */
> > +	unsigned long srcu_barrier_seq;		/* srcu_barrier seq #. */
> > +	struct mutex srcu_barrier_mutex;	/* Serialize barrier ops. */
> > +	struct completion srcu_barrier_completion;
> > +						/* Awaken barrier rq at end. */
> > +	atomic_t srcu_barrier_cpu_cnt;		/* # CPUs not yet posting a */
> > +						/*  callback for the barrier */
> > +						/*  operation. */
> 
> This change seems to have had the non-obvious effect of breaking the
> powerpc KVM build.
> 
> Because struct kvm contains two srcu_structs which are located
> before the kvm_arch struct, the increase in size of srcu_struct has
> caused the offset from the start of struct kvm to kvm_arch to be too big
> for some of our asm.
> 
> struct kvm {
> 	spinlock_t mmu_lock;
> 	struct mutex slots_lock;
> 	struct mm_struct *mm; /* userspace tied to this vm */
> 	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
> 	struct srcu_struct srcu;
> 	struct srcu_struct irq_srcu;
>         ...
> 	struct kvm_arch arch;
> 
> 
> Example error:
>   arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)
> 
> Where line 617 is:
>   lwz	r7,KVM_LPID(r9)
> 
> And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs
> new is:
> 
>   -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)	 #
>   +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)	 #
> 
> 
> We can probably just fix it by changing the asm to keep the address of
> kvm_arch in a register, and then offset from that.

Ouch!

What do you need from me?

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1627593 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On 20/04/2017 05:40, Michael Ellerman wrote:
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> 
>> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h
>> index f2b3bd6c6bc2..0400e211aa44 100644
>> --- a/include/linux/srcutree.h
>> +++ b/include/linux/srcutree.h
>> @@ -24,25 +24,75 @@
> ...
>> +/*
>> + * Per-SRCU-domain structure, similar in function to rcu_state.
>> + */
>>  struct srcu_struct {
>> -	unsigned long completed;
>> -	unsigned long srcu_gp_seq;
>> -	atomic_t srcu_exp_cnt;
>> -	struct srcu_array __percpu *per_cpu_ref;
>> -	spinlock_t queue_lock; /* protect ->srcu_cblist */
>> -	struct rcu_segcblist srcu_cblist;
>> +	struct srcu_node node[NUM_RCU_NODES];	/* Combining tree. */
>> +	struct srcu_node *level[RCU_NUM_LVLS + 1];
>> +						/* First node at each level. */
>> +	struct mutex srcu_cb_mutex;		/* Serialize CB preparation. */
>> +	spinlock_t gp_lock;			/* protect ->srcu_cblist */
>> +	struct mutex srcu_gp_mutex;		/* Serialize GP work. */
>> +	unsigned int srcu_idx;			/* Current rdr array element. */
>> +	unsigned long srcu_gp_seq;		/* Grace-period seq #. */
>> +	unsigned long srcu_gp_seq_needed;	/* Latest gp_seq needed. */
>> +	atomic_t srcu_exp_cnt;			/* # ongoing expedited GPs. */
>> +	struct srcu_data __percpu *sda;		/* Per-CPU srcu_data array. */
>> +	unsigned long srcu_barrier_seq;		/* srcu_barrier seq #. */
>> +	struct mutex srcu_barrier_mutex;	/* Serialize barrier ops. */
>> +	struct completion srcu_barrier_completion;
>> +						/* Awaken barrier rq at end. */
>> +	atomic_t srcu_barrier_cpu_cnt;		/* # CPUs not yet posting a */
>> +						/*  callback for the barrier */
>> +						/*  operation. */
> 
> This change seems to have had the non-obvious effect of breaking the
> powerpc KVM build.
> 
> Because struct kvm contains two srcu_structs which are located
> before the kvm_arch struct, the increase in size of srcu_struct has
> caused the offset from the start of struct kvm to kvm_arch to be too big
> for some of our asm.
> 
> struct kvm {
> 	spinlock_t mmu_lock;
> 	struct mutex slots_lock;
> 	struct mm_struct *mm; /* userspace tied to this vm */
> 	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
> 	struct srcu_struct srcu;
> 	struct srcu_struct irq_srcu;
>         ...
> 	struct kvm_arch arch;
> 
> 
> Example error:
>   arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)
> 
> Where line 617 is:
>   lwz	r7,KVM_LPID(r9)
> 
> And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs
> new is:
> 
>   -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)	 #
>   +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)	 #
> 
> 
> We can probably just fix it by changing the asm to keep the address of
> kvm_arch in a register, and then offset from that.

Michael, if you want to move the two srcu structs at the end of struct
kvm, that would be fine by me.  Please send a patch yourself so you can
test it on PPC.  Thanks,

Paolo

[toc] | [prev] | [next] | [standalone]

#1627879 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote:
> 
> 
> On 20/04/2017 05:40, Michael Ellerman wrote:
> > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> > 
> >> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h
> >> index f2b3bd6c6bc2..0400e211aa44 100644
> >> --- a/include/linux/srcutree.h
> >> +++ b/include/linux/srcutree.h
> >> @@ -24,25 +24,75 @@
> > ...
> >> +/*
> >> + * Per-SRCU-domain structure, similar in function to rcu_state.
> >> + */
> >>  struct srcu_struct {
> >> -	unsigned long completed;
> >> -	unsigned long srcu_gp_seq;
> >> -	atomic_t srcu_exp_cnt;
> >> -	struct srcu_array __percpu *per_cpu_ref;
> >> -	spinlock_t queue_lock; /* protect ->srcu_cblist */
> >> -	struct rcu_segcblist srcu_cblist;
> >> +	struct srcu_node node[NUM_RCU_NODES];	/* Combining tree. */
> >> +	struct srcu_node *level[RCU_NUM_LVLS + 1];
> >> +						/* First node at each level. */
> >> +	struct mutex srcu_cb_mutex;		/* Serialize CB preparation. */
> >> +	spinlock_t gp_lock;			/* protect ->srcu_cblist */
> >> +	struct mutex srcu_gp_mutex;		/* Serialize GP work. */
> >> +	unsigned int srcu_idx;			/* Current rdr array element. */
> >> +	unsigned long srcu_gp_seq;		/* Grace-period seq #. */
> >> +	unsigned long srcu_gp_seq_needed;	/* Latest gp_seq needed. */
> >> +	atomic_t srcu_exp_cnt;			/* # ongoing expedited GPs. */
> >> +	struct srcu_data __percpu *sda;		/* Per-CPU srcu_data array. */
> >> +	unsigned long srcu_barrier_seq;		/* srcu_barrier seq #. */
> >> +	struct mutex srcu_barrier_mutex;	/* Serialize barrier ops. */
> >> +	struct completion srcu_barrier_completion;
> >> +						/* Awaken barrier rq at end. */
> >> +	atomic_t srcu_barrier_cpu_cnt;		/* # CPUs not yet posting a */
> >> +						/*  callback for the barrier */
> >> +						/*  operation. */
> > 
> > This change seems to have had the non-obvious effect of breaking the
> > powerpc KVM build.
> > 
> > Because struct kvm contains two srcu_structs which are located
> > before the kvm_arch struct, the increase in size of srcu_struct has
> > caused the offset from the start of struct kvm to kvm_arch to be too big
> > for some of our asm.
> > 
> > struct kvm {
> > 	spinlock_t mmu_lock;
> > 	struct mutex slots_lock;
> > 	struct mm_struct *mm; /* userspace tied to this vm */
> > 	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
> > 	struct srcu_struct srcu;
> > 	struct srcu_struct irq_srcu;
> >         ...
> > 	struct kvm_arch arch;
> > 
> > 
> > Example error:
> >   arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)
> > 
> > Where line 617 is:
> >   lwz	r7,KVM_LPID(r9)
> > 
> > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs
> > new is:
> > 
> >   -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)	 #
> >   +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)	 #
> > 
> > 
> > We can probably just fix it by changing the asm to keep the address of
> > kvm_arch in a register, and then offset from that.
> 
> Michael, if you want to move the two srcu structs at the end of struct
> kvm, that would be fine by me.  Please send a patch yourself so you can
> test it on PPC.  Thanks,

On the off-chance that it is at all helpful, I have added the commit
shown below to -rcu.  If it helps, I am happy to push this, but am
just as happy to drop it in favor of some other fix.  If at all possible,
I would like to get this into the upcoming merge window.

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1627907 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote:
>> On 20/04/2017 05:40, Michael Ellerman wrote:
>> > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
>> > 
>> > This change seems to have had the non-obvious effect of breaking the
>> > powerpc KVM build.
...
>> 
>> Michael, if you want to move the two srcu structs at the end of struct
>> kvm, that would be fine by me.  Please send a patch yourself so you can
>> test it on PPC.  Thanks,
>
> On the off-chance that it is at all helpful, I have added the commit
> shown below to -rcu.

As shown below :)

> If it helps, I am happy to push this, but am just as happy to drop it
> in favor of some other fix. If at all possible, I would like to get
> this into the upcoming merge window.

Thanks, this looks perfect to me, and if you're happy to put it on top
of your tree that would limit the breakage to a smaller history window,
so that would be ideal.

Tested-by: Michael Ellerman <mpe@ellerman.id.au>

cheers


From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date: Thu, 20 Apr 2017 17:30:06 -0700
Subject: kvm: Move srcu_struct fields to end of struct kvm

Parallelizing SRCU callback handling increased the size of srcu_struct,
which moved the kvm_arch field within the kvm struct out of reach of
powerpc's current assembly code, resulting in the following sort of
build error:

arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)

This commit moves the srcu_struct fields in the kvm structure to follow
the kvm_arch field, which again allows powerpc's assembly code to
reach it.

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reported-by: Michael Ellerman <michaele@au1.ibm.com>
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/kvm_host.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 2c14ad9..96c8e29 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -375,8 +375,6 @@ struct kvm {
 	struct mutex slots_lock;
 	struct mm_struct *mm; /* userspace tied to this vm */
 	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
-	struct srcu_struct srcu;
-	struct srcu_struct irq_srcu;
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 
 	/*
@@ -429,6 +427,8 @@ struct kvm {
 	struct list_head devices;
 	struct dentry *debugfs_dentry;
 	struct kvm_stat_data **debugfs_stat_data;
+	struct srcu_struct srcu;
+	struct srcu_struct irq_srcu;
 };
 
 #define kvm_err(fmt, ...) \
-- 
cgit v1.1

[toc] | [prev] | [next] | [standalone]

#1627942 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On Fri, Apr 21, 2017 at 11:42:01AM +1000, Michael Ellerman wrote:
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> > On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote:
> >> On 20/04/2017 05:40, Michael Ellerman wrote:
> >> > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> >> > 
> >> > This change seems to have had the non-obvious effect of breaking the
> >> > powerpc KVM build.
> ...
> >> 
> >> Michael, if you want to move the two srcu structs at the end of struct
> >> kvm, that would be fine by me.  Please send a patch yourself so you can
> >> test it on PPC.  Thanks,
> >
> > On the off-chance that it is at all helpful, I have added the commit
> > shown below to -rcu.
> 
> As shown below :)
> 
> > If it helps, I am happy to push this, but am just as happy to drop it
> > in favor of some other fix. If at all possible, I would like to get
> > this into the upcoming merge window.
> 
> Thanks, this looks perfect to me, and if you're happy to put it on top
> of your tree that would limit the breakage to a smaller history window,
> so that would be ideal.
> 
> Tested-by: Michael Ellerman <mpe@ellerman.id.au>

Thank you, Michael!

Paolo, does this look good to you?  My upstream maintainer will be much
more likely to take this with your ack.  ;-)

							Thanx, Paul

> cheers
> 
> 
> >From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Date: Thu, 20 Apr 2017 17:30:06 -0700
> Subject: kvm: Move srcu_struct fields to end of struct kvm
> 
> Parallelizing SRCU callback handling increased the size of srcu_struct,
> which moved the kvm_arch field within the kvm struct out of reach of
> powerpc's current assembly code, resulting in the following sort of
> build error:
> 
> arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0x000000000000b328 is not between 0xffffffffffff8000 and 0x0000000000007fff)
> 
> This commit moves the srcu_struct fields in the kvm structure to follow
> the kvm_arch field, which again allows powerpc's assembly code to
> reach it.
> 
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Reported-by: Michael Ellerman <michaele@au1.ibm.com>
> Reported-by: kbuild test robot <fengguang.wu@intel.com>
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
>  include/linux/kvm_host.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 2c14ad9..96c8e29 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -375,8 +375,6 @@ struct kvm {
>  	struct mutex slots_lock;
>  	struct mm_struct *mm; /* userspace tied to this vm */
>  	struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM];
> -	struct srcu_struct srcu;
> -	struct srcu_struct irq_srcu;
>  	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
> 
>  	/*
> @@ -429,6 +427,8 @@ struct kvm {
>  	struct list_head devices;
>  	struct dentry *debugfs_dentry;
>  	struct kvm_stat_data **debugfs_stat_data;
> +	struct srcu_struct srcu;
> +	struct srcu_struct irq_srcu;
>  };
> 
>  #define kvm_err(fmt, ...) \
> -- 
> cgit v1.1

[toc] | [prev] | [next] | [standalone]

#1628005 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On 21/04/2017 06:17, Paul E. McKenney wrote:
>> Thanks, this looks perfect to me, and if you're happy to put it on top
>> of your tree that would limit the breakage to a smaller history window,
>> so that would be ideal.
>>
>> Tested-by: Michael Ellerman <mpe@ellerman.id.au>
> Thank you, Michael!
> 
> Paolo, does this look good to you?  My upstream maintainer will be much
> more likely to take this with your ack.  ;-)

Sure,

Acked-by: Paolo Bonzini <pbonzini@redhat.com>

since I don't expect conflicts between whoever applies this patch and
the KVM tree.

Paolo

[toc] | [prev] | [next] | [standalone]

#1628220 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote:
> 
> 
> On 21/04/2017 06:17, Paul E. McKenney wrote:
> >> Thanks, this looks perfect to me, and if you're happy to put it on top
> >> of your tree that would limit the breakage to a smaller history window,
> >> so that would be ideal.
> >>
> >> Tested-by: Michael Ellerman <mpe@ellerman.id.au>
> > Thank you, Michael!
> > 
> > Paolo, does this look good to you?  My upstream maintainer will be much
> > more likely to take this with your ack.  ;-)
> 
> Sure,
> 
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> since I don't expect conflicts between whoever applies this patch and
> the KVM tree.

Applied, thank you!

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1628788 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> writes:
> On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote:
>> On 21/04/2017 06:17, Paul E. McKenney wrote:
>> >> Thanks, this looks perfect to me, and if you're happy to put it on top
>> >> of your tree that would limit the breakage to a smaller history window,
>> >> so that would be ideal.
>> >>
>> >> Tested-by: Michael Ellerman <mpe@ellerman.id.au>
>> > Thank you, Michael!
>> > 
>> > Paolo, does this look good to you?  My upstream maintainer will be much
>> > more likely to take this with your ack.  ;-)
>> 
>> Sure,
>> 
>> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
>> 
>> since I don't expect conflicts between whoever applies this patch and
>> the KVM tree.
>
> Applied, thank you!

Thanks all.

cheers

[toc] | [prev] | [next] | [standalone]

#1627918 — Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)

Paolo Bonzini <pbonzini@redhat.com> writes:
> Michael, if you want to move the two srcu structs at the end of struct
> kvm, that would be fine by me.  Please send a patch yourself so you can
> test it on PPC.  Thanks,

Paul beat me to it :)

I've confirmed it fixes the build break and the resulting kernel can
boot a KVM guest happily.

cheers

[toc] | [prev] | [standalone]

Page 5 of 5 — ← Prev page 1 2 3 4 [5]

Back to top | Article view | linux.kernel

csiph-web