Groups > linux.kernel > #1240676 > unrolled thread

[PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4

Started by	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
First post	2015-10-06 18:30 +0200
Last post	2015-10-06 18:40 +0200
Articles	20 on this page of 66 — 6 participants

Back to article view | Back to linux.kernel

  [PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:30 +0200
    [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() Peter Zijlstra <peterz@infradead.org> - 2015-10-07 16:30 +0200
        Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 18:30 +0200
          Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() Peter Zijlstra <peterz@infradead.org> - 2015-10-08 11:10 +0200
            Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-08 17:10 +0200
              Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() Peter Zijlstra <peterz@infradead.org> - 2015-10-08 17:20 +0200
                Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-08 17:20 +0200
                  Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() Josh Triplett <josh@joshtriplett.org> - 2015-10-08 20:10 +0200
                    Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 02:20 +0200
                      Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() Josh Triplett <josh@joshtriplett.org> - 2015-10-09 02:50 +0200
                        Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for  synchronize_sched_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 06:00 +0200
    [PATCH tip/core/rcu 10/18] rcu: Stop silencing lockdep false positive for expedited grace periods "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 08/18] rcu: Make ->cpu_no_qs be a union for aggregate OR "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-06 22:30 +0200
        Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 23:00 +0200
          Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 10:00 +0200
            Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2015-10-07 10:50 +0200
              Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 13:10 +0200
                Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 14:00 +0200
                  Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 14:10 +0200
                  Re: Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation kbuild test robot <lkp@intel.com> - 2015-10-07 14:10 +0200
                  Re: Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation kbuild test robot <lkp@intel.com> - 2015-10-07 14:10 +0200
                  Re: Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 14:20 +0200
                    Re: [kbuild-all] [PATCH tip/core/rcu 02/18] rcu: Move  rcu_report_exp_rnp() to allow consolidation Fengguang Wu <lkp@intel.com> - 2015-10-07 15:50 +0200
                      Re: [kbuild-all] [PATCH tip/core/rcu 02/18] rcu: Move  rcu_report_exp_rnp() to allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 16:00 +0200
                        Re: [kbuild-all] [PATCH tip/core/rcu 02/18] rcu: Move  rcu_report_exp_rnp() to allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 16:30 +0200
                        Re: [kbuild-all] [PATCH tip/core/rcu 02/18] rcu: Move  rcu_report_exp_rnp() to allow consolidation Fengguang Wu <lkp@intel.com> - 2015-10-07 16:30 +0200
                  Re: Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation kbuild test robot <lkp@intel.com> - 2015-10-07 14:20 +0200
                  Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 17:20 +0200
                    Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-08 12:30 +0200
                Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 17:20 +0200
            Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 16:40 +0200
              Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-07 16:50 +0200
                Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 18:50 +0200
                  Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-08 11:50 +0200
                    Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-08 17:40 +0200
                      Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-08 19:20 +0200
                        Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-08 19:50 +0200
                        Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-09 02:20 +0200
                          Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to  allow consolidation Peter Zijlstra <peterz@infradead.org> - 2015-10-09 10:50 +0200
    [PATCH tip/core/rcu 05/18] rcu: Move synchronize_sched_expedited() to combining tree "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 16/18] rcu: Add tasks to expedited stall-warning messages "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 12/18] cpu: Remove try_get_online_cpus() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 03/18] rcu: Consolidate tree setup for synchronize_rcu_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 13/18] rcu: Prepare for consolidating expedited CPU selection "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 07/18] rcu: Invert passed_quiesce and rename to cpu_no_qs "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm for RCU expedited grace period "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period Peter Zijlstra <peterz@infradead.org> - 2015-10-07 15:30 +0200
        Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 20:20 +0200
      Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period Peter Zijlstra <peterz@infradead.org> - 2015-10-07 15:40 +0200
        Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 17:50 +0200
      Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period Peter Zijlstra <peterz@infradead.org> - 2015-10-07 15:50 +0200
        Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 18:20 +0200
          Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period Peter Zijlstra <peterz@infradead.org> - 2015-10-08 11:10 +0200
      Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period Peter Zijlstra <peterz@infradead.org> - 2015-10-07 15:50 +0200
        Re: [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm  for RCU expedited grace period "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 18:20 +0200
    [PATCH tip/core/rcu 09/18] rcu: Switch synchronize_sched_expedited() to IPI "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      Re: [PATCH tip/core/rcu 09/18] rcu: Switch  synchronize_sched_expedited() to IPI Peter Zijlstra <peterz@infradead.org> - 2015-10-07 16:20 +0200
        Re: [PATCH tip/core/rcu 09/18] rcu: Switch  synchronize_sched_expedited() to IPI "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-07 18:30 +0200
    [PATCH tip/core/rcu 15/18] rcu: Add online/offline info to expedited stall warning message "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 14/18] rcu: Consolidate expedited CPU selection "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
    [PATCH tip/core/rcu 01/18] rcu: Use rsp->expedited_wq instead of sync_rcu_preempt_exp_wq "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      [PATCH tip/core/rcu 06/18] rcu: Rename qs_pending to core_needs_qs "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200
      [PATCH tip/core/rcu 17/18] rcu: Enable stall warnings for synchronize_rcu_expedited() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-10-06 18:40 +0200

Page 1 of 4 [1] 2 3 4 Next page →

#1240676 — [PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 18:30 +0200
Subject	[PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4
Message-ID	<qgDhN-2Zi-27@gated-at.bofh.it>

Hello!

This commit continues the effort to reduce the OS jitter from RCU's
expedited grace-period primitives, while also loosening the coupling
between CPU hotplug and RCU's expedited grace-period primitives:

1.	Use rsp->expedited_wq instead of sync_rcu_preempt_exp_wq to
	enable later code consolidation.

2.	Move rcu_report_exp_rnp() to allow later code consolidation.

3.	Consolidate combining-tree bitmaks setup for the initialization
	portion of synchronize_rcu_expedited().

4.	Use single-stage IPI algorithm for preemptible-RCU expedited
	grace periods.

5.	Make synchronize_sched_expedited() use the combining tree
	to reduce memory contention when waiting for quiescent states.

6.	Rename ->qs_pending to ->core_needs_qs to better match this
	field's use.

7.	Invert ->passed_quiesce and rename to ->cpu_no_qs in order to
	enable later aggregate-OR for requests for normal and expedited
	grace periods.

8.	Make ->cpu_no_qs be a union for aggregate OR.

9.	Switch synchronize_sched_expedited() from stop-CPUs to IPI.

10.	Stop silencing lockdep false positive for expedited grace periods,
	given that synchronize_rcu_expedited() no longer invokes
	synchronize_sched_expedited(), eliminating the apparent deadlock.
	(Just for the record, there never was a real deadlock.)

11.	Stop excluding CPU hotplug in synchronize_sched_expedited().

12.	Remove try_get_online_cpus(), which is now no longer used.

13.	Bring sync_sched_exp_select_cpus() into alignment with
	sync_rcu_exp_select_cpus() as a first step towards consolidating
	them into one function.

14.	Consolidate expedited CPU selection, now that #13 enabled it.

15.	Add online/offline information to expedited stall warning message.

16.	Dump blocking tasks in expedited stall-warning messages.

17.	Enable stall warnings for synchronize_rcu_expedited().

18.	Improve synchronize_sched_expedited() CPU-hotplug handling.

							Thanx, Paul

------------------------------------------------------------------------

 b/Documentation/RCU/trace.txt |   32 +-
 b/include/linux/cpu.h         |    2 
 b/include/linux/sched.h       |   10 
 b/kernel/cpu.c                |   13 
 b/kernel/rcu/tree.c           |  561 ++++++++++++++++++++++++++++++----------
 b/kernel/rcu/tree.h           |   50 ++-
 b/kernel/rcu/tree_plugin.h    |  579 ++++++++++++++++++++++--------------------
 b/kernel/rcu/tree_trace.c     |   10 
 8 files changed, 782 insertions(+), 475 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1240679 — [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 18:40 +0200
Subject	[PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qgDrr-3aC-1@gated-at.bofh.it>
In reply to	#1240676

Earlier versions of synchronize_sched_expedited() can prematurely end
grace periods due to the fact that a CPU marked as cpu_is_offline()
can still be using RCU read-side critical sections during the time that
CPU makes its last pass through the scheduler and into the idle loop
and during the time that a given CPU is in the process of coming online.
This commit therefore eliminates this window by adding additional
interaction with the CPU-hotplug operations.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 62 insertions(+), 6 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index ed957c3b6c86..80c834c46b8d 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -246,17 +246,23 @@ static int rcu_gp_in_progress(struct rcu_state *rsp)
  */
 void rcu_sched_qs(void)
 {
+	unsigned long flags;
+
 	if (__this_cpu_read(rcu_sched_data.cpu_no_qs.s)) {
 		trace_rcu_grace_period(TPS("rcu_sched"),
 				       __this_cpu_read(rcu_sched_data.gpnum),
 				       TPS("cpuqs"));
 		__this_cpu_write(rcu_sched_data.cpu_no_qs.b.norm, false);
+		if (!__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
+			return;
+		local_irq_save(flags);
 		if (__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp)) {
 			__this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, false);
 			rcu_report_exp_rdp(&rcu_sched_state,
 					   this_cpu_ptr(&rcu_sched_data),
 					   true);
 		}
+		local_irq_restore(flags);
 	}
 }
 
@@ -3553,7 +3559,10 @@ static void rcu_report_exp_cpu_mult(struct rcu_state *rsp, struct rcu_node *rnp,
 
 	raw_spin_lock_irqsave(&rnp->lock, flags);
 	smp_mb__after_unlock_lock();
-	WARN_ON_ONCE((rnp->expmask & mask) != mask);
+	if (!(rnp->expmask & mask)) {
+		raw_spin_unlock_irqrestore(&rnp->lock, flags);
+		return;
+	}
 	rnp->expmask &= ~mask;
 	__rcu_report_exp_rnp(rsp, rnp, wake, flags); /* Releases rnp->lock. */
 }
@@ -3644,12 +3653,37 @@ static struct rcu_node *exp_funnel_lock(struct rcu_state *rsp, unsigned long s)
 }
 
 /* Invoked on each online non-idle CPU for expedited quiescent state. */
-static void synchronize_sched_expedited_cpu_stop(void *data)
+static void sync_sched_exp_handler(void *data)
 {
+	struct rcu_data *rdp;
+	struct rcu_node *rnp;
+	struct rcu_state *rsp = data;
+
+	rdp = this_cpu_ptr(rsp->rda);
+	rnp = rdp->mynode;
+	if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) ||
+	    __this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
+		return;
 	__this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, true);
 	resched_cpu(smp_processor_id());
 }
 
+/* Send IPI for expedited cleanup if needed at end of CPU-hotplug operation. */
+static void sync_sched_exp_online_cleanup(int cpu)
+{
+	struct rcu_data *rdp;
+	int ret;
+	struct rcu_node *rnp;
+	struct rcu_state *rsp = &rcu_sched_state;
+
+	rdp = per_cpu_ptr(rsp->rda, cpu);
+	rnp = rdp->mynode;
+	if (!(READ_ONCE(rnp->expmask) & rdp->grpmask))
+		return;
+	ret = smp_call_function_single(cpu, sync_sched_exp_handler, rsp, 0);
+	WARN_ON_ONCE(ret);
+}
+
 /*
  * Select the nodes that the upcoming expedited grace period needs
  * to wait for.
@@ -3677,7 +3711,6 @@ static void sync_rcu_exp_select_cpus(struct rcu_state *rsp,
 			struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
 
 			if (raw_smp_processor_id() == cpu ||
-			    cpu_is_offline(cpu) ||
 			    !(atomic_add_return(0, &rdtp->dynticks) & 0x1))
 				mask_ofl_test |= rdp->grpmask;
 		}
@@ -3697,9 +3730,28 @@ static void sync_rcu_exp_select_cpus(struct rcu_state *rsp,
 		for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask <<= 1) {
 			if (!(mask_ofl_ipi & mask))
 				continue;
+retry_ipi:
 			ret = smp_call_function_single(cpu, func, rsp, 0);
-			if (!ret)
+			if (!ret) {
 				mask_ofl_ipi &= ~mask;
+			} else {
+				/* Failed, raced with offline. */
+				raw_spin_lock_irqsave(&rnp->lock, flags);
+				if (cpu_online(cpu) &&
+				    (rnp->expmask & mask)) {
+					raw_spin_unlock_irqrestore(&rnp->lock,
+								   flags);
+					schedule_timeout_uninterruptible(1);
+					if (cpu_online(cpu) &&
+					    (rnp->expmask & mask))
+						goto retry_ipi;
+					raw_spin_lock_irqsave(&rnp->lock,
+							      flags);
+				}
+				if (!(rnp->expmask & mask))
+					mask_ofl_ipi &= ~mask;
+				raw_spin_unlock_irqrestore(&rnp->lock, flags);
+			}
 		}
 		/* Report quiescent states for those that went offline. */
 		mask_ofl_test |= mask_ofl_ipi;
@@ -3796,7 +3848,7 @@ void synchronize_sched_expedited(void)
 		return;  /* Someone else did our work for us. */
 
 	rcu_exp_gp_seq_start(rsp);
-	sync_rcu_exp_select_cpus(rsp, synchronize_sched_expedited_cpu_stop);
+	sync_rcu_exp_select_cpus(rsp, sync_sched_exp_handler);
 	synchronize_sched_expedited_wait(rsp);
 
 	rcu_exp_gp_seq_end(rsp);
@@ -4183,6 +4235,7 @@ int rcu_cpu_notify(struct notifier_block *self,
 		break;
 	case CPU_ONLINE:
 	case CPU_DOWN_FAILED:
+		sync_sched_exp_online_cleanup(cpu);
 		rcu_boost_kthread_setaffinity(rnp, -1);
 		break;
 	case CPU_DOWN_PREPARE:
@@ -4195,7 +4248,10 @@ int rcu_cpu_notify(struct notifier_block *self,
 		break;
 	case CPU_DYING_IDLE:
 		/* QS for any half-done expedited RCU-sched GP. */
-		rcu_sched_qs();
+		preempt_disable();
+		rcu_report_exp_rdp(&rcu_sched_state,
+				   this_cpu_ptr(rcu_sched_state.rda), true);
+		preempt_enable();
 
 		for_each_rcu_flavor(rsp) {
 			rcu_cleanup_dying_idle_cpu(cpu, rsp);
-- 
2.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1241554 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-07 16:30 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qgXTc-7qA-31@gated-at.bofh.it>
In reply to	#1240679

On Tue, Oct 06, 2015 at 09:29:37AM -0700, Paul E. McKenney wrote:
>  void rcu_sched_qs(void)
>  {
> +	unsigned long flags;
> +
>  	if (__this_cpu_read(rcu_sched_data.cpu_no_qs.s)) {
>  		trace_rcu_grace_period(TPS("rcu_sched"),
>  				       __this_cpu_read(rcu_sched_data.gpnum),
>  				       TPS("cpuqs"));
>  		__this_cpu_write(rcu_sched_data.cpu_no_qs.b.norm, false);
> +		if (!__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
> +			return;
> +		local_irq_save(flags);
>  		if (__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp)) {
>  			__this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, false);
>  			rcu_report_exp_rdp(&rcu_sched_state,
>  					   this_cpu_ptr(&rcu_sched_data),
>  					   true);
>  		}
> +		local_irq_restore(flags);
>  	}
>  }

*sigh*.. still rare I suppose, but should we look at doing something
like this?

---
 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index fe819298c220..3d830c3491c4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3050,7 +3050,6 @@ static void __sched __schedule(void)
 
 	cpu = smp_processor_id();
 	rq = cpu_rq(cpu);
-	rcu_note_context_switch();
 	prev = rq->curr;
 
 	schedule_debug(prev);
@@ -3058,13 +3057,16 @@ static void __sched __schedule(void)
 	if (sched_feat(HRTICK))
 		hrtick_clear(rq);
 
+	local_irq_disable();
+	rcu_note_context_switch();
+
 	/*
 	 * Make sure that signal_pending_state()->signal_pending() below
 	 * can't be reordered with __set_current_state(TASK_INTERRUPTIBLE)
 	 * done by the caller to avoid the race with signal_wake_up().
 	 */
 	smp_mb__before_spinlock();
-	raw_spin_lock_irq(&rq->lock);
+	raw_spin_lock(&rq->lock);
 	lockdep_pin_lock(&rq->lock);
 
 	rq->clock_skip_update <<= 1; /* promote REQ to ACT */
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1241660 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-07 18:30 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qgZLk-1IJ-11@gated-at.bofh.it>
In reply to	#1241554

On Wed, Oct 07, 2015 at 04:26:27PM +0200, Peter Zijlstra wrote:
> On Tue, Oct 06, 2015 at 09:29:37AM -0700, Paul E. McKenney wrote:
> >  void rcu_sched_qs(void)
> >  {
> > +	unsigned long flags;
> > +
> >  	if (__this_cpu_read(rcu_sched_data.cpu_no_qs.s)) {
> >  		trace_rcu_grace_period(TPS("rcu_sched"),
> >  				       __this_cpu_read(rcu_sched_data.gpnum),
> >  				       TPS("cpuqs"));
> >  		__this_cpu_write(rcu_sched_data.cpu_no_qs.b.norm, false);
> > +		if (!__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
> > +			return;
> > +		local_irq_save(flags);
> >  		if (__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp)) {
> >  			__this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, false);
> >  			rcu_report_exp_rdp(&rcu_sched_state,
> >  					   this_cpu_ptr(&rcu_sched_data),
> >  					   true);
> >  		}
> > +		local_irq_restore(flags);
> >  	}
> >  }
> 
> *sigh*.. still rare I suppose, but should we look at doing something
> like this?

Indeed, that approach looks better than moving rcu_note_context_switch(),
which probably results in deadlocks.  I will update my patch accordingly.

							Thanx, Paul

> ---
>  kernel/sched/core.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index fe819298c220..3d830c3491c4 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3050,7 +3050,6 @@ static void __sched __schedule(void)
> 
>  	cpu = smp_processor_id();
>  	rq = cpu_rq(cpu);
> -	rcu_note_context_switch();
>  	prev = rq->curr;
> 
>  	schedule_debug(prev);
> @@ -3058,13 +3057,16 @@ static void __sched __schedule(void)
>  	if (sched_feat(HRTICK))
>  		hrtick_clear(rq);
> 
> +	local_irq_disable();
> +	rcu_note_context_switch();
> +
>  	/*
>  	 * Make sure that signal_pending_state()->signal_pending() below
>  	 * can't be reordered with __set_current_state(TASK_INTERRUPTIBLE)
>  	 * done by the caller to avoid the race with signal_wake_up().
>  	 */
>  	smp_mb__before_spinlock();
> -	raw_spin_lock_irq(&rq->lock);
> +	raw_spin_lock(&rq->lock);
>  	lockdep_pin_lock(&rq->lock);
> 
>  	rq->clock_skip_update <<= 1; /* promote REQ to ACT */
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242100 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-08 11:10 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhfn4-7oy-15@gated-at.bofh.it>
In reply to	#1241660

On Wed, Oct 07, 2015 at 09:26:53AM -0700, Paul E. McKenney wrote:
> On Wed, Oct 07, 2015 at 04:26:27PM +0200, Peter Zijlstra wrote:

> Indeed, that approach looks better than moving rcu_note_context_switch(),
> which probably results in deadlocks.  I will update my patch accordingly.

Yeah, calling rcu_note_context_switch() under the rq->lock is asking for
trouble we don't need.

Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242498 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-08 17:10 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhkZs-76Q-15@gated-at.bofh.it>
In reply to	#1242100

On Thu, Oct 08, 2015 at 11:01:28AM +0200, Peter Zijlstra wrote:
> On Wed, Oct 07, 2015 at 09:26:53AM -0700, Paul E. McKenney wrote:
> > On Wed, Oct 07, 2015 at 04:26:27PM +0200, Peter Zijlstra wrote:
> 
> > Indeed, that approach looks better than moving rcu_note_context_switch(),
> > which probably results in deadlocks.  I will update my patch accordingly.
> 
> Yeah, calling rcu_note_context_switch() under the rq->lock is asking for
> trouble we don't need.

Please see below for the fixed version.  Thoughts?

(Queued for 4.5, want some serious testing on this.)

							Thanx, Paul

------------------------------------------------------------------------

commit 3ab3edf72a59a800e6e59ad3128f5b3b251b8962
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Date:   Wed Oct 7 09:10:48 2015 -0700

    rcu: Stop disabling interrupts in scheduler fastpaths
    
    We need the scheduler's fastpaths to be, well, fast, and unnecessarily
    disabling and re-enabling interrupts is not necessarily consistent with
    this goal.  Especially given that there are regions of the scheduler that
    already have interrupts disabled.
    
    This commit therefore moves the call to rcu_note_context_switch()
    to one of the interrupts-disabled regions of the scheduler, and
    removes the now-redundant disabling and re-enabling of interrupts from
    rcu_note_context_switch() and the functions it calls.
    
    Reported-by: Peter Zijlstra <peterz@infradead.org>
    Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    [ paulmck: Shift rcu_note_context_switch() to avoid deadlock, as suggested
      by Peter Zijlstra. ]

diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 60d15a080d7c..9d3eda39bcd2 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -37,7 +37,7 @@ void rcu_cpu_stall_reset(void);
 /*
  * Note a virtualization-based context switch.  This is simply a
  * wrapper around rcu_note_context_switch(), which allows TINY_RCU
- * to save a few bytes.
+ * to save a few bytes. The caller must have disabled interrupts.
  */
 static inline void rcu_virt_note_context_switch(int cpu)
 {
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index c9780fc47391..fbc9b5574e48 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -295,17 +295,16 @@ EXPORT_PER_CPU_SYMBOL_GPL(rcu_qs_ctr);
  * We inform the RCU core by emulating a zero-duration dyntick-idle
  * period, which we in turn do by incrementing the ->dynticks counter
  * by two.
+ *
+ * The caller must have disabled interrupts.
  */
 static void rcu_momentary_dyntick_idle(void)
 {
-	unsigned long flags;
 	struct rcu_data *rdp;
 	struct rcu_dynticks *rdtp;
 	int resched_mask;
 	struct rcu_state *rsp;
 
-	local_irq_save(flags);
-
 	/*
 	 * Yes, we can lose flag-setting operations.  This is OK, because
 	 * the flag will be set again after some delay.
@@ -335,13 +334,12 @@ static void rcu_momentary_dyntick_idle(void)
 		smp_mb__after_atomic(); /* Later stuff after QS. */
 		break;
 	}
-	local_irq_restore(flags);
 }
 
 /*
  * Note a context switch.  This is a quiescent state for RCU-sched,
  * and requires special handling for preemptible RCU.
- * The caller must have disabled preemption.
+ * The caller must have disabled interrupts.
  */
 void rcu_note_context_switch(void)
 {
@@ -371,9 +369,14 @@ EXPORT_SYMBOL_GPL(rcu_note_context_switch);
  */
 void rcu_all_qs(void)
 {
+	unsigned long flags;
+
 	barrier(); /* Avoid RCU read-side critical sections leaking down. */
-	if (unlikely(raw_cpu_read(rcu_sched_qs_mask)))
+	if (unlikely(raw_cpu_read(rcu_sched_qs_mask))) {
+		local_irq_save(flags);
 		rcu_momentary_dyntick_idle();
+		local_irq_restore(flags);
+	}
 	this_cpu_inc(rcu_qs_ctr);
 	barrier(); /* Avoid RCU read-side critical sections leaking up. */
 }
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 97dfa7d57f79..7087fb047e2d 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -146,8 +146,8 @@ static void __init rcu_bootup_announce(void)
  * the corresponding expedited grace period will also be the end of the
  * normal grace period.
  */
-static void rcu_preempt_ctxt_queue(struct rcu_node *rnp, struct rcu_data *rdp,
-				   unsigned long flags) __releases(rnp->lock)
+static void rcu_preempt_ctxt_queue(struct rcu_node *rnp, struct rcu_data *rdp)
+	__releases(rnp->lock) /* But leaves rrupts disabled. */
 {
 	int blkd_state = (rnp->gp_tasks ? RCU_GP_TASKS : 0) +
 			 (rnp->exp_tasks ? RCU_EXP_TASKS : 0) +
@@ -235,7 +235,7 @@ static void rcu_preempt_ctxt_queue(struct rcu_node *rnp, struct rcu_data *rdp,
 		rnp->gp_tasks = &t->rcu_node_entry;
 	if (!rnp->exp_tasks && (blkd_state & RCU_EXP_BLKD))
 		rnp->exp_tasks = &t->rcu_node_entry;
-	raw_spin_unlock(&rnp->lock);
+	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
 
 	/*
 	 * Report the quiescent state for the expedited GP.  This expedited
@@ -250,7 +250,6 @@ static void rcu_preempt_ctxt_queue(struct rcu_node *rnp, struct rcu_data *rdp,
 	} else {
 		WARN_ON_ONCE(t->rcu_read_unlock_special.b.exp_need_qs);
 	}
-	local_irq_restore(flags);
 }
 
 /*
@@ -285,12 +284,11 @@ static void rcu_preempt_qs(void)
  * predating the current grace period drain, in other words, until
  * rnp->gp_tasks becomes NULL.
  *
- * Caller must disable preemption.
+ * Caller must disable interrupts.
  */
 static void rcu_preempt_note_context_switch(void)
 {
 	struct task_struct *t = current;
-	unsigned long flags;
 	struct rcu_data *rdp;
 	struct rcu_node *rnp;
 
@@ -300,7 +298,7 @@ static void rcu_preempt_note_context_switch(void)
 		/* Possibly blocking in an RCU read-side critical section. */
 		rdp = this_cpu_ptr(rcu_state_p->rda);
 		rnp = rdp->mynode;
-		raw_spin_lock_irqsave(&rnp->lock, flags);
+		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
 		smp_mb__after_unlock_lock();
 		t->rcu_read_unlock_special.b.blocked = true;
 		t->rcu_blocked_node = rnp;
@@ -317,7 +315,7 @@ static void rcu_preempt_note_context_switch(void)
 				       (rnp->qsmask & rdp->grpmask)
 				       ? rnp->gpnum
 				       : rnp->gpnum + 1);
-		rcu_preempt_ctxt_queue(rnp, rdp, flags);
+		rcu_preempt_ctxt_queue(rnp, rdp);
 	} else if (t->rcu_read_lock_nesting < 0 &&
 		   t->rcu_read_unlock_special.s) {
 
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index c4e607873d6f..ac246b0b987a 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3056,7 +3056,6 @@ static void __sched __schedule(void)
 
 	cpu = smp_processor_id();
 	rq = cpu_rq(cpu);
-	rcu_note_context_switch();
 	prev = rq->curr;
 
 	schedule_debug(prev);
@@ -3064,13 +3063,16 @@ static void __sched __schedule(void)
 	if (sched_feat(HRTICK))
 		hrtick_clear(rq);
 
+	local_irq_disable();
+	rcu_note_context_switch();
+
 	/*
 	 * Make sure that signal_pending_state()->signal_pending() below
 	 * can't be reordered with __set_current_state(TASK_INTERRUPTIBLE)
 	 * done by the caller to avoid the race with signal_wake_up().
 	 */
 	smp_mb__before_spinlock();
-	raw_spin_lock_irq(&rq->lock);
+	raw_spin_lock(&rq->lock);
 	lockdep_pin_lock(&rq->lock);
 
 	rq->clock_skip_update <<= 1; /* promote REQ to ACT */

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242515 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-08 17:20 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhl98-7i7-19@gated-at.bofh.it>
In reply to	#1242498

On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> Please see below for the fixed version.  Thoughts?

> +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */

What them 'rrupts' about? ;-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242516 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-08 17:20 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhl98-7i7-21@gated-at.bofh.it>
In reply to	#1242515

On Thu, Oct 08, 2015 at 05:12:42PM +0200, Peter Zijlstra wrote:
> On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> > Please see below for the fixed version.  Thoughts?
> 
> > +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> > +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> > +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
> 
> What them 'rrupts' about? ;-)

Interrupts when it won't fit.  I suppose I could use IRQs instead.  ;-)

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242663 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	Josh Triplett <josh@joshtriplett.org>
Date	2015-10-08 20:10 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhnNE-2Gn-13@gated-at.bofh.it>
In reply to	#1242516

On Thu, Oct 08, 2015 at 08:19:03AM -0700, Paul E. McKenney wrote:
> On Thu, Oct 08, 2015 at 05:12:42PM +0200, Peter Zijlstra wrote:
> > On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> > > Please see below for the fixed version.  Thoughts?
> > 
> > > +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> > > +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> > > +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
> > 
> > What them 'rrupts' about? ;-)
> 
> Interrupts when it won't fit.  I suppose I could use IRQs instead.  ;-)

In this particular case, "IRQs" works just as well; however, in general,
this seems like an excellent example of when to ignore the 80-column
guideline. :)

- Josh Triplett
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242848 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-09 02:20 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhtzH-2tg-13@gated-at.bofh.it>
In reply to	#1242663

On Thu, Oct 08, 2015 at 11:01:14AM -0700, Josh Triplett wrote:
> On Thu, Oct 08, 2015 at 08:19:03AM -0700, Paul E. McKenney wrote:
> > On Thu, Oct 08, 2015 at 05:12:42PM +0200, Peter Zijlstra wrote:
> > > On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> > > > Please see below for the fixed version.  Thoughts?
> > > 
> > > > +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> > > > +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> > > > +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
> > > 
> > > What them 'rrupts' about? ;-)
> > 
> > Interrupts when it won't fit.  I suppose I could use IRQs instead.  ;-)
> 
> In this particular case, "IRQs" works just as well; however, in general,
> this seems like an excellent example of when to ignore the 80-column
> guideline. :)

But but but...   You are talking to someone who used actual PUNCHED CARDS
in real life in a paying job!!!  ;-)

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1242949 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	Josh Triplett <josh@joshtriplett.org>
Date	2015-10-09 02:50 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhu2M-31O-53@gated-at.bofh.it>
In reply to	#1242848

On Thu, Oct 08, 2015 at 05:11:11PM -0700, Paul E. McKenney wrote:
> On Thu, Oct 08, 2015 at 11:01:14AM -0700, Josh Triplett wrote:
> > On Thu, Oct 08, 2015 at 08:19:03AM -0700, Paul E. McKenney wrote:
> > > On Thu, Oct 08, 2015 at 05:12:42PM +0200, Peter Zijlstra wrote:
> > > > On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> > > > > Please see below for the fixed version.  Thoughts?
> > > > 
> > > > > +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> > > > > +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> > > > > +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
> > > > 
> > > > What them 'rrupts' about? ;-)
> > > 
> > > Interrupts when it won't fit.  I suppose I could use IRQs instead.  ;-)
> > 
> > In this particular case, "IRQs" works just as well; however, in general,
> > this seems like an excellent example of when to ignore the 80-column
> > guideline. :)
> 
> But but but...   You are talking to someone who used actual PUNCHED CARDS
> in real life in a paying job!!!  ;-)

And I learned on a DOS system with 80x25 text mode.  Let us revel in
wonderment at the capabilities of modern systems. :)

- Josh Triplett
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1243022 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-09 06:00 +0200
Subject	Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()
Message-ID	<qhx0D-7jS-9@gated-at.bofh.it>
In reply to	#1242949

On Thu, Oct 08, 2015 at 05:48:13PM -0700, Josh Triplett wrote:
> On Thu, Oct 08, 2015 at 05:11:11PM -0700, Paul E. McKenney wrote:
> > On Thu, Oct 08, 2015 at 11:01:14AM -0700, Josh Triplett wrote:
> > > On Thu, Oct 08, 2015 at 08:19:03AM -0700, Paul E. McKenney wrote:
> > > > On Thu, Oct 08, 2015 at 05:12:42PM +0200, Peter Zijlstra wrote:
> > > > > On Thu, Oct 08, 2015 at 08:06:39AM -0700, Paul E. McKenney wrote:
> > > > > > Please see below for the fixed version.  Thoughts?
> > > > > 
> > > > > > +	__releases(rnp->lock) /* But leaves rrupts disabled. */
> > > > > > +	raw_spin_unlock(&rnp->lock); /* rrupts remain disabled. */
> > > > > > +		raw_spin_lock(&rnp->lock); /* rrupts already disabled. */
> > > > > 
> > > > > What them 'rrupts' about? ;-)
> > > > 
> > > > Interrupts when it won't fit.  I suppose I could use IRQs instead.  ;-)
> > > 
> > > In this particular case, "IRQs" works just as well; however, in general,
> > > this seems like an excellent example of when to ignore the 80-column
> > > guideline. :)
> > 
> > But but but...   You are talking to someone who used actual PUNCHED CARDS
> > in real life in a paying job!!!  ;-)
> 
> And I learned on a DOS system with 80x25 text mode.  Let us revel in
> wonderment at the capabilities of modern systems. :)

But of course!  On my new 2880x1620 screen, I can put four 80x24 xterms
on each row!

Too bad that I cannot actually read them without an external monitor,
and my current external monitors are only 1920x1600.  ;-)

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240680 — [PATCH tip/core/rcu 10/18] rcu: Stop silencing lockdep false positive for expedited grace periods

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 18:40 +0200
Subject	[PATCH tip/core/rcu 10/18] rcu: Stop silencing lockdep false positive for expedited grace periods
Message-ID	<qgDrs-3aC-5@gated-at.bofh.it>
In reply to	#1240676

This reverts commit af859beaaba4 (rcu: Silence lockdep false positive
for expedited grace periods).  Because synchronize_rcu_expedited()
no longer invokes synchronize_sched_expedited(), ->exp_funnel_mutex
acquisition is no longer nested, so the false positive no longer happens.
This commit therefore removes the extra lockdep data structures, as they
are no longer needed.
---
 kernel/rcu/tree.c | 17 ++---------------
 kernel/rcu/tree.h |  8 --------
 2 files changed, 2 insertions(+), 23 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 869e58b92c53..57b83f6d5263 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -71,7 +71,6 @@ MODULE_ALIAS("rcutree");
 static struct lock_class_key rcu_node_class[RCU_NUM_LVLS];
 static struct lock_class_key rcu_fqs_class[RCU_NUM_LVLS];
 static struct lock_class_key rcu_exp_class[RCU_NUM_LVLS];
-static struct lock_class_key rcu_exp_sched_class[RCU_NUM_LVLS];
 
 /*
  * In order to export the rcu_state name to the tracing tools, it
@@ -4095,7 +4094,6 @@ static void rcu_init_new_rnp(struct rcu_node *rnp_leaf)
 static void __init
 rcu_boot_init_percpu_data(int cpu, struct rcu_state *rsp)
 {
-	static struct lock_class_key rcu_exp_sched_rdp_class;
 	unsigned long flags;
 	struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
 	struct rcu_node *rnp = rcu_get_root(rsp);
@@ -4111,10 +4109,6 @@ rcu_boot_init_percpu_data(int cpu, struct rcu_state *rsp)
 	mutex_init(&rdp->exp_funnel_mutex);
 	rcu_boot_init_nocb_percpu_data(rdp);
 	raw_spin_unlock_irqrestore(&rnp->lock, flags);
-	if (rsp == &rcu_sched_state)
-		lockdep_set_class_and_name(&rdp->exp_funnel_mutex,
-					   &rcu_exp_sched_rdp_class,
-					   "rcu_data_exp_sched");
 }
 
 /*
@@ -4340,7 +4334,6 @@ static void __init rcu_init_one(struct rcu_state *rsp,
 	static const char * const buf[] = RCU_NODE_NAME_INIT;
 	static const char * const fqs[] = RCU_FQS_NAME_INIT;
 	static const char * const exp[] = RCU_EXP_NAME_INIT;
-	static const char * const exp_sched[] = RCU_EXP_SCHED_NAME_INIT;
 	static u8 fl_mask = 0x1;
 
 	int levelcnt[RCU_NUM_LVLS];		/* # nodes in each level. */
@@ -4400,14 +4393,8 @@ static void __init rcu_init_one(struct rcu_state *rsp,
 			INIT_LIST_HEAD(&rnp->blkd_tasks);
 			rcu_init_one_nocb(rnp);
 			mutex_init(&rnp->exp_funnel_mutex);
-			if (rsp == &rcu_sched_state)
-				lockdep_set_class_and_name(
-					&rnp->exp_funnel_mutex,
-					&rcu_exp_sched_class[i], exp_sched[i]);
-			else
-				lockdep_set_class_and_name(
-					&rnp->exp_funnel_mutex,
-					&rcu_exp_class[i], exp[i]);
+			lockdep_set_class_and_name(&rnp->exp_funnel_mutex,
+						   &rcu_exp_class[i], exp[i]);
 		}
 	}
 
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index 1b969cef8fe4..6f3b63b68886 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -70,8 +70,6 @@
 #  define RCU_NODE_NAME_INIT  { "rcu_node_0" }
 #  define RCU_FQS_NAME_INIT   { "rcu_node_fqs_0" }
 #  define RCU_EXP_NAME_INIT   { "rcu_node_exp_0" }
-#  define RCU_EXP_SCHED_NAME_INIT \
-			      { "rcu_node_exp_sched_0" }
 #elif NR_CPUS <= RCU_FANOUT_2
 #  define RCU_NUM_LVLS	      2
 #  define NUM_RCU_LVL_0	      1
@@ -81,8 +79,6 @@
 #  define RCU_NODE_NAME_INIT  { "rcu_node_0", "rcu_node_1" }
 #  define RCU_FQS_NAME_INIT   { "rcu_node_fqs_0", "rcu_node_fqs_1" }
 #  define RCU_EXP_NAME_INIT   { "rcu_node_exp_0", "rcu_node_exp_1" }
-#  define RCU_EXP_SCHED_NAME_INIT \
-			      { "rcu_node_exp_sched_0", "rcu_node_exp_sched_1" }
 #elif NR_CPUS <= RCU_FANOUT_3
 #  define RCU_NUM_LVLS	      3
 #  define NUM_RCU_LVL_0	      1
@@ -93,8 +89,6 @@
 #  define RCU_NODE_NAME_INIT  { "rcu_node_0", "rcu_node_1", "rcu_node_2" }
 #  define RCU_FQS_NAME_INIT   { "rcu_node_fqs_0", "rcu_node_fqs_1", "rcu_node_fqs_2" }
 #  define RCU_EXP_NAME_INIT   { "rcu_node_exp_0", "rcu_node_exp_1", "rcu_node_exp_2" }
-#  define RCU_EXP_SCHED_NAME_INIT \
-			      { "rcu_node_exp_sched_0", "rcu_node_exp_sched_1", "rcu_node_exp_sched_2" }
 #elif NR_CPUS <= RCU_FANOUT_4
 #  define RCU_NUM_LVLS	      4
 #  define NUM_RCU_LVL_0	      1
@@ -106,8 +100,6 @@
 #  define RCU_NODE_NAME_INIT  { "rcu_node_0", "rcu_node_1", "rcu_node_2", "rcu_node_3" }
 #  define RCU_FQS_NAME_INIT   { "rcu_node_fqs_0", "rcu_node_fqs_1", "rcu_node_fqs_2", "rcu_node_fqs_3" }
 #  define RCU_EXP_NAME_INIT   { "rcu_node_exp_0", "rcu_node_exp_1", "rcu_node_exp_2", "rcu_node_exp_3" }
-#  define RCU_EXP_SCHED_NAME_INIT \
-			      { "rcu_node_exp_sched_0", "rcu_node_exp_sched_1", "rcu_node_exp_sched_2", "rcu_node_exp_sched_3" }
 #else
 # error "CONFIG_RCU_FANOUT insufficient for NR_CPUS"
 #endif /* #if (NR_CPUS) <= RCU_FANOUT_1 */
-- 
2.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240681 — [PATCH tip/core/rcu 08/18] rcu: Make ->cpu_no_qs be a union for aggregate OR

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 18:40 +0200
Subject	[PATCH tip/core/rcu 08/18] rcu: Make ->cpu_no_qs be a union for aggregate OR
Message-ID	<qgDrs-3aC-9@gated-at.bofh.it>
In reply to	#1240676

This commit converts the rcu_data structure's ->cpu_no_qs field
to a union.  The bytewise side of this union allows individual access
to indications as to whether this CPU needs to find a quiescent state
for a normal (.norm) and/or expedited (.exp) grace period.  The setwise
side of the union allows testing whether or not a quiescent state is
needed at all, for either type of grace period.

For now, only .norm is used.  A later commit will introduce the expedited
usage.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree.c        | 22 +++++++++++-----------
 kernel/rcu/tree.h        | 14 +++++++++++++-
 kernel/rcu/tree_plugin.h |  6 +++---
 kernel/rcu/tree_trace.c  |  2 +-
 4 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 31e7021ced4d..3e2875b38eae 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -245,21 +245,21 @@ static int rcu_gp_in_progress(struct rcu_state *rsp)
  */
 void rcu_sched_qs(void)
 {
-	if (__this_cpu_read(rcu_sched_data.cpu_no_qs)) {
+	if (__this_cpu_read(rcu_sched_data.cpu_no_qs.s)) {
 		trace_rcu_grace_period(TPS("rcu_sched"),
 				       __this_cpu_read(rcu_sched_data.gpnum),
 				       TPS("cpuqs"));
-		__this_cpu_write(rcu_sched_data.cpu_no_qs, false);
+		__this_cpu_write(rcu_sched_data.cpu_no_qs.b.norm, false);
 	}
 }
 
 void rcu_bh_qs(void)
 {
-	if (__this_cpu_read(rcu_bh_data.cpu_no_qs)) {
+	if (__this_cpu_read(rcu_bh_data.cpu_no_qs.s)) {
 		trace_rcu_grace_period(TPS("rcu_bh"),
 				       __this_cpu_read(rcu_bh_data.gpnum),
 				       TPS("cpuqs"));
-		__this_cpu_write(rcu_bh_data.cpu_no_qs, false);
+		__this_cpu_write(rcu_bh_data.cpu_no_qs.b.norm, false);
 	}
 }
 
@@ -1744,7 +1744,7 @@ static bool __note_gp_changes(struct rcu_state *rsp, struct rcu_node *rnp,
 		 */
 		rdp->gpnum = rnp->gpnum;
 		trace_rcu_grace_period(rsp->name, rdp->gpnum, TPS("cpustart"));
-		rdp->cpu_no_qs = true;
+		rdp->cpu_no_qs.b.norm = true;
 		rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_qs_ctr);
 		rdp->core_needs_qs = !!(rnp->qsmask & rdp->grpmask);
 		zero_cpu_stall_ticks(rdp);
@@ -2337,7 +2337,7 @@ rcu_report_qs_rdp(int cpu, struct rcu_state *rsp, struct rcu_data *rdp)
 	rnp = rdp->mynode;
 	raw_spin_lock_irqsave(&rnp->lock, flags);
 	smp_mb__after_unlock_lock();
-	if ((rdp->cpu_no_qs &&
+	if ((rdp->cpu_no_qs.b.norm &&
 	     rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr)) ||
 	    rdp->gpnum != rnp->gpnum || rnp->completed == rnp->gpnum ||
 	    rdp->gpwrap) {
@@ -2348,7 +2348,7 @@ rcu_report_qs_rdp(int cpu, struct rcu_state *rsp, struct rcu_data *rdp)
 		 * We will instead need a new quiescent state that lies
 		 * within the current grace period.
 		 */
-		rdp->cpu_no_qs = true;	/* need qs for new gp. */
+		rdp->cpu_no_qs.b.norm = true;	/* need qs for new gp. */
 		rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_qs_ctr);
 		raw_spin_unlock_irqrestore(&rnp->lock, flags);
 		return;
@@ -2395,7 +2395,7 @@ rcu_check_quiescent_state(struct rcu_state *rsp, struct rcu_data *rdp)
 	 * Was there a quiescent state since the beginning of the grace
 	 * period? If no, then exit and wait for the next call.
 	 */
-	if (rdp->cpu_no_qs &&
+	if (rdp->cpu_no_qs.b.norm &&
 	    rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr))
 		return;
 
@@ -3828,11 +3828,11 @@ static int __rcu_pending(struct rcu_state *rsp, struct rcu_data *rdp)
 
 	/* Is the RCU core waiting for a quiescent state from this CPU? */
 	if (rcu_scheduler_fully_active &&
-	    rdp->core_needs_qs && rdp->cpu_no_qs &&
+	    rdp->core_needs_qs && rdp->cpu_no_qs.b.norm &&
 	    rdp->rcu_qs_ctr_snap == __this_cpu_read(rcu_qs_ctr)) {
 		rdp->n_rp_core_needs_qs++;
 	} else if (rdp->core_needs_qs &&
-		   (!rdp->cpu_no_qs ||
+		   (!rdp->cpu_no_qs.b.norm ||
 		    rdp->rcu_qs_ctr_snap != __this_cpu_read(rcu_qs_ctr))) {
 		rdp->n_rp_report_qs++;
 		return 1;
@@ -4155,7 +4155,7 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp)
 	rdp->beenonline = true;	 /* We have now been online. */
 	rdp->gpnum = rnp->completed; /* Make CPU later note any new GP. */
 	rdp->completed = rnp->completed;
-	rdp->cpu_no_qs = true;
+	rdp->cpu_no_qs.b.norm = true;
 	rdp->rcu_qs_ctr_snap = per_cpu(rcu_qs_ctr, cpu);
 	rdp->core_needs_qs = false;
 	trace_rcu_grace_period(rsp->name, rdp->gpnum, TPS("cpuonl"));
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index ded4ceebed76..3eee48bcf52b 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -286,6 +286,18 @@ struct rcu_node {
 	for ((rnp) = (rsp)->level[rcu_num_lvls - 1]; \
 	     (rnp) < &(rsp)->node[rcu_num_nodes]; (rnp)++)
 
+/*
+ * Union to allow "aggregate OR" operation on the need for a quiescent
+ * state by the normal and expedited grace periods.
+ */
+union rcu_noqs {
+	struct {
+		u8 norm;
+		u8 exp;
+	} b; /* Bits. */
+	u16 s; /* Set of bits, aggregate OR here. */
+};
+
 /* Index values for nxttail array in struct rcu_data. */
 #define RCU_DONE_TAIL		0	/* Also RCU_WAIT head. */
 #define RCU_WAIT_TAIL		1	/* Also RCU_NEXT_READY head. */
@@ -302,7 +314,7 @@ struct rcu_data {
 					/*  is aware of having started. */
 	unsigned long	rcu_qs_ctr_snap;/* Snapshot of rcu_qs_ctr to check */
 					/*  for rcu_all_qs() invocations. */
-	bool		cpu_no_qs;	/* No QS yet for this CPU. */
+	union rcu_noqs	cpu_no_qs;	/* No QSes yet for this CPU. */
 	bool		core_needs_qs;	/* Core waits for quiesc state. */
 	bool		beenonline;	/* CPU online at least once. */
 	bool		gpwrap;		/* Possible gpnum/completed wrap. */
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 6977ff0dccb9..7880202f1e38 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -265,11 +265,11 @@ static void rcu_preempt_ctxt_queue(struct rcu_node *rnp, struct rcu_data *rdp,
  */
 static void rcu_preempt_qs(void)
 {
-	if (__this_cpu_read(rcu_data_p->cpu_no_qs)) {
+	if (__this_cpu_read(rcu_data_p->cpu_no_qs.s)) {
 		trace_rcu_grace_period(TPS("rcu_preempt"),
 				       __this_cpu_read(rcu_data_p->gpnum),
 				       TPS("cpuqs"));
-		__this_cpu_write(rcu_data_p->cpu_no_qs, false);
+		__this_cpu_write(rcu_data_p->cpu_no_qs.b.norm, false);
 		barrier(); /* Coordinate with rcu_preempt_check_callbacks(). */
 		current->rcu_read_unlock_special.b.need_qs = false;
 	}
@@ -620,7 +620,7 @@ static void rcu_preempt_check_callbacks(void)
 	}
 	if (t->rcu_read_lock_nesting > 0 &&
 	    __this_cpu_read(rcu_data_p->core_needs_qs) &&
-	    __this_cpu_read(rcu_data_p->cpu_no_qs))
+	    __this_cpu_read(rcu_data_p->cpu_no_qs.b.norm))
 		t->rcu_read_unlock_special.b.need_qs = true;
 }
 
diff --git a/kernel/rcu/tree_trace.c b/kernel/rcu/tree_trace.c
index d373e57109b8..999c3672f990 100644
--- a/kernel/rcu/tree_trace.c
+++ b/kernel/rcu/tree_trace.c
@@ -121,7 +121,7 @@ static void print_one_rcu_data(struct seq_file *m, struct rcu_data *rdp)
 		   rdp->cpu,
 		   cpu_is_offline(rdp->cpu) ? '!' : ' ',
 		   ulong2long(rdp->completed), ulong2long(rdp->gpnum),
-		   rdp->cpu_no_qs,
+		   rdp->cpu_no_qs.b.norm,
 		   rdp->rcu_qs_ctr_snap == per_cpu(rcu_qs_ctr, rdp->cpu),
 		   rdp->core_needs_qs);
 	seq_printf(m, " dt=%d/%llx/%d df=%lu",
-- 
2.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240682 — [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 18:40 +0200
Subject	[PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgDrs-3aC-11@gated-at.bofh.it>
In reply to	#1240676

This is a nearly pure code-movement commit, moving rcu_report_exp_rnp(),
sync_rcu_preempt_exp_done(), and rcu_preempted_readers_exp() so
that later commits can make synchronize_sched_expedited() use them.
The non-code-movement portion of this commit tags rcu_report_exp_rnp()
as __maybe_unused to avoid build errors when CONFIG_PREEMPT=n.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree.c        | 66 ++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/rcu/tree_plugin.h | 66 ------------------------------------------------
 2 files changed, 66 insertions(+), 66 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 53d66ebb4811..59af27d8bc6a 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3379,6 +3379,72 @@ static bool rcu_exp_gp_seq_done(struct rcu_state *rsp, unsigned long s)
 	return rcu_seq_done(&rsp->expedited_sequence, s);
 }
 
+/*
+ * Return non-zero if there are any tasks in RCU read-side critical
+ * sections blocking the current preemptible-RCU expedited grace period.
+ * If there is no preemptible-RCU expedited grace period currently in
+ * progress, returns zero unconditionally.
+ */
+static int rcu_preempted_readers_exp(struct rcu_node *rnp)
+{
+	return rnp->exp_tasks != NULL;
+}
+
+/*
+ * return non-zero if there is no RCU expedited grace period in progress
+ * for the specified rcu_node structure, in other words, if all CPUs and
+ * tasks covered by the specified rcu_node structure have done their bit
+ * for the current expedited grace period.  Works only for preemptible
+ * RCU -- other RCU implementation use other means.
+ *
+ * Caller must hold the root rcu_node's exp_funnel_mutex.
+ */
+static int sync_rcu_preempt_exp_done(struct rcu_node *rnp)
+{
+	return !rcu_preempted_readers_exp(rnp) &&
+	       READ_ONCE(rnp->expmask) == 0;
+}
+
+/*
+ * Report the exit from RCU read-side critical section for the last task
+ * that queued itself during or before the current expedited preemptible-RCU
+ * grace period.  This event is reported either to the rcu_node structure on
+ * which the task was queued or to one of that rcu_node structure's ancestors,
+ * recursively up the tree.  (Calm down, calm down, we do the recursion
+ * iteratively!)
+ *
+ * Caller must hold the root rcu_node's exp_funnel_mutex.
+ */
+static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
+					      struct rcu_node *rnp, bool wake)
+{
+	unsigned long flags;
+	unsigned long mask;
+
+	raw_spin_lock_irqsave(&rnp->lock, flags);
+	smp_mb__after_unlock_lock();
+	for (;;) {
+		if (!sync_rcu_preempt_exp_done(rnp)) {
+			raw_spin_unlock_irqrestore(&rnp->lock, flags);
+			break;
+		}
+		if (rnp->parent == NULL) {
+			raw_spin_unlock_irqrestore(&rnp->lock, flags);
+			if (wake) {
+				smp_mb(); /* EGP done before wake_up(). */
+				wake_up(&rsp->expedited_wq);
+			}
+			break;
+		}
+		mask = rnp->grpmask;
+		raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
+		rnp = rnp->parent;
+		raw_spin_lock(&rnp->lock); /* irqs already disabled */
+		smp_mb__after_unlock_lock();
+		rnp->expmask &= ~mask;
+	}
+}
+
 /* Common code for synchronize_{rcu,sched}_expedited() work-done checking. */
 static bool sync_exp_work_done(struct rcu_state *rsp, struct rcu_node *rnp,
 			       struct rcu_data *rdp,
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 72df006de798..e73be8539978 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -536,72 +536,6 @@ void synchronize_rcu(void)
 EXPORT_SYMBOL_GPL(synchronize_rcu);
 
 /*
- * Return non-zero if there are any tasks in RCU read-side critical
- * sections blocking the current preemptible-RCU expedited grace period.
- * If there is no preemptible-RCU expedited grace period currently in
- * progress, returns zero unconditionally.
- */
-static int rcu_preempted_readers_exp(struct rcu_node *rnp)
-{
-	return rnp->exp_tasks != NULL;
-}
-
-/*
- * return non-zero if there is no RCU expedited grace period in progress
- * for the specified rcu_node structure, in other words, if all CPUs and
- * tasks covered by the specified rcu_node structure have done their bit
- * for the current expedited grace period.  Works only for preemptible
- * RCU -- other RCU implementation use other means.
- *
- * Caller must hold the root rcu_node's exp_funnel_mutex.
- */
-static int sync_rcu_preempt_exp_done(struct rcu_node *rnp)
-{
-	return !rcu_preempted_readers_exp(rnp) &&
-	       READ_ONCE(rnp->expmask) == 0;
-}
-
-/*
- * Report the exit from RCU read-side critical section for the last task
- * that queued itself during or before the current expedited preemptible-RCU
- * grace period.  This event is reported either to the rcu_node structure on
- * which the task was queued or to one of that rcu_node structure's ancestors,
- * recursively up the tree.  (Calm down, calm down, we do the recursion
- * iteratively!)
- *
- * Caller must hold the root rcu_node's exp_funnel_mutex.
- */
-static void rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
-			       bool wake)
-{
-	unsigned long flags;
-	unsigned long mask;
-
-	raw_spin_lock_irqsave(&rnp->lock, flags);
-	smp_mb__after_unlock_lock();
-	for (;;) {
-		if (!sync_rcu_preempt_exp_done(rnp)) {
-			raw_spin_unlock_irqrestore(&rnp->lock, flags);
-			break;
-		}
-		if (rnp->parent == NULL) {
-			raw_spin_unlock_irqrestore(&rnp->lock, flags);
-			if (wake) {
-				smp_mb(); /* EGP done before wake_up(). */
-				wake_up(&rsp->expedited_wq);
-			}
-			break;
-		}
-		mask = rnp->grpmask;
-		raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
-		rnp = rnp->parent;
-		raw_spin_lock(&rnp->lock); /* irqs already disabled */
-		smp_mb__after_unlock_lock();
-		rnp->expmask &= ~mask;
-	}
-}
-
-/*
  * Snapshot the tasks blocking the newly started preemptible-RCU expedited
  * grace period for the specified rcu_node structure, phase 1.  If there
  * are such tasks, set the ->expmask bits up the rcu_node tree and also
-- 
2.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240965 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-06 22:30 +0200
Subject	Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgH22-8u5-1@gated-at.bofh.it>
In reply to	#1240682

On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
> +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
> +					      struct rcu_node *rnp, bool wake)
> +{
> +	unsigned long flags;
> +	unsigned long mask;
> +
> +	raw_spin_lock_irqsave(&rnp->lock, flags);

Normally we require a comment with barriers, explaining the order and
the pairing etc.. :-)

> +	smp_mb__after_unlock_lock();
> +	for (;;) {
> +		if (!sync_rcu_preempt_exp_done(rnp)) {
> +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> +			break;
> +		}
> +		if (rnp->parent == NULL) {
> +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> +			if (wake) {
> +				smp_mb(); /* EGP done before wake_up(). */
> +				wake_up(&rsp->expedited_wq);
> +			}
> +			break;
> +		}
> +		mask = rnp->grpmask;
> +		raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
> +		rnp = rnp->parent;
> +		raw_spin_lock(&rnp->lock); /* irqs already disabled */
> +		smp_mb__after_unlock_lock();
> +		rnp->expmask &= ~mask;
> +	}
> +}
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240992 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Date	2015-10-06 23:00 +0200
Subject	Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgHv4-AR-21@gated-at.bofh.it>
In reply to	#1240965

On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote:
> On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
> > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
> > +					      struct rcu_node *rnp, bool wake)
> > +{
> > +	unsigned long flags;
> > +	unsigned long mask;
> > +
> > +	raw_spin_lock_irqsave(&rnp->lock, flags);
> 
> Normally we require a comment with barriers, explaining the order and
> the pairing etc.. :-)
> 
> > +	smp_mb__after_unlock_lock();

Hmmmm...  That is not good.

Worse yet, I am missing comments on most of the pre-existing barriers
of this form.

The purpose is to enforce the heavy-weight grace-period memory-ordering
guarantees documented in the synchronize_sched() header comment and
elsewhere.  They pair with anything you might use to check for violation
of these guarantees, or, simiarly, any ordering that you might use when
relying on these guarantees.

I could add something like  "/* Enforce GP memory ordering. */"

Or perhaps "/* See synchronize_sched() header. */"

I do not propose reproducing the synchronize_sched() header on each
of these.  That would be verbose, even for me!  ;-)

Other thoughts?

							Thanx, Paul

> > +	for (;;) {
> > +		if (!sync_rcu_preempt_exp_done(rnp)) {
> > +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> > +			break;
> > +		}
> > +		if (rnp->parent == NULL) {
> > +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> > +			if (wake) {
> > +				smp_mb(); /* EGP done before wake_up(). */
> > +				wake_up(&rsp->expedited_wq);
> > +			}
> > +			break;
> > +		}
> > +		mask = rnp->grpmask;
> > +		raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
> > +		rnp = rnp->parent;
> > +		raw_spin_lock(&rnp->lock); /* irqs already disabled */
> > +		smp_mb__after_unlock_lock();
> > +		rnp->expmask &= ~mask;
> > +	}
> > +}
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1241208 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-07 10:00 +0200
Subject	Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgRNM-6WR-7@gated-at.bofh.it>
In reply to	#1240992

On Tue, Oct 06, 2015 at 01:58:50PM -0700, Paul E. McKenney wrote:
> On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote:
> > On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
> > > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
> > > +					      struct rcu_node *rnp, bool wake)
> > > +{
> > > +	unsigned long flags;
> > > +	unsigned long mask;
> > > +
> > > +	raw_spin_lock_irqsave(&rnp->lock, flags);
> > 
> > Normally we require a comment with barriers, explaining the order and
> > the pairing etc.. :-)
> > 
> > > +	smp_mb__after_unlock_lock();
> 
> Hmmmm...  That is not good.
> 
> Worse yet, I am missing comments on most of the pre-existing barriers
> of this form.

Yes I noticed.. :/

> The purpose is to enforce the heavy-weight grace-period memory-ordering
> guarantees documented in the synchronize_sched() header comment and
> elsewhere.

> They pair with anything you might use to check for violation
> of these guarantees, or, simiarly, any ordering that you might use when
> relying on these guarantees.

I'm sure you know what that means, but I've no clue ;-) That is, I
wouldn't know where to start looking in the RCU implementation to verify
the barrier is either needed or sufficient. Unless you mean _everywhere_
:-)

> I could add something like  "/* Enforce GP memory ordering. */"
> 
> Or perhaps "/* See synchronize_sched() header. */"
> 
> I do not propose reproducing the synchronize_sched() header on each
> of these.  That would be verbose, even for me!  ;-)
> 
> Other thoughts?

Well, this is an UNLOCK+LOCK on non-matching lock variables upgrade to
full barrier thing, right?

To me its not clear which UNLOCK we even match here. I've just read the
sync_sched() header, but that doesn't help me either, so referring to
that isn't really helpful either.

In any case, I don't want to make too big a fuzz here, but I just
stumbled over a lot of unannotated barriers and figured I ought to say
something about it.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1241241 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date	2015-10-07 10:50 +0200
Subject	Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgSAa-86i-27@gated-at.bofh.it>
In reply to	#1241208

----- On Oct 7, 2015, at 3:51 AM, Peter Zijlstra peterz@infradead.org wrote:

> On Tue, Oct 06, 2015 at 01:58:50PM -0700, Paul E. McKenney wrote:
>> On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote:
>> > On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
>> > > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
>> > > +					      struct rcu_node *rnp, bool wake)
>> > > +{
>> > > +	unsigned long flags;
>> > > +	unsigned long mask;
>> > > +
>> > > +	raw_spin_lock_irqsave(&rnp->lock, flags);
>> > 
>> > Normally we require a comment with barriers, explaining the order and
>> > the pairing etc.. :-)
>> > 
>> > > +	smp_mb__after_unlock_lock();
>> 
>> Hmmmm...  That is not good.
>> 
>> Worse yet, I am missing comments on most of the pre-existing barriers
>> of this form.
> 
> Yes I noticed.. :/
> 
>> The purpose is to enforce the heavy-weight grace-period memory-ordering
>> guarantees documented in the synchronize_sched() header comment and
>> elsewhere.
> 
>> They pair with anything you might use to check for violation
>> of these guarantees, or, simiarly, any ordering that you might use when
>> relying on these guarantees.
> 
> I'm sure you know what that means, but I've no clue ;-) That is, I
> wouldn't know where to start looking in the RCU implementation to verify
> the barrier is either needed or sufficient. Unless you mean _everywhere_
> :-)

One example is the new membarrier system call. It relies on synchronize_sched()
to enforce this:

from kernel/membarrier.c:

 * All memory accesses performed in program order from each targeted thread
 * is guaranteed to be ordered with respect to sys_membarrier(). If we use
 * the semantic "barrier()" to represent a compiler barrier forcing memory
 * accesses to be performed in program order across the barrier, and
 * smp_mb() to represent explicit memory barriers forcing full memory
 * ordering across the barrier, we have the following ordering table for
 * each pair of barrier(), sys_membarrier() and smp_mb():
 *
 * The pair ordering is detailed as (O: ordered, X: not ordered):
 *
 *                        barrier()   smp_mb() sys_membarrier()
 *        barrier()          X           X            O
 *        smp_mb()           X           O            O
 *        sys_membarrier()   O           O            O

And include/uapi/linux/membarrier.h:

 * @MEMBARRIER_CMD_SHARED:  Execute a memory barrier on all running threads.
 *                          Upon return from system call, the caller thread
 *                          is ensured that all running threads have passed
 *                          through a state where all memory accesses to
 *                          user-space addresses match program order between
 *                          entry to and return from the system call
 *                          (non-running threads are de facto in such a
 *                          state). This covers threads from all processes
 *                          running on the system. This command returns 0.

I hope this sheds light on a userspace-facing interface to
synchronize_sched() and clarifies its expected semantic a bit.

Thanks,

Mathieu

> 
>> I could add something like  "/* Enforce GP memory ordering. */"
>> 
>> Or perhaps "/* See synchronize_sched() header. */"
>> 
>> I do not propose reproducing the synchronize_sched() header on each
>> of these.  That would be verbose, even for me!  ;-)
>> 
>> Other thoughts?
> 
> Well, this is an UNLOCK+LOCK on non-matching lock variables upgrade to
> full barrier thing, right?
> 
> To me its not clear which UNLOCK we even match here. I've just read the
> sync_sched() header, but that doesn't help me either, so referring to
> that isn't really helpful either.
> 
> In any case, I don't want to make too big a fuzz here, but I just
> stumbled over a lot of unannotated barriers and figured I ought to say
> something about it.

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1241372 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

From	Peter Zijlstra <peterz@infradead.org>
Date	2015-10-07 13:10 +0200
Subject	Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation
Message-ID	<qgULF-2Y1-31@gated-at.bofh.it>
In reply to	#1241241

On Wed, Oct 07, 2015 at 08:42:05AM +0000, Mathieu Desnoyers wrote:
> ----- On Oct 7, 2015, at 3:51 AM, Peter Zijlstra peterz@infradead.org wrote:
> 
> > On Tue, Oct 06, 2015 at 01:58:50PM -0700, Paul E. McKenney wrote:
> >> On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote:
> >> > On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
> >> > > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
> >> > > +					      struct rcu_node *rnp, bool wake)
> >> > > +{
> >> > > +	unsigned long flags;
> >> > > +	unsigned long mask;
> >> > > +
> >> > > +	raw_spin_lock_irqsave(&rnp->lock, flags);
> >> > 
> >> > Normally we require a comment with barriers, explaining the order and
> >> > the pairing etc.. :-)
> >> > 
> >> > > +	smp_mb__after_unlock_lock();
> >> 
> >> Hmmmm...  That is not good.
> >> 
> >> Worse yet, I am missing comments on most of the pre-existing barriers
> >> of this form.
> > 
> > Yes I noticed.. :/
> > 
> >> The purpose is to enforce the heavy-weight grace-period memory-ordering
> >> guarantees documented in the synchronize_sched() header comment and
> >> elsewhere.
> > 
> >> They pair with anything you might use to check for violation
> >> of these guarantees, or, simiarly, any ordering that you might use when
> >> relying on these guarantees.
> > 
> > I'm sure you know what that means, but I've no clue ;-) That is, I
> > wouldn't know where to start looking in the RCU implementation to verify
> > the barrier is either needed or sufficient. Unless you mean _everywhere_
> > :-)
> 
> One example is the new membarrier system call. It relies on synchronize_sched()
> to enforce this:

That again doesn't explain which UNLOCKs with non-matching lock values
it pairs with and what particular ordering is important here.

I'm fully well aware of what sync_sched() guarantees and how one can use
it, that is not the issue, what I'm saying is that a generic description
of sync_sched() doesn't help in figuring out WTH that barrier is for and
which other code I should also inspect.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

Page 1 of 4 [1] 2 3 4 Next page →

csiph-web

[PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4

Contents

#1240676 — [PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4

#1240679 — [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1241554 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1241660 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242100 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242498 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242515 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242516 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242663 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242848 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1242949 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1243022 — Re: [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited()

#1240680 — [PATCH tip/core/rcu 10/18] rcu: Stop silencing lockdep false positive for expedited grace periods

#1240681 — [PATCH tip/core/rcu 08/18] rcu: Make ->cpu_no_qs be a union for aggregate OR

#1240682 — [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

#1240965 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

#1240992 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

#1241208 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

#1241241 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

#1241372 — Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation