Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1558894 > unrolled thread

[PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11

Back to article view | Back to linux.kernel

Contents

  [PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 19/20] rcu: Eliminate unused expedited_normal counter "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 12/20] rcu: Re-enable TASKS_RCU for User Mode Linux "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 20/20] rcu: Add lockdep checks to synchronous expedited primitives "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 16/20] rcu: Enable RCU tracepoints by default to aid in debugging "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 03/20] Fix: Disable sys_membarrier when nohz_full is enabled "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 17/20] rcu: Fix comment in rcu_organize_nocb_kthreads() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 10/20] rcu: Add functions to test for trivial grace periods "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
      Re: [PATCH tip/core/rcu 10/20] rcu: Add functions to test for  trivial grace periods Josh Triplett <josh@joshtriplett.org> - 2017-01-16 09:10 +0100
    [PATCH tip/core/rcu 04/20] rcu: Only dump stalled-tasks stacks if there was a real stall "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 18/20] llist: Clarify comments about when locking is needed "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 15/20] rcu: Make rcu_cpu_starting() use its "cpu" argument "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 05/20] rcu: Remove unneeded rcu_process_callbacks() declarations "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    [PATCH tip/core/rcu 01/20] rcu: update: Make RCU_EXPEDITE_BOOT be the default "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-01-14 10:20 +0100
    Re: [PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11 Josh Triplett <josh@joshtriplett.org> - 2017-01-16 09:20 +0100

#1558894 — [PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11

Hello!

This series provides miscellaneous fixes:

1.	Make RCU_EXPEDITE_BOOT be the default in order to speed up
	boot, courtesy of Sebastian Andrzej Siewior.

2.	Make RCU suspicious-access lockdep splats use pr_err().

3.	Disable sys_membarrier when nohz_full is enabled, courtesy
	of Mathieu Desnoyers.

4.	Only dump stalled-tasks stacks if there was a real stall, courtesy
	of Byungchul Park.

5.	Remove unneeded rcu_process_callbacks() declarations.

6.	Remove unused but set "mask" variable, courtesy of Tobias Klauser.

7.	Remove short-term CPU kicking.

8.	Add long-term CPU kicking.

9.	Once again use NMI-based stack traces in stall warnings.

10.	Add functions to test for trivial grace periods.

11.	Make cond_resched() provide RCU quiescent state, which will
	hopefully lead to the removal of cond_resched_rcu_qs().

12.	Re-enable TASKS_RCU for User Mode Linux.

13.	Don't wake rcuc/X kthreads on NOCB CPUs.

14.	Add comment headers to expedited-grace-period counter functions.

15.	Make rcu_cpu_starting() use its "cpu" argument.

16.	Enable RCU tracepoints by default to aid in debugging, courtesy
	of Matt Fleming.

17.	Fix comment in rcu_organize_nocb_kthreads(), which no longer
	spawns kthreads.

18.	Clarify comments about when llist locking is needed, courtesy
	of Joel Fernandes.

19.	Eliminate unused expedited_normal counter.

20.	Add lockdep checks to synchronous expedited primitives.

							Thanx, Paul

------------------------------------------------------------------------

 Documentation/RCU/trace.txt |    5 ---
 include/linux/llist.h       |   37 +++++++++++++++-----------
 include/linux/rcupdate.h    |    8 +++++
 include/linux/rcutiny.h     |   10 +++++++
 include/linux/rcutree.h     |    2 +
 include/linux/sched.h       |    3 +-
 init/Kconfig                |   14 ----------
 kernel/locking/lockdep.c    |   12 ++++----
 kernel/membarrier.c         |    4 ++
 kernel/rcu/tiny.c           |    2 -
 kernel/rcu/tree.c           |   61 +++++++++++++++++++++++++++++++-------------
 kernel/rcu/tree.h           |    1 
 kernel/rcu/tree_exp.h       |   28 +++++++++++++++++++-
 kernel/rcu/tree_plugin.h    |   16 ++++++++++-
 kernel/rcu/tree_trace.c     |    3 --
 kernel/rcu/update.c         |    6 +---
 kernel/sched/core.c         |    1 
 lib/Kconfig.debug           |    1 
 18 files changed, 143 insertions(+), 71 deletions(-)

[toc] | [next] | [standalone]

#1558895 — [PATCH tip/core/rcu 19/20] rcu: Eliminate unused expedited_normal counter

Expedited grace periods no longer fall back to normal grace periods
in response to lock contention, given that expedited grace periods
now use the rcu_node tree so as to avoid contention.  This commit
therfore removes the expedited_normal counter.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/trace.txt | 5 +----
 kernel/rcu/tree.h           | 1 -
 kernel/rcu/tree_trace.c     | 3 +--
 3 files changed, 2 insertions(+), 7 deletions(-)

diff --git a/Documentation/RCU/trace.txt b/Documentation/RCU/trace.txt
index 00a3a38b375a..6549012033f9 100644
--- a/Documentation/RCU/trace.txt
+++ b/Documentation/RCU/trace.txt
@@ -237,7 +237,7 @@ o	"ktl" is the low-order 16 bits (in hexadecimal) of the count of
 
 The output of "cat rcu/rcu_preempt/rcuexp" looks as follows:
 
-s=21872 wd1=0 wd2=0 wd3=5 n=0 enq=0 sc=21872
+s=21872 wd1=0 wd2=0 wd3=5 enq=0 sc=21872
 
 These fields are as follows:
 
@@ -249,9 +249,6 @@ o	"wd1", "wd2", and "wd3" are the number of times that an attempt
 	completed an expedited grace period that satisfies the attempted
 	request.  "Our work is done."
 
-o	"n" is number of times that a concurrent CPU-hotplug operation
-	forced a fallback to a normal grace period.
-
 o	"enq" is the number of quiescent states still outstanding.
 
 o	"sc" is the number of times that the attempt to start a
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index fe98dd24adf8..8f750dffb0dd 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -521,7 +521,6 @@ struct rcu_state {
 	struct mutex exp_mutex;			/* Serialize expedited GP. */
 	struct mutex exp_wake_mutex;		/* Serialize wakeup. */
 	unsigned long expedited_sequence;	/* Take a ticket. */
-	atomic_long_t expedited_normal;		/* # fallbacks to normal. */
 	atomic_t expedited_need_qs;		/* # CPUs left to check in. */
 	struct swait_queue_head expedited_wq;	/* Wait for check-ins. */
 	int ncpus_snap;				/* # CPUs seen last time. */
diff --git a/kernel/rcu/tree_trace.c b/kernel/rcu/tree_trace.c
index b1f28972872c..2e932cd1da31 100644
--- a/kernel/rcu/tree_trace.c
+++ b/kernel/rcu/tree_trace.c
@@ -194,9 +194,8 @@ static int show_rcuexp(struct seq_file *m, void *v)
 		s2 += atomic_long_read(&rdp->exp_workdone2);
 		s3 += atomic_long_read(&rdp->exp_workdone3);
 	}
-	seq_printf(m, "s=%lu wd0=%lu wd1=%lu wd2=%lu wd3=%lu n=%lu enq=%d sc=%lu\n",
+	seq_printf(m, "s=%lu wd0=%lu wd1=%lu wd2=%lu wd3=%lu enq=%d sc=%lu\n",
 		   rsp->expedited_sequence, s0, s1, s2, s3,
-		   atomic_long_read(&rsp->expedited_normal),
 		   atomic_read(&rsp->expedited_need_qs),
 		   rsp->expedited_sequence / 2);
 	return 0;
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558896 — [PATCH tip/core/rcu 12/20] rcu: Re-enable TASKS_RCU for User Mode Linux

Now that User Mode Linux supports arch_irqs_disabled_flags(), this
commit re-enables TASKS_RCU for User Mode Linux.

Reported-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 init/Kconfig | 1 -
 1 file changed, 1 deletion(-)

diff --git a/init/Kconfig b/init/Kconfig
index 96e6d56acd50..d0caacb4bc14 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -529,7 +529,6 @@ config SRCU
 config TASKS_RCU
 	bool
 	default n
-	depends on !UML
 	select SRCU
 	help
 	  This option enables a task-based RCU implementation that uses
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558897 — [PATCH tip/core/rcu 20/20] rcu: Add lockdep checks to synchronous expedited primitives

The non-expedited synchronize_*rcu() primitives have lockdep checks, but
their expedited counterparts lack these checks.  This commit therefore
adds these checks to the expedited synchronize_*rcu() primitives.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree_exp.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index 303df97bbfc5..f3e214898e3a 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -639,6 +639,11 @@ void synchronize_sched_expedited(void)
 {
 	struct rcu_state *rsp = &rcu_sched_state;
 
+	RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
+			 lock_is_held(&rcu_lock_map) ||
+			 lock_is_held(&rcu_sched_lock_map),
+			 "Illegal synchronize_sched_expedited() in RCU read-side critical section");
+
 	/* If only one CPU, this is automatically a grace period. */
 	if (rcu_blocking_is_gp())
 		return;
@@ -708,6 +713,11 @@ void synchronize_rcu_expedited(void)
 {
 	struct rcu_state *rsp = rcu_state_p;
 
+	RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
+			 lock_is_held(&rcu_lock_map) ||
+			 lock_is_held(&rcu_sched_lock_map),
+			 "Illegal synchronize_rcu_expedited() in RCU read-side critical section");
+
 	if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
 		return;
 	_synchronize_rcu_expedited(rsp, sync_rcu_exp_handler);
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558898 — [PATCH tip/core/rcu 16/20] rcu: Enable RCU tracepoints by default to aid in debugging

From: Matt Fleming <matt@codeblueprint.co.uk>

While debugging a performance issue I needed to understand why
RCU sofitrqs were firing so frequently.

Unfortunately, the RCU callback tracepoints are hidden behind
CONFIG_RCU_TRACE which defaults to off in the upstream kernel and is
likely to also be disabled in enterprise distribution configs.

Enable it by default for CONFIG_TREE_RCU. However, we must keep it
disabled for tiny RCU, because it would otherwise pull in a large
amount of code that would make tiny RCU less than tiny.

I ran some file system metadata intensive workloads (git checkout,
FS-Mark) on a variety of machines with this patch and saw no
detectable change in performance.

Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 lib/Kconfig.debug | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b06848a104e6..5655bd95919a 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1450,6 +1450,7 @@ config RCU_CPU_STALL_TIMEOUT
 config RCU_TRACE
 	bool "Enable tracing for RCU"
 	depends on DEBUG_KERNEL
+	default y if TREE_RCU
 	select TRACE_CLOCK
 	help
 	  This option provides tracing in RCU which presents stats
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558899 — [PATCH tip/core/rcu 03/20] Fix: Disable sys_membarrier when nohz_full is enabled

From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Userspace applications should be allowed to expect the membarrier system
call with MEMBARRIER_CMD_SHARED command to issue memory barriers on
nohz_full CPUs, but synchronize_sched() does not take those into
account.

Given that we do not want unrelated processes to be able to affect
real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier
is invoked on a kernel with enabled nohz_full CPUs.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Josh Triplett <josh@joshtriplett.org>
CC: Steven Rostedt <rostedt@goodmis.org>
CC: <stable@vger.kernel.org>	[3.10+]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Acked-by: Lai Jiangshan <jiangshanlai@gmail.com>
---
 kernel/membarrier.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/membarrier.c b/kernel/membarrier.c
index 536c727a56e9..9f9284f37f8d 100644
--- a/kernel/membarrier.c
+++ b/kernel/membarrier.c
@@ -16,6 +16,7 @@
 
 #include <linux/syscalls.h>
 #include <linux/membarrier.h>
+#include <linux/tick.h>
 
 /*
  * Bitmask made from a "or" of all commands within enum membarrier_cmd,
@@ -51,6 +52,9 @@
  */
 SYSCALL_DEFINE2(membarrier, int, cmd, int, flags)
 {
+	/* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */
+	if (tick_nohz_full_enabled())
+		return -ENOSYS;
 	if (unlikely(flags))
 		return -EINVAL;
 	switch (cmd) {
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558900 — [PATCH tip/core/rcu 17/20] rcu: Fix comment in rcu_organize_nocb_kthreads()

It used to be that the rcuo callback-offload kthreads were spawned
in rcu_organize_nocb_kthreads(), and the comment before the "for"
loop says as much.  However, this spawning has long since moved to
the CPU-hotplug code, so this commit fixes this comment.

Reported-by: Michalis Kokologiannakis <mixaskok@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree_plugin.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e92d67a3fad2..9e295c3516dc 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -2377,8 +2377,9 @@ static void __init rcu_organize_nocb_kthreads(struct rcu_state *rsp)
 	}
 
 	/*
-	 * Each pass through this loop sets up one rcu_data structure and
-	 * spawns one rcu_nocb_kthread().
+	 * Each pass through this loop sets up one rcu_data structure.
+	 * Should the corresponding CPU come online in the future, then
+	 * we will spawn the needed set of rcu_nocb_kthread() kthreads.
 	 */
 	for_each_cpu(cpu, rcu_nocb_mask) {
 		rdp = per_cpu_ptr(rsp->rda, cpu);
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558901 — [PATCH tip/core/rcu 10/20] rcu: Add functions to test for trivial grace periods

Under some circumstances, RCU grace periods are zero cost.  For
RCU-preempt, this is the case during boot, and for RCU-bh and RCU-sched,
this is the case if there is only one CPU.  This means that RCU users
might wish to dispense with grace-period-avoidance strategies when
grace periods are zero cost, so this commit adds rcu_trivial_gp(),
rcu_bh_trivial_gp(), and rcu_sched_trivial_gp() to test for these
conditions.  Because the conditions leading to zero-cost grace periods
can change at any time (for example, when a second CPU is onlined), these
functions should be used as performance hints, and must not be relied
on for correctness.  For example, even if rcu_trivial_gp() returns true,
you are required to invoke synchronize_rcu().

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/rcupdate.h |  8 ++++++++
 include/linux/rcutiny.h  | 10 ++++++++++
 include/linux/rcutree.h  |  2 ++
 kernel/rcu/tree.c        | 24 ++++++++++++++++++++++++
 kernel/rcu/tree_plugin.h | 11 +++++++++++
 5 files changed, 55 insertions(+)

diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 01f71e1d2e94..a6222478b87d 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -293,6 +293,7 @@ void __rcu_read_lock(void);
 void __rcu_read_unlock(void);
 void rcu_read_unlock_special(struct task_struct *t);
 void synchronize_rcu(void);
+bool rcu_trivial_gp(void);
 
 /*
  * Defined as a macro as it is a very low level header included from
@@ -448,6 +449,13 @@ bool __rcu_is_watching(void);
 #define RCU_SCHEDULER_INIT	1
 #define RCU_SCHEDULER_RUNNING	2
 
+#ifndef CONFIG_PREEMPT_RCU
+static inline bool rcu_trivial_gp(void)
+{
+	return rcu_sched_trivial_gp();
+}
+#endif /* #ifndef CONFIG_PREEMPT_RCU */
+
 /*
  * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic
  * initialization and destruction of rcu_head on the stack. rcu_head structures
diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index ac81e4063b40..a77dafe79813 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -82,6 +82,16 @@ static inline void synchronize_sched_expedited(void)
 	synchronize_sched();
 }
 
+static inline bool rcu_sched_trivial_gp(void)
+{
+	return true;
+}
+
+static inline bool rcu_bh_trivial_gp(void)
+{
+	return true;
+}
+
 static inline void kfree_call_rcu(struct rcu_head *head,
 				  rcu_callback_t func)
 {
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 63a4e4cf40a5..fcd61cb08851 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -47,6 +47,8 @@ static inline void rcu_virt_note_context_switch(int cpu)
 void synchronize_rcu_bh(void);
 void synchronize_sched_expedited(void);
 void synchronize_rcu_expedited(void);
+bool rcu_sched_trivial_gp(void);
+bool rcu_bh_trivial_gp(void);
 
 void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func);
 
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index d7b63b88434b..ed5a17aca281 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3293,6 +3293,18 @@ void synchronize_sched(void)
 EXPORT_SYMBOL_GPL(synchronize_sched);
 
 /**
+ * rcu_sched_trivial_gp - Are RCU-sched grace periods trivially zero cost?
+ *
+ * Returns true if RCU-sched grace periods are currently zero cost, which
+ * they are if there is only one CPU.  Note that unless you take steps to
+ * prevent it, the number of CPUs might change at any time.
+ */
+bool rcu_sched_trivial_gp(void)
+{
+	return rcu_blocking_is_gp();
+}
+
+/**
  * synchronize_rcu_bh - wait until an rcu_bh grace period has elapsed.
  *
  * Control will return to the caller some time after a full rcu_bh grace
@@ -3320,6 +3332,18 @@ void synchronize_rcu_bh(void)
 EXPORT_SYMBOL_GPL(synchronize_rcu_bh);
 
 /**
+ * rcu_bh_trivial_gp - Are RCU-bh grace periods trivially zero cost?
+ *
+ * Returns true if RCU-bh grace periods are currently zero cost, which
+ * they are if there is only one CPU.  Note that unless you take steps to
+ * prevent it, the number of CPUs might change at any time.
+ */
+bool rcu_bh_trivial_gp(void)
+{
+	return rcu_blocking_is_gp();
+}
+
+/**
  * get_state_synchronize_rcu - Snapshot current RCU state
  *
  * Returns a cookie that is used by a later call to cond_synchronize_rcu()
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 56583e764ebf..e92d67a3fad2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -680,6 +680,17 @@ void synchronize_rcu(void)
 EXPORT_SYMBOL_GPL(synchronize_rcu);
 
 /**
+ * rcu_trivial_gp - Are RCU grace periods trivially zero cost?
+ *
+ * Returns true if RCU grace periods are currently zero cost, which
+ * they are during boot.
+ */
+bool rcu_trivial_gp(void)
+{
+	return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE;
+}
+
+/**
  * rcu_barrier - Wait until all in-flight call_rcu() callbacks complete.
  *
  * Note that this primitive does not necessarily wait for an RCU grace period
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1559506 — Re: [PATCH tip/core/rcu 10/20] rcu: Add functions to test for trivial grace periods

On Sat, Jan 14, 2017 at 01:13:11AM -0800, Paul E. McKenney wrote:
> Under some circumstances, RCU grace periods are zero cost.  For
> RCU-preempt, this is the case during boot, and for RCU-bh and RCU-sched,
> this is the case if there is only one CPU.  This means that RCU users
> might wish to dispense with grace-period-avoidance strategies when
> grace periods are zero cost, so this commit adds rcu_trivial_gp(),
> rcu_bh_trivial_gp(), and rcu_sched_trivial_gp() to test for these
> conditions.  Because the conditions leading to zero-cost grace periods
> can change at any time (for example, when a second CPU is onlined), these
> functions should be used as performance hints, and must not be relied
> on for correctness.  For example, even if rcu_trivial_gp() returns true,
> you are required to invoke synchronize_rcu().
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Do you have anything planned that uses these functions?

Exposing this information, rather than just letting callers call
synchronize_rcu() and sometimes get "free" grace periods, seems
potentially error-prone.

If you keep these functions, please expand the comments above them to
explicitly include the explanation in the commit message, and
specifically that you must always call the corresponding synchronize
function even if these functions return true.

>  include/linux/rcupdate.h |  8 ++++++++
>  include/linux/rcutiny.h  | 10 ++++++++++
>  include/linux/rcutree.h  |  2 ++
>  kernel/rcu/tree.c        | 24 ++++++++++++++++++++++++
>  kernel/rcu/tree_plugin.h | 11 +++++++++++
>  5 files changed, 55 insertions(+)
> 
> diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> index 01f71e1d2e94..a6222478b87d 100644
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -293,6 +293,7 @@ void __rcu_read_lock(void);
>  void __rcu_read_unlock(void);
>  void rcu_read_unlock_special(struct task_struct *t);
>  void synchronize_rcu(void);
> +bool rcu_trivial_gp(void);
>  
>  /*
>   * Defined as a macro as it is a very low level header included from
> @@ -448,6 +449,13 @@ bool __rcu_is_watching(void);
>  #define RCU_SCHEDULER_INIT	1
>  #define RCU_SCHEDULER_RUNNING	2
>  
> +#ifndef CONFIG_PREEMPT_RCU
> +static inline bool rcu_trivial_gp(void)
> +{
> +	return rcu_sched_trivial_gp();
> +}
> +#endif /* #ifndef CONFIG_PREEMPT_RCU */
> +
>  /*
>   * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic
>   * initialization and destruction of rcu_head on the stack. rcu_head structures
> diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
> index ac81e4063b40..a77dafe79813 100644
> --- a/include/linux/rcutiny.h
> +++ b/include/linux/rcutiny.h
> @@ -82,6 +82,16 @@ static inline void synchronize_sched_expedited(void)
>  	synchronize_sched();
>  }
>  
> +static inline bool rcu_sched_trivial_gp(void)
> +{
> +	return true;
> +}
> +
> +static inline bool rcu_bh_trivial_gp(void)
> +{
> +	return true;
> +}
> +
>  static inline void kfree_call_rcu(struct rcu_head *head,
>  				  rcu_callback_t func)
>  {
> diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
> index 63a4e4cf40a5..fcd61cb08851 100644
> --- a/include/linux/rcutree.h
> +++ b/include/linux/rcutree.h
> @@ -47,6 +47,8 @@ static inline void rcu_virt_note_context_switch(int cpu)
>  void synchronize_rcu_bh(void);
>  void synchronize_sched_expedited(void);
>  void synchronize_rcu_expedited(void);
> +bool rcu_sched_trivial_gp(void);
> +bool rcu_bh_trivial_gp(void);
>  
>  void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func);
>  
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index d7b63b88434b..ed5a17aca281 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -3293,6 +3293,18 @@ void synchronize_sched(void)
>  EXPORT_SYMBOL_GPL(synchronize_sched);
>  
>  /**
> + * rcu_sched_trivial_gp - Are RCU-sched grace periods trivially zero cost?
> + *
> + * Returns true if RCU-sched grace periods are currently zero cost, which
> + * they are if there is only one CPU.  Note that unless you take steps to
> + * prevent it, the number of CPUs might change at any time.
> + */
> +bool rcu_sched_trivial_gp(void)
> +{
> +	return rcu_blocking_is_gp();
> +}
> +
> +/**
>   * synchronize_rcu_bh - wait until an rcu_bh grace period has elapsed.
>   *
>   * Control will return to the caller some time after a full rcu_bh grace
> @@ -3320,6 +3332,18 @@ void synchronize_rcu_bh(void)
>  EXPORT_SYMBOL_GPL(synchronize_rcu_bh);
>  
>  /**
> + * rcu_bh_trivial_gp - Are RCU-bh grace periods trivially zero cost?
> + *
> + * Returns true if RCU-bh grace periods are currently zero cost, which
> + * they are if there is only one CPU.  Note that unless you take steps to
> + * prevent it, the number of CPUs might change at any time.
> + */
> +bool rcu_bh_trivial_gp(void)
> +{
> +	return rcu_blocking_is_gp();
> +}
> +
> +/**
>   * get_state_synchronize_rcu - Snapshot current RCU state
>   *
>   * Returns a cookie that is used by a later call to cond_synchronize_rcu()
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index 56583e764ebf..e92d67a3fad2 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -680,6 +680,17 @@ void synchronize_rcu(void)
>  EXPORT_SYMBOL_GPL(synchronize_rcu);
>  
>  /**
> + * rcu_trivial_gp - Are RCU grace periods trivially zero cost?
> + *
> + * Returns true if RCU grace periods are currently zero cost, which
> + * they are during boot.
> + */
> +bool rcu_trivial_gp(void)
> +{
> +	return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE;
> +}
> +
> +/**
>   * rcu_barrier - Wait until all in-flight call_rcu() callbacks complete.
>   *
>   * Note that this primitive does not necessarily wait for an RCU grace period
> -- 
> 2.5.2
> 

[toc] | [prev] | [next] | [standalone]

#1558902 — [PATCH tip/core/rcu 04/20] rcu: Only dump stalled-tasks stacks if there was a real stall

From: Byungchul Park <byungchul.park@lge.com>

The print_other_cpu_stall() function currently unconditionally invokes
rcu_print_detail_task_stall().  This is OK because if there was a stall
sufficient to cause print_other_cpu_stall() to be invoked, that stall
is very likely to persist through the entire print_other_cpu_stall()
execution.  However, if the stall did not persist, the variable ndetected
will be zero, and that variable is already tested in an "if" statement.
Therefore, this commit moves the call to rcu_print_detail_task_stall()
under that pre-existing "if" to improve readability, with a very rare
reduction in overhead.

Signed-off-by: Byungchul Park <byungchul.park@lge.com>
[ paulmck: Reworked commit log. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index cb4e2056ccf3..6232d2f9a84e 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1379,6 +1379,9 @@ static void print_other_cpu_stall(struct rcu_state *rsp, unsigned long gpnum)
 	       (long)rsp->gpnum, (long)rsp->completed, totqlen);
 	if (ndetected) {
 		rcu_dump_cpu_stacks(rsp);
+
+		/* Complain about tasks blocking the grace period. */
+		rcu_print_detail_task_stall(rsp);
 	} else {
 		if (READ_ONCE(rsp->gpnum) != gpnum ||
 		    READ_ONCE(rsp->completed) == gpnum) {
@@ -1395,9 +1398,6 @@ static void print_other_cpu_stall(struct rcu_state *rsp, unsigned long gpnum)
 		}
 	}
 
-	/* Complain about tasks blocking the grace period. */
-	rcu_print_detail_task_stall(rsp);
-
 	rcu_check_gp_kthread_starvation(rsp);
 
 	panic_on_rcu_stall();
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558903 — [PATCH tip/core/rcu 18/20] llist: Clarify comments about when locking is needed

From: Joel Fernandes <joelaf@google.com>

llist.h comments are confusing about when locking is needed versus when it
isn't. Clarify these comments by being more descriptive about why locking is
needed for llist_del_first.

Cc: Ingo Molnar <mingo@kernel.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Huang Ying <ying.huang@intel.com>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 include/linux/llist.h | 37 +++++++++++++++++++++----------------
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/include/linux/llist.h b/include/linux/llist.h
index fd4ca0b4fe0f..171baa90f6f6 100644
--- a/include/linux/llist.h
+++ b/include/linux/llist.h
@@ -3,28 +3,33 @@
 /*
  * Lock-less NULL terminated single linked list
  *
- * If there are multiple producers and multiple consumers, llist_add
- * can be used in producers and llist_del_all can be used in
- * consumers.  They can work simultaneously without lock.  But
- * llist_del_first can not be used here.  Because llist_del_first
- * depends on list->first->next does not changed if list->first is not
- * changed during its operation, but llist_del_first, llist_add,
- * llist_add (or llist_del_all, llist_add, llist_add) sequence in
- * another consumer may violate that.
- *
- * If there are multiple producers and one consumer, llist_add can be
- * used in producers and llist_del_all or llist_del_first can be used
- * in the consumer.
- *
- * This can be summarized as follow:
+ * Cases where locking is not needed:
+ * If there are multiple producers and multiple consumers, llist_add can be
+ * used in producers and llist_del_all can be used in consumers simultaneously
+ * without locking. Also a single consumer can use llist_del_first while
+ * multiple producers simultaneously use llist_add, without any locking.
+ *
+ * Cases where locking is needed:
+ * If we have multiple consumers with llist_del_first used in one consumer, and
+ * llist_del_first or llist_del_all used in other consumers, then a lock is
+ * needed.  This is because llist_del_first depends on list->first->next not
+ * changing, but without lock protection, there's no way to be sure about that
+ * if a preemption happens in the middle of the delete operation and on being
+ * preempted back, the list->first is the same as before causing the cmpxchg in
+ * llist_del_first to succeed. For example, while a llist_del_first operation
+ * is in progress in one consumer, then a llist_del_first, llist_add,
+ * llist_add (or llist_del_all, llist_add, llist_add) sequence in another
+ * consumer may cause violations.
+ *
+ * This can be summarized as follows:
  *
  *           |   add    | del_first |  del_all
  * add       |    -     |     -     |     -
  * del_first |          |     L     |     L
  * del_all   |          |           |     -
  *
- * Where "-" stands for no lock is needed, while "L" stands for lock
- * is needed.
+ * Where, a particular row's operation can happen concurrently with a column's
+ * operation, with "-" being no lock needed, while "L" being lock is needed.
  *
  * The list entries deleted via llist_del_all can be traversed with
  * traversing function such as llist_for_each etc.  But the list
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558904 — [PATCH tip/core/rcu 15/20] rcu: Make rcu_cpu_starting() use its "cpu" argument

The rcu_cpu_starting() function uses this_cpu_ptr() to locate the
incoming CPU's rcu_data structure.  This works for the boot CPU and for
all CPUs onlined after rcu_init() executes (during very early boot).
Currently, this is the full set of CPUs, so all is well.  But if
anyone ever parallelizes boot before rcu_init() time, it will fail.
This commit therefore substitutes the rcu_cpu_starting() function's
this_cpu_pointer() for per_cpu_ptr(), future-proofing the code and
(arguably) improving readability.

This commit inadvertently fixes a latent bug: If there ever had been
more than just the boot CPU online at rcu_init() time, the old code
would not initialize the non-boot CPUs, but rather would repeatedly
initialize the boot CPU.

Reported-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index cac80e08b353..b1e803fbb46d 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3897,7 +3897,7 @@ void rcu_cpu_starting(unsigned int cpu)
 	struct rcu_state *rsp;
 
 	for_each_rcu_flavor(rsp) {
-		rdp = this_cpu_ptr(rsp->rda);
+		rdp = per_cpu_ptr(rsp->rda, cpu);
 		rnp = rdp->mynode;
 		mask = rdp->grpmask;
 		raw_spin_lock_irqsave_rcu_node(rnp, flags);
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558905 — [PATCH tip/core/rcu 05/20] rcu: Remove unneeded rcu_process_callbacks() declarations

The declarations of __rcu_process_callbacks() and rcu_process_callbacks()
are not needed, as the definition of both of these functions appear before
any uses.  This commit therefore removes both declarations.

Reported-by: "Ahmed, Iftekhar" <ahmedi@oregonstate.edu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcu/tiny.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/kernel/rcu/tiny.c b/kernel/rcu/tiny.c
index b23a4d076f3d..fa6a48d3917b 100644
--- a/kernel/rcu/tiny.c
+++ b/kernel/rcu/tiny.c
@@ -41,8 +41,6 @@
 
 /* Forward declarations for tiny_plugin.h. */
 struct rcu_ctrlblk;
-static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp);
-static void rcu_process_callbacks(struct softirq_action *unused);
 static void __call_rcu(struct rcu_head *head,
 		       rcu_callback_t func,
 		       struct rcu_ctrlblk *rcp);
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1558906 — [PATCH tip/core/rcu 01/20] rcu: update: Make RCU_EXPEDITE_BOOT be the default

From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

RCU_EXPEDITE_BOOT should speed up the boot process by enforcing
synchronize_rcu_expedited() instead of synchronize_rcu() during the boot
process. There should be no reason why one does not want this and there
is no need worry about real time latency at this point.
Therefore make it default.

Note that users wishing to avoid expediting entirely, for example when
bringing up new hardware possibly having flaky IPIs, can use the
rcu_normal boot parameter to override boot-time expediting.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ paulmck: Reworded commit log. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 init/Kconfig        | 13 -------------
 kernel/rcu/update.c |  6 ++----
 2 files changed, 2 insertions(+), 17 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 223b734abccd..96e6d56acd50 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -781,19 +781,6 @@ config RCU_NOCB_CPU_ALL
 
 endchoice
 
-config RCU_EXPEDITE_BOOT
-	bool
-	default n
-	help
-	  This option enables expedited grace periods at boot time,
-	  as if rcu_expedite_gp() had been invoked early in boot.
-	  The corresponding rcu_unexpedite_gp() is invoked from
-	  rcu_end_inkernel_boot(), which is intended to be invoked
-	  at the end of the kernel-only boot sequence, just before
-	  init is exec'ed.
-
-	  Accept the default if unsure.
-
 endmenu # "RCU Subsystem"
 
 config BUILD_BIN2C
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 4f6db7e6a117..9e03db9ea9c0 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -132,8 +132,7 @@ bool rcu_gp_is_normal(void)
 }
 EXPORT_SYMBOL_GPL(rcu_gp_is_normal);
 
-static atomic_t rcu_expedited_nesting =
-	ATOMIC_INIT(IS_ENABLED(CONFIG_RCU_EXPEDITE_BOOT) ? 1 : 0);
+static atomic_t rcu_expedited_nesting = ATOMIC_INIT(1);
 
 /*
  * Should normal grace-period primitives be expedited?  Intended for
@@ -182,8 +181,7 @@ EXPORT_SYMBOL_GPL(rcu_unexpedite_gp);
  */
 void rcu_end_inkernel_boot(void)
 {
-	if (IS_ENABLED(CONFIG_RCU_EXPEDITE_BOOT))
-		rcu_unexpedite_gp();
+	rcu_unexpedite_gp();
 	if (rcu_normal_after_boot)
 		WRITE_ONCE(rcu_normal, 1);
 }
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1559509

On Sat, Jan 14, 2017 at 01:12:55AM -0800, Paul E. McKenney wrote:
> Hello!
> 
> This series provides miscellaneous fixes:
> 
> 1.	Make RCU_EXPEDITE_BOOT be the default in order to speed up
> 	boot, courtesy of Sebastian Andrzej Siewior.
> 
> 2.	Make RCU suspicious-access lockdep splats use pr_err().
> 
> 3.	Disable sys_membarrier when nohz_full is enabled, courtesy
> 	of Mathieu Desnoyers.
> 
> 4.	Only dump stalled-tasks stacks if there was a real stall, courtesy
> 	of Byungchul Park.
> 
> 5.	Remove unneeded rcu_process_callbacks() declarations.
> 
> 6.	Remove unused but set "mask" variable, courtesy of Tobias Klauser.
> 
> 7.	Remove short-term CPU kicking.
> 
> 8.	Add long-term CPU kicking.
> 
> 9.	Once again use NMI-based stack traces in stall warnings.
> 
> 10.	Add functions to test for trivial grace periods.
> 
> 11.	Make cond_resched() provide RCU quiescent state, which will
> 	hopefully lead to the removal of cond_resched_rcu_qs().
> 
> 12.	Re-enable TASKS_RCU for User Mode Linux.
> 
> 13.	Don't wake rcuc/X kthreads on NOCB CPUs.
> 
> 14.	Add comment headers to expedited-grace-period counter functions.
> 
> 15.	Make rcu_cpu_starting() use its "cpu" argument.
> 
> 16.	Enable RCU tracepoints by default to aid in debugging, courtesy
> 	of Matt Fleming.
> 
> 17.	Fix comment in rcu_organize_nocb_kthreads(), which no longer
> 	spawns kthreads.
> 
> 18.	Clarify comments about when llist locking is needed, courtesy
> 	of Joel Fernandes.
> 
> 19.	Eliminate unused expedited_normal counter.
> 
> 20.	Add lockdep checks to synchronous expedited primitives.

I replied to patches 2, 8, 10, and 11 with comments.  For the rest:
Reviewed-by: Josh Triplett <josh@joshtriplett.org>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web