Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1622335 > unrolled thread

[PATCH tip/core/rcu 0/9] Documentation updates for 4.12

Back to article view | Back to linux.kernel

Contents

  [PATCH tip/core/rcu 0/9] Documentation updates for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 6/9] doc: Update rcu_assign_pointer() definition in whatisRCU.txt "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 9/9] doc: Update control-dependencies section of memory-barriers.txt "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 8/9] doc: Emphasize that "toy" RCU requires recursive rwlock "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 7/9] doc: Update the comparisons rule in rcu_dereference.txt "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 3/9] doc: Update stallwarn.txt to make causes more prominent "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200
    [PATCH tip/core/rcu 5/9] doc: Update requirements based on recent changes "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 18:50 +0200

#1622335 — [PATCH tip/core/rcu 0/9] Documentation updates for 4.12

Hello!

This series provides documentation updates:

1.	Synchronous RCU grace periods are now legal throughout boot.

2.	Add mid-boot operation to expedited grace periods.

3.	Update stallwarn.txt to make causes more prominent.

4.	Update RCU data-structure documentation for rcu_segcblist.
	(Corresponding code changes are in the SRCU series.)

5.	Update requirements based on recent changes.

6.	doc: Update rcu_assign_pointer() definition in whatisRCU.txt.

7.	Update the comparisons rule in rcu_dereference.txt, courtesy
	of Michalis Kokologiannakis.

8.	Emphasize that "toy" RCU requires recursive rwlock.

9.	Update control-dependencies section of memory-barriers.txt,
	courtesy of pierre Kuo.

							Thanx, Paul

------------------------------------------------------------------------

 RCU/Design/Data-Structures/Data-Structures.html                 |  207 ++++++----
 RCU/Design/Data-Structures/nxtlist.svg                          |   34 -
 RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.html |   47 ++
 RCU/Design/Requirements/Requirements.html                       |  201 ++++++---
 RCU/rcu_dereference.txt                                         |    9 
 RCU/stallwarn.txt                                               |  190 ++++-----
 RCU/whatisRCU.txt                                               |   33 -
 memory-barriers.txt                                             |    2 
 8 files changed, 469 insertions(+), 254 deletions(-)

[toc] | [next] | [standalone]

#1622336 — [PATCH tip/core/rcu 6/9] doc: Update rcu_assign_pointer() definition in whatisRCU.txt

The rcu_assign_pointer() macro has changed over time, and the version
in Documentation/RCU/whatisRCU.txt has not kept up.  This commit brings
it into 2017, albeit in a simplified fashion.

Reported-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/whatisRCU.txt | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
index 5cbd8b2395b8..6b0337008f9c 100644
--- a/Documentation/RCU/whatisRCU.txt
+++ b/Documentation/RCU/whatisRCU.txt
@@ -587,20 +587,21 @@ It is extremely simple:
 		write_unlock(&rcu_gp_mutex);
 	}
 
-[You can ignore rcu_assign_pointer() and rcu_dereference() without
-missing much.  But here they are anyway.  And whatever you do, don't
-forget about them when submitting patches making use of RCU!]
-
-	#define rcu_assign_pointer(p, v)	({ \
-							smp_wmb(); \
-							(p) = (v); \
-						})
-
-	#define rcu_dereference(p)     ({ \
-					typeof(p) _________p1 = p; \
-					smp_read_barrier_depends(); \
-					(_________p1); \
-					})
+[You can ignore rcu_assign_pointer() and rcu_dereference() without missing
+much.  But here are simplified versions anyway.  And whatever you do,
+don't forget about them when submitting patches making use of RCU!]
+
+	#define rcu_assign_pointer(p, v) \
+	({ \
+		smp_store_release(&(p), (v)); \
+	})
+
+	#define rcu_dereference(p) \
+	({ \
+		typeof(p) _________p1 = p; \
+		smp_read_barrier_depends(); \
+		(_________p1); \
+	})
 
 
 The rcu_read_lock() and rcu_read_unlock() primitive read-acquire
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622339 — [PATCH tip/core/rcu 9/9] doc: Update control-dependencies section of memory-barriers.txt

From: pierre Kuo <vichy.kuo@gmail.com>

In the following example, if MAX is defined to be 1, then the compiler
knows (Q % MAX) is equal to zero.  The compiler can therefore throw
away the "then" branch (and the "if"), retaining only the "else" branch.

	q = READ_ONCE(a);
	if (q % MAX) {
		WRITE_ONCE(b, 1);
		do_something();
	} else {
		WRITE_ONCE(b, 2);
		do_something_else();
	}

It is therefore necessary to modify the example like this:

        q = READ_ONCE(a);
-       WRITE_ONCE(b, 1);
+       WRITE_ONCE(b, 2);
        do_something_else();

Signed-off-by: pierre Kuo <vichy.kuo@gmail.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/memory-barriers.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index d2b0a8d81258..08329cb857ed 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -768,7 +768,7 @@ equal to zero, in which case the compiler is within its rights to
 transform the above code into the following:
 
 	q = READ_ONCE(a);
-	WRITE_ONCE(b, 1);
+	WRITE_ONCE(b, 2);
 	do_something_else();
 
 Given this transformation, the CPU is not required to respect the ordering
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622341 — [PATCH tip/core/rcu 8/9] doc: Emphasize that "toy" RCU requires recursive rwlock

Reported-by: "yangzc@uit.com.cn" <yangzc@uit.com.cn>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
index 6b0337008f9c..8c131a1c62ea 100644
--- a/Documentation/RCU/whatisRCU.txt
+++ b/Documentation/RCU/whatisRCU.txt
@@ -562,7 +562,9 @@ This section presents a "toy" RCU implementation that is based on
 familiar locking primitives.  Its overhead makes it a non-starter for
 real-life use, as does its lack of scalability.  It is also unsuitable
 for realtime use, since it allows scheduling latency to "bleed" from
-one read-side critical section to another.
+one read-side critical section to another.  It also assumes recursive
+reader-writer locks:  If you try this with non-recursive locks, and
+you allow nested rcu_read_lock() calls, you can deadlock.
 
 However, it is probably the easiest implementation to relate to, so is
 a good starting point.
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622344 — [PATCH tip/core/rcu 7/9] doc: Update the comparisons rule in rcu_dereference.txt

From: Michalis Kokologiannakis <mixaskok@gmail.com>

When an RCU-protected pointer is fetched but never dereferenced
rcu_access_pointer() should be used in place of rcu_dereference().
This commit explicitly records this very fact in Documentation/
RCU/rcu_dereference.txt, in order to prevent the usage of
rcu_dereference() in comparisons.

Signed-off-by: Michalis Kokologiannakis <mixaskok@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/rcu_dereference.txt | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Documentation/RCU/rcu_dereference.txt b/Documentation/RCU/rcu_dereference.txt
index c0bf2441a2ba..b2a613f16d74 100644
--- a/Documentation/RCU/rcu_dereference.txt
+++ b/Documentation/RCU/rcu_dereference.txt
@@ -138,6 +138,15 @@ o	Be very careful about comparing pointers obtained from
 		This sort of comparison occurs frequently when scanning
 		RCU-protected circular linked lists.
 
+		Note that if checks for being within an RCU read-side
+		critical section are not required and the pointer is never
+		dereferenced, rcu_access_pointer() should be used in place
+		of rcu_dereference(). The rcu_access_pointer() primitive
+		does not require an enclosing read-side critical section,
+		and also omits the smp_read_barrier_depends() included in
+		rcu_dereference(), which in turn should provide a small
+		performance gain in some CPUs (e.g., the DEC Alpha).
+
 	o	The comparison is against a pointer that references memory
 		that was initialized "a long time ago."  The reason
 		this is safe is that even if misordering occurs, the
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622345 — [PATCH tip/core/rcu 3/9] doc: Update stallwarn.txt to make causes more prominent

This commit rearranges the Documentation/RCU/stallwarn.txt file to
put the list of issues that can cause RCU CPU stall warnings near
the beginning of the document.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/stallwarn.txt | 190 +++++++++++++++++++++-------------------
 1 file changed, 100 insertions(+), 90 deletions(-)

diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt
index e93d04133fe7..96a3d81837e1 100644
--- a/Documentation/RCU/stallwarn.txt
+++ b/Documentation/RCU/stallwarn.txt
@@ -1,9 +1,102 @@
 Using RCU's CPU Stall Detector
 
-The rcu_cpu_stall_suppress module parameter enables RCU's CPU stall
-detector, which detects conditions that unduly delay RCU grace periods.
-This module parameter enables CPU stall detection by default, but
-may be overridden via boot-time parameter or at runtime via sysfs.
+This document first discusses what sorts of issues RCU's CPU stall
+detector can locate, and then discusses kernel parameters and Kconfig
+options that can be used to fine-tune the detector's operation.  Finally,
+this document explains the stall detector's "splat" format.
+
+
+What Causes RCU CPU Stall Warnings?
+
+So your kernel printed an RCU CPU stall warning.  The next question is
+"What caused it?"  The following problems can result in RCU CPU stall
+warnings:
+
+o	A CPU looping in an RCU read-side critical section.
+
+o	A CPU looping with interrupts disabled.
+
+o	A CPU looping with preemption disabled.  This condition can
+	result in RCU-sched stalls and, if ksoftirqd is in use, RCU-bh
+	stalls.
+
+o	A CPU looping with bottom halves disabled.  This condition can
+	result in RCU-sched and RCU-bh stalls.
+
+o	For !CONFIG_PREEMPT kernels, a CPU looping anywhere in the
+	kernel without invoking schedule().  Note that cond_resched()
+	does not necessarily prevent RCU CPU stall warnings.  Therefore,
+	if the looping in the kernel is really expected and desirable
+	behavior, you might need to replace some of the cond_resched()
+	calls with calls to cond_resched_rcu_qs().
+
+o	Booting Linux using a console connection that is too slow to
+	keep up with the boot-time console-message rate.  For example,
+	a 115Kbaud serial console can be -way- too slow to keep up
+	with boot-time message rates, and will frequently result in
+	RCU CPU stall warning messages.  Especially if you have added
+	debug printk()s.
+
+o	Anything that prevents RCU's grace-period kthreads from running.
+	This can result in the "All QSes seen" console-log message.
+	This message will include information on when the kthread last
+	ran and how often it should be expected to run.
+
+o	A CPU-bound real-time task in a CONFIG_PREEMPT kernel, which might
+	happen to preempt a low-priority task in the middle of an RCU
+	read-side critical section.   This is especially damaging if
+	that low-priority task is not permitted to run on any other CPU,
+	in which case the next RCU grace period can never complete, which
+	will eventually cause the system to run out of memory and hang.
+	While the system is in the process of running itself out of
+	memory, you might see stall-warning messages.
+
+o	A CPU-bound real-time task in a CONFIG_PREEMPT_RT kernel that
+	is running at a higher priority than the RCU softirq threads.
+	This will prevent RCU callbacks from ever being invoked,
+	and in a CONFIG_PREEMPT_RCU kernel will further prevent
+	RCU grace periods from ever completing.  Either way, the
+	system will eventually run out of memory and hang.  In the
+	CONFIG_PREEMPT_RCU case, you might see stall-warning
+	messages.
+
+o	A hardware or software issue shuts off the scheduler-clock
+	interrupt on a CPU that is not in dyntick-idle mode.  This
+	problem really has happened, and seems to be most likely to
+	result in RCU CPU stall warnings for CONFIG_NO_HZ_COMMON=n kernels.
+
+o	A bug in the RCU implementation.
+
+o	A hardware failure.  This is quite unlikely, but has occurred
+	at least once in real life.  A CPU failed in a running system,
+	becoming unresponsive, but not causing an immediate crash.
+	This resulted in a series of RCU CPU stall warnings, eventually
+	leading the realization that the CPU had failed.
+
+The RCU, RCU-sched, RCU-bh, and RCU-tasks implementations have CPU stall
+warning.  Note that SRCU does -not- have CPU stall warnings.  Please note
+that RCU only detects CPU stalls when there is a grace period in progress.
+No grace period, no CPU stall warnings.
+
+To diagnose the cause of the stall, inspect the stack traces.
+The offending function will usually be near the top of the stack.
+If you have a series of stall warnings from a single extended stall,
+comparing the stack traces can often help determine where the stall
+is occurring, which will usually be in the function nearest the top of
+that portion of the stack which remains the same from trace to trace.
+If you can reliably trigger the stall, ftrace can be quite helpful.
+
+RCU bugs can often be debugged with the help of CONFIG_RCU_TRACE
+and with RCU's event tracing.  For information on RCU's event tracing,
+see include/trace/events/rcu.h.
+
+
+Fine-Tuning the RCU CPU Stall Detector
+
+The rcuupdate.rcu_cpu_stall_suppress module parameter disables RCU's
+CPU stall detector, which detects conditions that unduly delay RCU grace
+periods.  This module parameter enables CPU stall detection by default,
+but may be overridden via boot-time parameter or at runtime via sysfs.
 The stall detector's idea of what constitutes "unduly delayed" is
 controlled by a set of kernel configuration variables and cpp macros:
 
@@ -56,6 +149,9 @@ rcupdate.rcu_task_stall_timeout
 	And continues with the output of sched_show_task() for each
 	task stalling the current RCU-tasks grace period.
 
+
+Interpreting RCU's CPU Stall-Detector "Splats"
+
 For non-RCU-tasks flavors of RCU, when a CPU detects that it is stalling,
 it will print a message similar to the following:
 
@@ -178,89 +274,3 @@ grace period is in flight.
 
 It is entirely possible to see stall warnings from normal and from
 expedited grace periods at about the same time from the same run.
-
-
-What Causes RCU CPU Stall Warnings?
-
-So your kernel printed an RCU CPU stall warning.  The next question is
-"What caused it?"  The following problems can result in RCU CPU stall
-warnings:
-
-o	A CPU looping in an RCU read-side critical section.
-	
-o	A CPU looping with interrupts disabled.  This condition can
-	result in RCU-sched and RCU-bh stalls.
-
-o	A CPU looping with preemption disabled.  This condition can
-	result in RCU-sched stalls and, if ksoftirqd is in use, RCU-bh
-	stalls.
-
-o	A CPU looping with bottom halves disabled.  This condition can
-	result in RCU-sched and RCU-bh stalls.
-
-o	For !CONFIG_PREEMPT kernels, a CPU looping anywhere in the
-	kernel without invoking schedule().  Note that cond_resched()
-	does not necessarily prevent RCU CPU stall warnings.  Therefore,
-	if the looping in the kernel is really expected and desirable
-	behavior, you might need to replace some of the cond_resched()
-	calls with calls to cond_resched_rcu_qs().
-
-o	Booting Linux using a console connection that is too slow to
-	keep up with the boot-time console-message rate.  For example,
-	a 115Kbaud serial console can be -way- too slow to keep up
-	with boot-time message rates, and will frequently result in
-	RCU CPU stall warning messages.  Especially if you have added
-	debug printk()s.
-
-o	Anything that prevents RCU's grace-period kthreads from running.
-	This can result in the "All QSes seen" console-log message.
-	This message will include information on when the kthread last
-	ran and how often it should be expected to run.
-
-o	A CPU-bound real-time task in a CONFIG_PREEMPT kernel, which might
-	happen to preempt a low-priority task in the middle of an RCU
-	read-side critical section.   This is especially damaging if
-	that low-priority task is not permitted to run on any other CPU,
-	in which case the next RCU grace period can never complete, which
-	will eventually cause the system to run out of memory and hang.
-	While the system is in the process of running itself out of
-	memory, you might see stall-warning messages.
-
-o	A CPU-bound real-time task in a CONFIG_PREEMPT_RT kernel that
-	is running at a higher priority than the RCU softirq threads.
-	This will prevent RCU callbacks from ever being invoked,
-	and in a CONFIG_PREEMPT_RCU kernel will further prevent
-	RCU grace periods from ever completing.  Either way, the
-	system will eventually run out of memory and hang.  In the
-	CONFIG_PREEMPT_RCU case, you might see stall-warning
-	messages.
-
-o	A hardware or software issue shuts off the scheduler-clock
-	interrupt on a CPU that is not in dyntick-idle mode.  This
-	problem really has happened, and seems to be most likely to
-	result in RCU CPU stall warnings for CONFIG_NO_HZ_COMMON=n kernels.
-
-o	A bug in the RCU implementation.
-
-o	A hardware failure.  This is quite unlikely, but has occurred
-	at least once in real life.  A CPU failed in a running system,
-	becoming unresponsive, but not causing an immediate crash.
-	This resulted in a series of RCU CPU stall warnings, eventually
-	leading the realization that the CPU had failed.
-
-The RCU, RCU-sched, RCU-bh, and RCU-tasks implementations have CPU stall
-warning.  Note that SRCU does -not- have CPU stall warnings.  Please note
-that RCU only detects CPU stalls when there is a grace period in progress.
-No grace period, no CPU stall warnings.
-
-To diagnose the cause of the stall, inspect the stack traces.
-The offending function will usually be near the top of the stack.
-If you have a series of stall warnings from a single extended stall,
-comparing the stack traces can often help determine where the stall
-is occurring, which will usually be in the function nearest the top of
-that portion of the stack which remains the same from trace to trace.
-If you can reliably trigger the stall, ftrace can be quite helpful.
-
-RCU bugs can often be debugged with the help of CONFIG_RCU_TRACE
-and with RCU's event tracing.  For information on RCU's event tracing,
-see include/trace/events/rcu.h.
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622346 — [PATCH tip/core/rcu 5/9] doc: Update requirements based on recent changes

These changes include lighter-weight expedited grace periods, the fact
that expedited grace periods and rcu_barrier() no longer block CPU
hotplug, some HTML font fixups, noting that rcu_barrier() need not wait
for a grace period (even if callbacks are posted), the fact that SRCU
read-side critical sections can be used from offline CPUs, and the fact
that SRCU now maintains per-CPU callback lists.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 .../RCU/Design/Requirements/Requirements.html      | 120 ++++++++++++++++-----
 1 file changed, 94 insertions(+), 26 deletions(-)

diff --git a/Documentation/RCU/Design/Requirements/Requirements.html b/Documentation/RCU/Design/Requirements/Requirements.html
index 999b3ed3444e..f60adf112663 100644
--- a/Documentation/RCU/Design/Requirements/Requirements.html
+++ b/Documentation/RCU/Design/Requirements/Requirements.html
@@ -659,8 +659,9 @@ systems with more than one CPU:
 	In other words, a given instance of <tt>synchronize_rcu()</tt>
 	can avoid waiting on a given RCU read-side critical section only
 	if it can prove that <tt>synchronize_rcu()</tt> started first.
+	</font>
 
-	<p>
+	<p><font color="ffffff">
 	A related question is &ldquo;When <tt>rcu_read_lock()</tt>
 	doesn't generate any code, why does it matter how it relates
 	to a grace period?&rdquo;
@@ -675,8 +676,9 @@ systems with more than one CPU:
 	within the critical section, in which case none of the accesses
 	within the critical section may observe the effects of any
 	access following the grace period.
+	</font>
 
-	<p>
+	<p><font color="ffffff">
 	As of late 2016, mathematical models of RCU take this
 	viewpoint, for example, see slides&nbsp;62 and&nbsp;63
 	of the
@@ -1616,8 +1618,8 @@ CPUs should at least make reasonable forward progress.
 In return for its shorter latencies, <tt>synchronize_rcu_expedited()</tt>
 is permitted to impose modest degradation of real-time latency
 on non-idle online CPUs.
-That said, it will likely be necessary to take further steps to reduce this
-degradation, hopefully to roughly that of a scheduling-clock interrupt.
+Here, &ldquo;modest&rdquo; means roughly the same latency
+degradation as a scheduling-clock interrupt.
 
 <p>
 There are a number of situations where even
@@ -1913,12 +1915,9 @@ This requirement is another factor driving batching of grace periods,
 but it is also the driving force behind the checks for large numbers
 of queued RCU callbacks in the <tt>call_rcu()</tt> code path.
 Finally, high update rates should not delay RCU read-side critical
-sections, although some read-side delays can occur when using
+sections, although some small read-side delays can occur when using
 <tt>synchronize_rcu_expedited()</tt>, courtesy of this function's use
-of <tt>try_stop_cpus()</tt>.
-(In the future, <tt>synchronize_rcu_expedited()</tt> will be
-converted to use lighter-weight inter-processor interrupts (IPIs),
-but this will still disturb readers, though to a much smaller degree.)
+of <tt>smp_call_function_single()</tt>.
 
 <p>
 Although all three of these corner cases were understood in the early
@@ -2192,7 +2191,7 @@ Unfortunately, <tt>synchronize_rcu()</tt> can't do this until all of
 its kthreads are spawned, which doesn't happen until some time during
 <tt>early_initcalls()</tt> time.
 But this is no excuse:  RCU is nevertheless required to correctly handle
-synchronous grace periods during this time period, which it currently does.
+synchronous grace periods during this time period.
 Once all of its kthreads are up and running, RCU starts running
 normally.
 
@@ -2206,8 +2205,10 @@ normally.
 <tr><th align="left">Answer:</th></tr>
 <tr><td bgcolor="#ffffff"><font color="ffffff">
 	Very carefully!
+	</font>
 
-	<p>During the &ldquo;dead zone&rdquo; between the time that the
+	<p><font color="ffffff">
+	During the &ldquo;dead zone&rdquo; between the time that the
 	scheduler spawns the first task and the time that all of RCU's
 	kthreads have been spawned, all synchronous grace periods are
 	handled by the expedited grace-period mechanism.
@@ -2220,8 +2221,10 @@ normally.
 	using workqueues, as is required to avoid problems that would
 	otherwise occur when a user task received a POSIX signal while
 	driving an expedited grace period.
+	</font>
 
-	<p>And yes, this does mean that it is unhelpful to send POSIX
+	<p><font color="ffffff">
+	And yes, this does mean that it is unhelpful to send POSIX
 	signals to random tasks between the time that the scheduler
 	spawns its first kthread and the time that RCU's kthreads
 	have all been spawned.
@@ -2308,12 +2311,61 @@ situation, and Dipankar Sarma incorporated <tt>rcu_barrier()</tt> into RCU.
 The need for <tt>rcu_barrier()</tt> for module unloading became
 apparent later.
 
+<p>
+<b>Important note</b>: The <tt>rcu_barrier()</tt> function is not,
+repeat, <i>not</i>, obligated to wait for a grace period.
+It is instead only required to wait for RCU callbacks that have
+already been posted.
+Therefore, if there are no RCU callbacks posted anywhere in the system,
+<tt>rcu_barrier()</tt> is within its rights to return immediately.
+Even if there are callbacks posted, <tt>rcu_barrier()</tt> does not
+necessarily need to wait for a grace period.
+
+<table>
+<tr><th>&nbsp;</th></tr>
+<tr><th align="left">Quick Quiz:</th></tr>
+<tr><td>
+	Wait a minute!
+	Each RCU callbacks must wait for a grace period to complete,
+	and <tt>rcu_barrier()</tt> must wait for each pre-existing
+	callback to be invoked.
+	Doesn't <tt>rcu_barrier()</tt> therefore need to wait for
+	a full grace period if there is even one callback posted anywhere
+	in the system?
+</td></tr>
+<tr><th align="left">Answer:</th></tr>
+<tr><td bgcolor="#ffffff"><font color="ffffff">
+	Absolutely not!!!
+	</font>
+
+	<p><font color="ffffff">
+	Yes, each RCU callbacks must wait for a grace period to complete,
+	but it might well be partly (or even completely) finished waiting
+	by the time <tt>rcu_barrier()</tt> is invoked.
+	In that case, <tt>rcu_barrier()</tt> need only wait for the
+	remaining portion of the grace period to elapse.
+	So even if there are quite a few callbacks posted,
+	<tt>rcu_barrier()</tt> might well return quite quickly.
+	</font>
+
+	<p><font color="ffffff">
+	So if you need to wait for a grace period as well as for all
+	pre-existing callbacks, you will need to invoke both
+	<tt>synchronize_rcu()</tt> and <tt>rcu_barrier()</tt>.
+	If latency is a concern, you can always use workqueues
+	to invoke them concurrently.
+</font></td></tr>
+<tr><td>&nbsp;</td></tr>
+</table>
+
 <h3><a name="Hotplug CPU">Hotplug CPU</a></h3>
 
 <p>
 The Linux kernel supports CPU hotplug, which means that CPUs
 can come and go.
-It is of course illegal to use any RCU API member from an offline CPU.
+It is of course illegal to use any RCU API member from an offline CPU,
+with the exception of <a href="#Sleepable RCU">SRCU</a> read-side
+critical sections.
 This requirement was present from day one in DYNIX/ptx, but
 on the other hand, the Linux kernel's CPU-hotplug implementation
 is &ldquo;interesting.&rdquo;
@@ -2323,19 +2375,18 @@ The Linux-kernel CPU-hotplug implementation has notifiers that
 are used to allow the various kernel subsystems (including RCU)
 to respond appropriately to a given CPU-hotplug operation.
 Most RCU operations may be invoked from CPU-hotplug notifiers,
-including even normal synchronous grace-period operations
-such as <tt>synchronize_rcu()</tt>.
-However, expedited grace-period operations such as
-<tt>synchronize_rcu_expedited()</tt> are not supported,
-due to the fact that current implementations block CPU-hotplug
-operations, which could result in deadlock.
+including even synchronous grace-period operations such as
+<tt>synchronize_rcu()</tt> and <tt>synchronize_rcu_expedited()</tt>.
 
 <p>
-In addition, all-callback-wait operations such as
+However, all-callback-wait operations such as
 <tt>rcu_barrier()</tt> are also not supported, due to the
 fact that there are phases of CPU-hotplug operations where
 the outgoing CPU's callbacks will not be invoked until after
 the CPU-hotplug operation ends, which could also result in deadlock.
+Furthermore, <tt>rcu_barrier()</tt> blocks CPU-hotplug operations
+during its execution, which results in another type of deadlock
+when invoked from a CPU-hotplug notifier.
 
 <h3><a name="Scheduler and RCU">Scheduler and RCU</a></h3>
 
@@ -2877,6 +2928,27 @@ API, which, in combination with <tt>srcu_read_unlock()</tt>,
 guarantees a full memory barrier.
 
 <p>
+Also unlike other RCU flavors, SRCU's callbacks-wait function
+<tt>srcu_barrier()</tt> may be invoked from CPU-hotplug notifiers,
+though this is not necessarily a good idea.
+The reason that this is possible is that SRCU is insensitive
+to whether or not a CPU is online, which means that <tt>srcu_barrier()</tt>
+need not exclude CPU-hotplug operations.
+
+<p>
+As of v4.12, SRCU's callbacks are maintained per-CPU, eliminating
+a locking bottleneck present in prior kernel versions.
+Although this will allow users to put much heavier stress on
+<tt>call_srcu()</tt>, it is important to note that SRCU does not
+yet take any special steps to deal with callback flooding.
+So if you are posting (say) 10,000 SRCU callbacks per second per CPU,
+you are probably totally OK, but if you intend to post (say) 1,000,000
+SRCU callbacks per second per CPU, please run some tests first.
+SRCU just might need a few adjustment to deal with that sort of load.
+Of course, your mileage may vary based on the speed of your CPUs and
+the size of your memory.
+
+<p>
 The
 <a href="https://lwn.net/Articles/609973/#RCU Per-Flavor API Table">SRCU API</a>
 includes
@@ -3034,8 +3106,8 @@ to do some redesign to avoid this scalability problem.
 
 <p>
 RCU disables CPU hotplug in a few places, perhaps most notably in the
-expedited grace-period and <tt>rcu_barrier()</tt> operations.
-If there is a strong reason to use expedited grace periods in CPU-hotplug
+<tt>rcu_barrier()</tt> operations.
+If there is a strong reason to use <tt>rcu_barrier()</tt> in CPU-hotplug
 notifiers, it will be necessary to avoid disabling CPU hotplug.
 This would introduce some complexity, so there had better be a <i>very</i>
 good reason.
@@ -3109,9 +3181,5 @@ Andy Lutomirski for their help in rendering
 this article human readable, and to Michelle Rankin for her support
 of this effort.
 Other contributions are acknowledged in the Linux kernel's git archive.
-The cartoon is copyright (c) 2013 by Melissa Broussard,
-and is provided
-under the terms of the Creative Commons Attribution-Share Alike 3.0
-United States license.
 
 </body></html>
-- 
2.5.2

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web