Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1622352 > unrolled thread

[PATCH tip/core/rcu 0/13] Miscellaneous fixes for 4.12

Back to article view | Back to linux.kernel

Contents

  [PATCH tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
      Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 11:20 +0200
        Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 18:10 +0200
          Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 18:30 +0200
            Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 19:00 +0200
              Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 19:10 +0200
                Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 19:40 +0200
                  Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 19:50 +0200
                    Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 20:20 +0200
                      Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 20:30 +0200
                        Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 20:50 +0200
                          Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-19 15:30 +0200
                            Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-19 15:50 +0200
                              Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 17:10 +0200
                                Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-19 17:50 +0200
                                  Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:20 +0200
                            Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 17:00 +0200
                  Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick Peter Zijlstra <peterz@infradead.org> - 2017-04-13 20:40 +0200
                    Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text  more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 21:50 +0200
    [PATCH tip/core/rcu 13/13] rcu: Fix typo in PER_RCU_NODE_PERIOD header comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 09/13] mm: Use static initialization for "srcu" "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 12/13] rcu: Use true/false in assignment to bool "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 05/13] rcu: Remove obsolete comment from rcu_future_gp_cleanup() header "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 03/13] types: Update obsolete callback_head comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 11/13] rcu: Use bool value directly "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 10/13] torture: Use correct path for Kconfig fragment for duplicates "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 02/13] lockdep: Use "WARNING" tag on lockdep splats "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
      Re: [PATCH tip/core/rcu 02/13] lockdep: Use "WARNING" tag on lockdep  splats Peter Zijlstra <peterz@infradead.org> - 2017-04-13 11:20 +0200
        Re: [PATCH tip/core/rcu 02/13] lockdep: Use "WARNING" tag on lockdep  splats "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 18:10 +0200
    [PATCH tip/core/rcu 06/13] hlist_add_tail_rcu disable sparse warning "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:00 +0200
    [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-12 19:10 +0200
      Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU Peter Zijlstra <peterz@infradead.org> - 2017-04-13 11:20 +0200
        Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU Vlastimil Babka <vbabka@suse.cz> - 2017-04-13 13:10 +0200
          Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 18:10 +0200
          Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU Peter Zijlstra <peterz@infradead.org> - 2017-04-13 18:20 +0200
            Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-13 19:30 +0200
            Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU Eric Dumazet <edumazet@google.com> - 2017-04-13 23:40 +0200
              Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU Peter Zijlstra <peterz@infradead.org> - 2017-04-14 10:50 +0200
                Re: [PATCH tip/core/rcu 01/13] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-14 15:50 +0200
    [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:30 +0200
      [PATCH v2 tip/core/rcu 11/11] rcu: Fix typo in PER_RCU_NODE_PERIOD header comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:30 +0200
      [PATCH v2 tip/core/rcu 03/11] types: Update obsolete callback_head comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:30 +0200
      [PATCH v2 tip/core/rcu 06/11] hlist_add_tail_rcu disable sparse warning "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:30 +0200
      [PATCH v2 tip/core/rcu 04/11] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
        Re: [PATCH v2 tip/core/rcu 04/11] rcu: Make RCU_FANOUT_LEAF help  text more explicit about skew_tick Josh Triplett <josh@joshtriplett.org> - 2017-04-18 02:20 +0200
          Re: [PATCH v2 tip/core/rcu 04/11] rcu: Make RCU_FANOUT_LEAF help  text more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 20:50 +0200
      [PATCH v2 tip/core/rcu 05/11] rcu: Remove obsolete comment from rcu_future_gp_cleanup() header "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
      [PATCH v2 tip/core/rcu 08/11] torture: Use correct path for Kconfig fragment for duplicates "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
      [PATCH v2 tip/core/rcu 09/11] rcu: Use bool value directly "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
      [PATCH v2 tip/core/rcu 10/11] rcu: Use true/false in assignment to bool "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
      [PATCH v2 tip/core/rcu 01/11] mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
        Re: [PATCH v2 tip/core/rcu 01/11] mm: Rename SLAB_DESTROY_BY_RCU to  SLAB_TYPESAFE_BY_RCU David Rientjes <rientjes@google.com> - 2017-04-18 02:20 +0200
      [PATCH v2 tip/core/rcu 02/11] lockdep: Use "WARNING" tag on lockdep splats "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
        Re: [PATCH v2 tip/core/rcu 02/11] lockdep: Use "WARNING" tag on  lockdep splats Josh Triplett <josh@joshtriplett.org> - 2017-04-19 17:10 +0200
          Re: [PATCH v2 tip/core/rcu 02/11] lockdep: Use "WARNING" tag on  lockdep splats "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:30 +0200
      [PATCH v2 tip/core/rcu 07/11] rcu: Improve comments for hotplug/suspend/hibernate functions "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-18 01:40 +0200
      Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 13:30 +0200
        Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 13:40 +0200
        Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Christian Borntraeger <borntraeger@de.ibm.com> - 2017-04-19 13:50 +0200
          Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 14:10 +0200
            Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Marc Zyngier <marc.zyngier@arm.com> - 2017-04-19 15:00 +0200
            Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 16:50 +0200
              Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Josh Triplett <josh@joshtriplett.org> - 2017-04-19 17:00 +0200
                Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 17:10 +0200
                  Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 17:20 +0200
              Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 17:00 +0200
                Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 17:20 +0200
          Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 15:30 +0200
            Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Christian Borntraeger <borntraeger@de.ibm.com> - 2017-04-19 15:30 +0200
        Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 15:10 +0200
          Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 15:20 +0200
            Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 17:40 +0200
              Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Peter Zijlstra <peterz@infradead.org> - 2017-04-19 17:50 +0200
                Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:20 +0200
      Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 06/11] hlist_add_tail_rcu disable sparse warning "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 11/11] rcu: Fix typo in PER_RCU_NODE_PERIOD header comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 05/11] rcu: Remove obsolete comment from rcu_future_gp_cleanup() header "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 08/11] torture: Use correct path for Kconfig fragment for duplicates "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 03/11] types: Update obsolete callback_head comment "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 10/11] rcu: Use true/false in assignment to bool "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 07/11] rcu: Improve comments for hotplug/suspend/hibernate functions "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 18:50 +0200
        [PATCH v3 tip/core/rcu 04/11] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:00 +0200
        [PATCH v3 tip/core/rcu 01/11] mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2017-04-19 19:00 +0200

Page 1 of 5  [1] 2 3 4 5  Next page →

#1622352 — [PATCH tip/core/rcu 0/13] Miscellaneous fixes for 4.12

Hello!

This series contains the following fixes:

1.	Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU.

2.	Use "WARNING" tag on RCU's lockdep splats.

3.	Update obsolete callback_head comment.

4.	Make RCU_FANOUT_LEAF help text more explicit about skew_tick.

5.	Remove obsolete comment from rcu_future_gp_cleanup() header.

6.	Disable sparse warning emitted by hlist_add_tail_rcu(), courtesy
	of Michael S. Tsirkin.

7.	Add smp_mb__after_atomic() to sync_exp_work_done().

8.	Improve comments for hotplug/suspend/hibernate functions.

9.	Use static initialization for "srcu" in mm/mmu_notifier.c.

10.	Use correct path for Kconfig fragment for duplicate rcutorture
	test scenarios.

11.	Use bool value directly for ->beenonline comparison, courtesy
	of Nicholas Mc Guire.

12.	Use true/false in assignment to bool variable rcu_nocb_poll,
	courtesy of Nicholas Mc Guire.

13.	Fix typo in PER_RCU_NODE_PERIOD header comment.

							Thanx, Paul

------------------------------------------------------------------------

 Documentation/RCU/00-INDEX                               |    2 
 Documentation/RCU/rculist_nulls.txt                      |    6 -
 Documentation/RCU/whatisRCU.txt                          |    3 
 drivers/gpu/drm/i915/i915_gem.c                          |    2 
 drivers/gpu/drm/i915/i915_gem_request.h                  |    2 
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c          |    2 
 fs/jbd2/journal.c                                        |    2 
 fs/signalfd.c                                            |    2 
 include/linux/dma-fence.h                                |    4 
 include/linux/rculist.h                                  |    3 
 include/linux/slab.h                                     |    6 -
 include/linux/types.h                                    |    2 
 include/net/sock.h                                       |    2 
 init/Kconfig                                             |   10 +
 kernel/fork.c                                            |    4 
 kernel/locking/lockdep.c                                 |   86 +++++++--------
 kernel/locking/rtmutex-debug.c                           |    9 -
 kernel/rcu/tree.c                                        |   49 ++++++--
 kernel/rcu/tree_exp.h                                    |    1 
 kernel/rcu/tree_plugin.h                                 |    2 
 kernel/signal.c                                          |    2 
 mm/kasan/kasan.c                                         |    6 -
 mm/kmemcheck.c                                           |    2 
 mm/mmu_notifier.c                                        |   14 --
 mm/rmap.c                                                |    4 
 mm/slab.c                                                |    6 -
 mm/slab.h                                                |    4 
 mm/slab_common.c                                         |    6 -
 mm/slob.c                                                |    6 -
 mm/slub.c                                                |   12 +-
 net/dccp/ipv4.c                                          |    2 
 net/dccp/ipv6.c                                          |    2 
 net/ipv4/tcp_ipv4.c                                      |    2 
 net/ipv6/tcp_ipv6.c                                      |    2 
 net/llc/af_llc.c                                         |    2 
 net/llc/llc_conn.c                                       |    4 
 net/llc/llc_sap.c                                        |    2 
 net/netfilter/nf_conntrack_core.c                        |    8 -
 net/smc/af_smc.c                                         |    2 
 tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh |    2 
 40 files changed, 160 insertions(+), 129 deletions(-)

[toc] | [next] | [standalone]

#1622353 — [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

If you set RCU_FANOUT_LEAF too high, you can get lock contention
on the leaf rcu_node, and you should boot with the skew_tick kernel
parameter set in order to avoid this lock contention.  This commit
therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
this.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 init/Kconfig | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index a92f27da4a27..946e561e67b7 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
 	  initialization.  These systems tend to run CPU-bound, and thus
 	  are not helped by synchronized interrupts, and thus tend to
 	  skew them, which reduces lock contention enough that large
-	  leaf-level fanouts work well.
+	  leaf-level fanouts work well.  That said, setting leaf-level
+	  fanout to a large number will likely cause problematic
+	  lock contention on the leaf-level rcu_node structures unless
+	  you boot with the skew_tick kernel parameter.
 
 	  Select a specific number if testing RCU itself.
 
-	  Select the maximum permissible value for large systems.
+	  Select the maximum permissible value for large systems, but
+	  	please understand that you may also need to set the
+		skew_tick kernel boot parameter to avoid contention
+		on the rcu_node structure's locks.
 
 	  Take the default if unsure.
 
-- 
2.5.2

[toc] | [prev] | [next] | [standalone]

#1622838 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 12, 2017 at 09:55:40AM -0700, Paul E. McKenney wrote:
> If you set RCU_FANOUT_LEAF too high, you can get lock contention
> on the leaf rcu_node, and you should boot with the skew_tick kernel
> parameter set in order to avoid this lock contention.  This commit
> therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
> this.
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
>  init/Kconfig | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index a92f27da4a27..946e561e67b7 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
>  	  initialization.  These systems tend to run CPU-bound, and thus
>  	  are not helped by synchronized interrupts, and thus tend to
>  	  skew them, which reduces lock contention enough that large
> -	  leaf-level fanouts work well.
> +	  leaf-level fanouts work well.  That said, setting leaf-level
> +	  fanout to a large number will likely cause problematic
> +	  lock contention on the leaf-level rcu_node structures unless
> +	  you boot with the skew_tick kernel parameter.

Why mention a way out of a problem you shouldn't have to begin with?

Just state its bad and result in lock contention and leave it at that.

[toc] | [prev] | [next] | [standalone]

#1623149 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 11:15:35AM +0200, Peter Zijlstra wrote:
> On Wed, Apr 12, 2017 at 09:55:40AM -0700, Paul E. McKenney wrote:
> > If you set RCU_FANOUT_LEAF too high, you can get lock contention
> > on the leaf rcu_node, and you should boot with the skew_tick kernel
> > parameter set in order to avoid this lock contention.  This commit
> > therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
> > this.
> > 
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > ---
> >  init/Kconfig | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> > 
> > diff --git a/init/Kconfig b/init/Kconfig
> > index a92f27da4a27..946e561e67b7 100644
> > --- a/init/Kconfig
> > +++ b/init/Kconfig
> > @@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
> >  	  initialization.  These systems tend to run CPU-bound, and thus
> >  	  are not helped by synchronized interrupts, and thus tend to
> >  	  skew them, which reduces lock contention enough that large
> > -	  leaf-level fanouts work well.
> > +	  leaf-level fanouts work well.  That said, setting leaf-level
> > +	  fanout to a large number will likely cause problematic
> > +	  lock contention on the leaf-level rcu_node structures unless
> > +	  you boot with the skew_tick kernel parameter.
> 
> Why mention a way out of a problem you shouldn't have to begin with?
> 
> Just state its bad and result in lock contention and leave it at that.

To avoid people tuning huge machines having to wait for me to give
them an answer as to why they are suffering lock contention after
cranking up the value of RCU_FANOUT_LEAF.

Or am I missing your point?

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1623165 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 09:03:33AM -0700, Paul E. McKenney wrote:
> On Thu, Apr 13, 2017 at 11:15:35AM +0200, Peter Zijlstra wrote:
> > On Wed, Apr 12, 2017 at 09:55:40AM -0700, Paul E. McKenney wrote:
> > > If you set RCU_FANOUT_LEAF too high, you can get lock contention
> > > on the leaf rcu_node, and you should boot with the skew_tick kernel
> > > parameter set in order to avoid this lock contention.  This commit
> > > therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
> > > this.
> > > 
> > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > > ---
> > >  init/Kconfig | 10 ++++++++--
> > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/init/Kconfig b/init/Kconfig
> > > index a92f27da4a27..946e561e67b7 100644
> > > --- a/init/Kconfig
> > > +++ b/init/Kconfig
> > > @@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
> > >  	  initialization.  These systems tend to run CPU-bound, and thus
> > >  	  are not helped by synchronized interrupts, and thus tend to
> > >  	  skew them, which reduces lock contention enough that large
> > > -	  leaf-level fanouts work well.
> > > +	  leaf-level fanouts work well.  That said, setting leaf-level
> > > +	  fanout to a large number will likely cause problematic
> > > +	  lock contention on the leaf-level rcu_node structures unless
> > > +	  you boot with the skew_tick kernel parameter.
> > 
> > Why mention a way out of a problem you shouldn't have to begin with?
> > 
> > Just state its bad and result in lock contention and leave it at that.
> 
> To avoid people tuning huge machines having to wait for me to give
> them an answer as to why they are suffering lock contention after
> cranking up the value of RCU_FANOUT_LEAF.
> 
> Or am I missing your point?

Your answer should be: don't do that then. Not provide them a shady work
around.

tick skew isn't pretty and has other problems (there's a reason its not
on by default). You're then doing two things you shouldn't.

[toc] | [prev] | [next] | [standalone]

#1623183 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 06:19:48PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 09:03:33AM -0700, Paul E. McKenney wrote:
> > On Thu, Apr 13, 2017 at 11:15:35AM +0200, Peter Zijlstra wrote:
> > > On Wed, Apr 12, 2017 at 09:55:40AM -0700, Paul E. McKenney wrote:
> > > > If you set RCU_FANOUT_LEAF too high, you can get lock contention
> > > > on the leaf rcu_node, and you should boot with the skew_tick kernel
> > > > parameter set in order to avoid this lock contention.  This commit
> > > > therefore upgrades the RCU_FANOUT_LEAF help text to explicitly state
> > > > this.
> > > > 
> > > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > > > ---
> > > >  init/Kconfig | 10 ++++++++--
> > > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/init/Kconfig b/init/Kconfig
> > > > index a92f27da4a27..946e561e67b7 100644
> > > > --- a/init/Kconfig
> > > > +++ b/init/Kconfig
> > > > @@ -612,11 +612,17 @@ config RCU_FANOUT_LEAF
> > > >  	  initialization.  These systems tend to run CPU-bound, and thus
> > > >  	  are not helped by synchronized interrupts, and thus tend to
> > > >  	  skew them, which reduces lock contention enough that large
> > > > -	  leaf-level fanouts work well.
> > > > +	  leaf-level fanouts work well.  That said, setting leaf-level
> > > > +	  fanout to a large number will likely cause problematic
> > > > +	  lock contention on the leaf-level rcu_node structures unless
> > > > +	  you boot with the skew_tick kernel parameter.
> > > 
> > > Why mention a way out of a problem you shouldn't have to begin with?
> > > 
> > > Just state its bad and result in lock contention and leave it at that.
> > 
> > To avoid people tuning huge machines having to wait for me to give
> > them an answer as to why they are suffering lock contention after
> > cranking up the value of RCU_FANOUT_LEAF.
> > 
> > Or am I missing your point?
> 
> Your answer should be: don't do that then. Not provide them a shady work
> around.
> 
> tick skew isn't pretty and has other problems (there's a reason its not
> on by default). You're then doing two things you shouldn't.

The tick skew problem that I know of is energy efficiency for light
workloads.  This doesn't normally apply to the large heavily loaded
systems on which people skew ticks.

So what are the other problems?

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1623198 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 09:55:16AM -0700, Paul E. McKenney wrote:
> > > To avoid people tuning huge machines having to wait for me to give
> > > them an answer as to why they are suffering lock contention after
> > > cranking up the value of RCU_FANOUT_LEAF.

So is there a good reason to increase FANOUT_LEAF ?

> > > Or am I missing your point?
> > 
> > Your answer should be: don't do that then. Not provide them a shady work
> > around.
> > 
> > tick skew isn't pretty and has other problems (there's a reason its not
> > on by default). You're then doing two things you shouldn't.
> 
> The tick skew problem that I know of is energy efficiency for light
> workloads.  This doesn't normally apply to the large heavily loaded
> systems on which people skew ticks.
> 
> So what are the other problems?

If the jiffy updater bounces between CPUs (as is not uncommon) the
duration of the jiffy becomes an average (I think we fixed it where it
could go too fast by always jumping to a CPU which has a short jiffy,
but I'm not sure).

This further complicates some of the jiffy based loops (which arguably
should go away anyway).

And I have vague memories of it actually causing lock contention, but
I've forgotten how that worked.

[toc] | [prev] | [next] | [standalone]

#1623217 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 09:55:16AM -0700, Paul E. McKenney wrote:
> > > > To avoid people tuning huge machines having to wait for me to give
> > > > them an answer as to why they are suffering lock contention after
> > > > cranking up the value of RCU_FANOUT_LEAF.
> 
> So is there a good reason to increase FANOUT_LEAF ?

Increasing it reduces the number of rcu_node structures, and thus the
number of cache misses during grace-period initialization and cleanup.
This has proven necessary in the past on large machines having long
memory latencies.  And there are starting to be some pretty big machines
running in production, and even for typical commerical workloads.

> > > > Or am I missing your point?
> > > 
> > > Your answer should be: don't do that then. Not provide them a shady work
> > > around.
> > > 
> > > tick skew isn't pretty and has other problems (there's a reason its not
> > > on by default). You're then doing two things you shouldn't.
> > 
> > The tick skew problem that I know of is energy efficiency for light
> > workloads.  This doesn't normally apply to the large heavily loaded
> > systems on which people skew ticks.
> > 
> > So what are the other problems?
> 
> If the jiffy updater bounces between CPUs (as is not uncommon) the
> duration of the jiffy becomes an average (I think we fixed it where it
> could go too fast by always jumping to a CPU which has a short jiffy,
> but I'm not sure).
> 
> This further complicates some of the jiffy based loops (which arguably
> should go away anyway).

Fair enough.  And I am OK with jiffies going away as long as there
is a low-overhead rough-and-ready timing mechanism replacing it.

> And I have vague memories of it actually causing lock contention, but
> I've forgotten how that worked.

That is a new one on me.  I can easily see how not skewing ticks could
cause serious lock contention, but am missing how skewed ticks would
do so.

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1623222 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 10:31:00AM -0700, Paul E. McKenney wrote:
> On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 13, 2017 at 09:55:16AM -0700, Paul E. McKenney wrote:
> > > > > To avoid people tuning huge machines having to wait for me to give
> > > > > them an answer as to why they are suffering lock contention after
> > > > > cranking up the value of RCU_FANOUT_LEAF.
> > 
> > So is there a good reason to increase FANOUT_LEAF ?
> 
> Increasing it reduces the number of rcu_node structures, and thus the
> number of cache misses during grace-period initialization and cleanup.
> This has proven necessary in the past on large machines having long
> memory latencies.  And there are starting to be some pretty big machines
> running in production, and even for typical commerical workloads.

Is that perhaps a good moment to look at aligning the cpus in said nodes
with the cache topology?

[toc] | [prev] | [next] | [standalone]

#1623243 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 07:46:31PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 10:31:00AM -0700, Paul E. McKenney wrote:
> > On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote:
> > > On Thu, Apr 13, 2017 at 09:55:16AM -0700, Paul E. McKenney wrote:
> > > > > > To avoid people tuning huge machines having to wait for me to give
> > > > > > them an answer as to why they are suffering lock contention after
> > > > > > cranking up the value of RCU_FANOUT_LEAF.
> > > 
> > > So is there a good reason to increase FANOUT_LEAF ?
> > 
> > Increasing it reduces the number of rcu_node structures, and thus the
> > number of cache misses during grace-period initialization and cleanup.
> > This has proven necessary in the past on large machines having long
> > memory latencies.  And there are starting to be some pretty big machines
> > running in production, and even for typical commerical workloads.
> 
> Is that perhaps a good moment to look at aligning the cpus in said nodes
> with the cache topology?

We have been here before, haven't we?  Over and over again.  ;-)

As always...

First get me some system-level data showing that the current layout is
causing a real problem.  RCU's fastpath code doesn't come anywhere near
the rcu_node tree, so in the absence of such data, I of course remain
quite doubtful that there is a real need.  And painfully aware of the
required increase in complexity.

But if there is a real need demonstrated by real system-level data,
I will of course make the needed changes, as I have done many times in
the past in response to other requests.

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1623245 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 11:19:26AM -0700, Paul E. McKenney wrote:

> First get me some system-level data showing that the current layout is
> causing a real problem.  RCU's fastpath code doesn't come anywhere near
> the rcu_node tree, so in the absence of such data, I of course remain
> quite doubtful that there is a real need.  And painfully aware of the
> required increase in complexity.
> 
> But if there is a real need demonstrated by real system-level data,
> I will of course make the needed changes, as I have done many times in
> the past in response to other requests.

I read what you wrote here:

> > > Increasing it reduces the number of rcu_node structures, and thus the
> > > number of cache misses during grace-period initialization and cleanup.
> > > This has proven necessary in the past on large machines having long
> > > memory latencies.  And there are starting to be some pretty big machines
> > > running in production, and even for typical commerical workloads.

to mean you had exactly that pain. Or am I now totally not understanding
you?

[toc] | [prev] | [next] | [standalone]

#1623276 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 08:23:09PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 11:19:26AM -0700, Paul E. McKenney wrote:
> 
> > First get me some system-level data showing that the current layout is
> > causing a real problem.  RCU's fastpath code doesn't come anywhere near
> > the rcu_node tree, so in the absence of such data, I of course remain
> > quite doubtful that there is a real need.  And painfully aware of the
> > required increase in complexity.
> > 
> > But if there is a real need demonstrated by real system-level data,
> > I will of course make the needed changes, as I have done many times in
> > the past in response to other requests.
> 
> I read what you wrote here:
> 
> > > > Increasing it reduces the number of rcu_node structures, and thus the
> > > > number of cache misses during grace-period initialization and cleanup.
> > > > This has proven necessary in the past on large machines having long
> > > > memory latencies.  And there are starting to be some pretty big machines
> > > > running in production, and even for typical commerical workloads.
> 
> to mean you had exactly that pain. Or am I now totally not understanding
> you?

I believe that you are missing the fact that RCU grace-period
initialization and cleanup walks through the rcu_node tree breadth
first, using rcu_for_each_node_breadth_first().  This macro (shown below)
implements this breadth-first walk using a simple sequential traversal of
the ->node[] array that provides the structures making up the rcu_node
tree.  As you can see, this scan is completely independent of how CPU
numbers might be mapped to rcu_data slots in the leaf rcu_node structures.

							Thanx, Paul

/*
 * Do a full breadth-first scan of the rcu_node structures for the
 * specified rcu_state structure.
 */
#define rcu_for_each_node_breadth_first(rsp, rnp) \
	for ((rnp) = &(rsp)->node[0]; \
	     (rnp) < &(rsp)->node[rcu_num_nodes]; (rnp)++)

[toc] | [prev] | [next] | [standalone]

#1626111 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 11:42:32AM -0700, Paul E. McKenney wrote:

> I believe that you are missing the fact that RCU grace-period
> initialization and cleanup walks through the rcu_node tree breadth
> first, using rcu_for_each_node_breadth_first().

Indeed. That is the part I completely missed.

>                                                 This macro (shown below)
> implements this breadth-first walk using a simple sequential traversal of
> the ->node[] array that provides the structures making up the rcu_node
> tree.  As you can see, this scan is completely independent of how CPU
> numbers might be mapped to rcu_data slots in the leaf rcu_node structures.

So this code is clearly not a hotpath, but still its performance
matters?

Seems like you cannot win here :/

[toc] | [prev] | [next] | [standalone]

#1626123 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 19, 2017 at 03:22:26PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 11:42:32AM -0700, Paul E. McKenney wrote:
> 
> > I believe that you are missing the fact that RCU grace-period
> > initialization and cleanup walks through the rcu_node tree breadth
> > first, using rcu_for_each_node_breadth_first().
> 
> Indeed. That is the part I completely missed.
> 
> >                                                 This macro (shown below)
> > implements this breadth-first walk using a simple sequential traversal of
> > the ->node[] array that provides the structures making up the rcu_node
> > tree.  As you can see, this scan is completely independent of how CPU
> > numbers might be mapped to rcu_data slots in the leaf rcu_node structures.
> 
> So this code is clearly not a hotpath, but still its performance
> matters?
> 
> Seems like you cannot win here :/

So I sort of see what that code does, but I cannot quite grasp from the
comments near there _why_ it is doing this.

My thinking is that normal (active CPUs) will update their state at tick
time through the tree, and once the state reaches the root node, IOW all
CPUs agree they've observed that particular state, we advance the global
state, rinse repeat. That's how tree-rcu works.

NOHZ-idle stuff would be excluded entirely; that is, if we're allowed to
go idle we're up-to-date, and completely drop out of the state tracking.
When we become active again, we can simply sync the CPU's state to the
active state and go from there -- ignoring whatever happened in the
mean-time.

So why do we have to do machine wide updates? How can we get at the end
up a grace period without all CPUs already agreeing that its complete?

/me puzzled.

[toc] | [prev] | [next] | [standalone]

#1626300 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 19, 2017 at 03:48:35PM +0200, Peter Zijlstra wrote:
> On Wed, Apr 19, 2017 at 03:22:26PM +0200, Peter Zijlstra wrote:
> > On Thu, Apr 13, 2017 at 11:42:32AM -0700, Paul E. McKenney wrote:
> > 
> > > I believe that you are missing the fact that RCU grace-period
> > > initialization and cleanup walks through the rcu_node tree breadth
> > > first, using rcu_for_each_node_breadth_first().
> > 
> > Indeed. That is the part I completely missed.
> > 
> > >                                                 This macro (shown below)
> > > implements this breadth-first walk using a simple sequential traversal of
> > > the ->node[] array that provides the structures making up the rcu_node
> > > tree.  As you can see, this scan is completely independent of how CPU
> > > numbers might be mapped to rcu_data slots in the leaf rcu_node structures.
> > 
> > So this code is clearly not a hotpath, but still its performance
> > matters?
> > 
> > Seems like you cannot win here :/
> 
> So I sort of see what that code does, but I cannot quite grasp from the
> comments near there _why_ it is doing this.
> 
> My thinking is that normal (active CPUs) will update their state at tick
> time through the tree, and once the state reaches the root node, IOW all
> CPUs agree they've observed that particular state, we advance the global
> state, rinse repeat. That's how tree-rcu works.
> 
> NOHZ-idle stuff would be excluded entirely; that is, if we're allowed to
> go idle we're up-to-date, and completely drop out of the state tracking.
> When we become active again, we can simply sync the CPU's state to the
> active state and go from there -- ignoring whatever happened in the
> mean-time.
> 
> So why do we have to do machine wide updates? How can we get at the end
> up a grace period without all CPUs already agreeing that its complete?
> 
> /me puzzled.

This a decent overall summary of how RCU grace periods work, but there
are quite a few corner cases that complicate things.  In this email,
I will focus on just one of them, starting with CPUs returning from
NOHZ-idle state.

In theory, you are correct when you say that we could have CPUs sync up
with current RCU state immediately upon return from idle.  In practice,
people are already screaming at me about the single CPU-local atomic
operation and memory barriers, so adding code on the idle-exit fastpath
to acquire the leaf rcu_node structure's lock and grab the current
state would do nothing but cause Marc Zyngier and many others to report
performance bugs to me.

And even that would not be completely sufficient.  After all, the state
in the leaf rcu_node structure will be out of date during grace-period
initialization and cleanup.  So to -completely- synchronize state for
the incoming CPU, I would have to acquire the root rcu_node structure's
lock and look at the live state.  Needless to say, the performance and
scalability implications of acquiring a global lock on each and every
idle exit event is not going to be at all pretty.

This means that even non-idle CPUs must necessarily be allowed to have
different about which grace period is currently in effect.  We simply
cannot have total agreement on when a given grace period starts or
ends, because such agreement is just too expensive.  Therefore, when a
grace period begins, the grace-period kthread scans the rcu_node tree
propagating this transition through the rcu_node tree.  And similarly
when a grace period ends.

Because the rcu_node tree is mapped into a dense array, and because
the scan proceeds in index order, the scan operation is pretty much
best-case for the cache hardware.  But on large machines with large
cache-miss latencies, it can still inflict a bit of pain -- almost all
of which has been addressed by the switch to grace-period kthreads.

Hey, you asked!!!  ;-)

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1626411 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 19, 2017 at 08:08:09AM -0700, Paul E. McKenney wrote:
> And even that would not be completely sufficient.  After all, the state
> in the leaf rcu_node structure will be out of date during grace-period
> initialization and cleanup.  So to -completely- synchronize state for
> the incoming CPU, I would have to acquire the root rcu_node structure's
> lock and look at the live state.  Needless to say, the performance and
> scalability implications of acquiring a global lock on each and every
> idle exit event is not going to be at all pretty.

Arguably you could use a seqlock to read the global state. Will still
ponder things a bit more, esp. those bugs you pointed me at from just
reading gpnum.

[toc] | [prev] | [next] | [standalone]

#1626440 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 19, 2017 at 05:40:40PM +0200, Peter Zijlstra wrote:
> On Wed, Apr 19, 2017 at 08:08:09AM -0700, Paul E. McKenney wrote:
> > And even that would not be completely sufficient.  After all, the state
> > in the leaf rcu_node structure will be out of date during grace-period
> > initialization and cleanup.  So to -completely- synchronize state for
> > the incoming CPU, I would have to acquire the root rcu_node structure's
> > lock and look at the live state.  Needless to say, the performance and
> > scalability implications of acquiring a global lock on each and every
> > idle exit event is not going to be at all pretty.
> 
> Arguably you could use a seqlock to read the global state. Will still
> ponder things a bit more, esp. those bugs you pointed me at from just
> reading gpnum.

Looking forward to hearing what you come up with!

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1626268 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Wed, Apr 19, 2017 at 03:22:26PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 11:42:32AM -0700, Paul E. McKenney wrote:
> 
> > I believe that you are missing the fact that RCU grace-period
> > initialization and cleanup walks through the rcu_node tree breadth
> > first, using rcu_for_each_node_breadth_first().
> 
> Indeed. That is the part I completely missed.
> 
> >                                                 This macro (shown below)
> > implements this breadth-first walk using a simple sequential traversal of
> > the ->node[] array that provides the structures making up the rcu_node
> > tree.  As you can see, this scan is completely independent of how CPU
> > numbers might be mapped to rcu_data slots in the leaf rcu_node structures.
> 
> So this code is clearly not a hotpath, but still its performance
> matters?
> 
> Seems like you cannot win here :/

Welcome to my world!!!  ;-)

But yes, running on 4096-CPU systems can put some serious stress on
some surprising areas.  Especially when those systems have cache-miss
latencies well in excess of a microsecond, and the users are nevertheless
expecting scheduling latencies well below 100 microseconds.

It was a fun challenge, I grant you that!

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

#1623262 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 10:31:00AM -0700, Paul E. McKenney wrote:
> On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote:

> > And I have vague memories of it actually causing lock contention, but
> > I've forgotten how that worked.
> 
> That is a new one on me.  I can easily see how not skewing ticks could
> cause serious lock contention, but am missing how skewed ticks would
> do so.

It could've been something like cacheline bouncing. Where with a
synchronized tick, the (global) cacheline would get used by all CPUs on
a node before heading out to the next node etc.. Where with a skewed
tick, it would forever bounce around.

[toc] | [prev] | [next] | [standalone]

#1623302 — Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick

On Thu, Apr 13, 2017 at 08:29:39PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 13, 2017 at 10:31:00AM -0700, Paul E. McKenney wrote:
> > On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote:
> 
> > > And I have vague memories of it actually causing lock contention, but
> > > I've forgotten how that worked.
> > 
> > That is a new one on me.  I can easily see how not skewing ticks could
> > cause serious lock contention, but am missing how skewed ticks would
> > do so.
> 
> It could've been something like cacheline bouncing. Where with a
> synchronized tick, the (global) cacheline would get used by all CPUs on
> a node before heading out to the next node etc.. Where with a skewed
> tick, it would forever bounce around.

In other words, motivating the order of the skewed ticks to be guided
by hardware locality?

							Thanx, Paul

[toc] | [prev] | [next] | [standalone]

Page 1 of 5  [1] 2 3 4 5  Next page →

Back to top | Article view | linux.kernel

csiph-web