Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1548155

[PATCH v4 06/20] futex: Allow direct attachment of futex_state objects to hash bucket

Show all headers | View raw

Currently, the futex state objects can only be located indirectly as

        hash bucket => futex_q => futex state

Actually it can be beneficial in some cases to locate the futex state
object directly from the hash bucket without the futex_q middleman.
Therefore, a new list head to link the futex state objects as well
as a new spinlock to manage them are added to the hash bucket.

To limit size increase for UP systems, these new fields are only for
SMP machines where the cacheline alignment of the hash bucket leaves
it with enough empty space for the new fields.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/futex.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/kernel/futex.c b/kernel/futex.c
index c8ff773..8b9e982 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -207,6 +207,11 @@ struct futex_state {
 	struct list_head list;
 
 	/*
+	 * Can link to fs_head in the owning hash bucket.
+	 */
+	struct list_head fs_list;
+
+	/*
 	 * The PI object:
 	 */
 	struct rt_mutex pi_mutex;
@@ -262,11 +267,24 @@ struct futex_q {
  * Hash buckets are shared by all the futex_keys that hash to the same
  * location.  Each key may have multiple futex_q structures, one for each task
  * waiting on a futex.
+ *
+ * Alternatively (in SMP), a key can be associated with a unique futex_state
+ * object where multiple waiters waiting for that futex can queue up in that
+ * futex_state object without using the futex_q structure. A separate
+ * futex_state lock (fs_lock) is used for processing those futex_state objects.
  */
 struct futex_hash_bucket {
 	atomic_t waiters;
 	spinlock_t lock;
 	struct plist_head chain;
+
+#ifdef CONFIG_SMP
+	/*
+	 * Fields for managing futex_state object list
+	 */
+	spinlock_t fs_lock;
+	struct list_head fs_head;
+#endif
 } ____cacheline_aligned_in_smp;
 
 /*
@@ -873,6 +891,8 @@ static int refill_futex_state_cache(void)
 		return -ENOMEM;
 
 	INIT_LIST_HEAD(&state->list);
+	INIT_LIST_HEAD(&state->fs_list);
+
 	/* pi_mutex gets initialized later */
 	state->owner = NULL;
 	atomic_set(&state->refcount, 1);
@@ -3363,6 +3383,10 @@ static int __init futex_init(void)
 		atomic_set(&futex_queues[i].waiters, 0);
 		plist_head_init(&futex_queues[i].chain);
 		spin_lock_init(&futex_queues[i].lock);
+#ifdef CONFIG_SMP
+		INIT_LIST_HEAD(&futex_queues[i].fs_head);
+		spin_lock_init(&futex_queues[i].fs_lock);
+#endif
 	}
 
 	return 0;
-- 
1.8.3.1

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v4 00/20] futex: Introducing throughput-optimized (TP) futexes Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 07/20] futex: Introduce throughput-optimized (TP) futexes Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
    Re: [PATCH v4 07/20] futex: Introduce throughput-optimized (TP)  futexes kbuild test robot <lkp@intel.com> - 2016-12-29 22:20 +0100
  [PATCH v4 08/20] TP-futex: Enable robust handling Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 06/20] futex: Allow direct attachment of futex_state objects to hash bucket Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 20/20] futex: Dump internal futex state via debugfs Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
    Re: [PATCH v4 20/20] futex: Dump internal futex state via debugfs kbuild test robot <lkp@intel.com> - 2016-12-29 20:00 +0100
  [PATCH v4 16/20] TP-futex: Group readers together in wait queue Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
    Re: [PATCH v4 16/20] TP-futex: Group readers together in wait queue kbuild test robot <lkp@intel.com> - 2016-12-29 21:00 +0100
    Re: [PATCH v4 16/20] TP-futex: Group readers together in wait queue kbuild test robot <lkp@intel.com> - 2016-12-29 21:20 +0100
  [PATCH v4 15/20] TP-futex: Enable kernel reader lock stealing Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 01/20] futex: Consolidate duplicated timer setup code Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 11/20] TP-futex: Add timeout support Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 04/20] futex: Consolidate pure pi_state_list add & delete codes to helpers Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 14/20] TP-futex: Support userspace reader/writer locks Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
    Re: [PATCH v4 14/20] TP-futex: Support userspace reader/writer locks kbuild test robot <lkp@intel.com> - 2016-12-29 19:40 +0100
  [PATCH v4 09/20] TP-futex: Implement lock handoff to prevent lock starvation Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100
  [PATCH v4 18/20] perf bench: New microbenchmark for userspace rwlock performance Waiman Long <longman@redhat.com> - 2016-12-29 17:20 +0100

csiph-web