Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1592470
| From | Peter Zijlstra <peterz@infradead.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() |
| Date | 2017-03-04 12:50 +0100 |
| Message-ID | <thgmd-6M-11@gated-at.bofh.it> (permalink) |
| References | <theNr-7vq-5@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
With the ultimate goal of keeping rt_mutex wait_list and futex_q
waiters consistent we want to split 'rt_mutex_futex_lock()' into finer
parts, such that only the actual blocking can be done without hb->lock
held.
This means we need to split rt_mutex_finish_proxy_lock() into two
parts, one that does the blocking and one that does remove_waiter()
when we fail to acquire.
When we do acquire, we can safely remove ourselves, since there is no
concurrency on the lock owner.
This means that, except for futex_lock_pi(), all wait_list
modifications are done with both hb->lock and wait_lock held.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/futex.c | 5 +++-
kernel/locking/rtmutex.c | 47 ++++++++++++++++++++++++++++++++++------
kernel/locking/rtmutex_common.h | 8 ++++--
3 files changed, 49 insertions(+), 11 deletions(-)
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -3012,10 +3012,13 @@ static int futex_wait_requeue_pi(u32 __u
*/
WARN_ON(!q.pi_state);
pi_mutex = &q.pi_state->pi_mutex;
- ret = rt_mutex_finish_proxy_lock(pi_mutex, to, &rt_waiter);
+ ret = rt_mutex_wait_proxy_lock(pi_mutex, to, &rt_waiter);
debug_rt_mutex_free_waiter(&rt_waiter);
spin_lock(q.lock_ptr);
+ if (ret && !rt_mutex_cleanup_proxy_lock(pi_mutex, &rt_waiter))
+ ret = 0;
+
/*
* Fixup the pi_state owner and possibly acquire the lock if we
* haven't already.
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -1747,21 +1747,23 @@ struct task_struct *rt_mutex_next_owner(
}
/**
- * rt_mutex_finish_proxy_lock() - Complete lock acquisition
+ * rt_mutex_wait_proxy_lock() - Wait for lock acquisition
* @lock: the rt_mutex we were woken on
* @to: the timeout, null if none. hrtimer should already have
* been started.
* @waiter: the pre-initialized rt_mutex_waiter
*
- * Complete the lock acquisition started our behalf by another thread.
+ * Wait for the the lock acquisition started on our behalf by
+ * rt_mutex_start_proxy_lock(). Upon failure, the caller must call
+ * rt_mutex_cleanup_proxy_lock().
*
* Returns:
* 0 - success
* <0 - error, one of -EINTR, -ETIMEDOUT
*
- * Special API call for PI-futex requeue support
+ * Special API call for PI-futex support
*/
-int rt_mutex_finish_proxy_lock(struct rt_mutex *lock,
+int rt_mutex_wait_proxy_lock(struct rt_mutex *lock,
struct hrtimer_sleeper *to,
struct rt_mutex_waiter *waiter)
{
@@ -1774,9 +1776,6 @@ int rt_mutex_finish_proxy_lock(struct rt
/* sleep on the mutex */
ret = __rt_mutex_slowlock(lock, TASK_INTERRUPTIBLE, to, waiter);
- if (unlikely(ret))
- remove_waiter(lock, waiter);
-
/*
* try_to_take_rt_mutex() sets the waiter bit unconditionally. We might
* have to fix that up.
@@ -1787,3 +1786,37 @@ int rt_mutex_finish_proxy_lock(struct rt
return ret;
}
+
+/**
+ * rt_mutex_cleanup_proxy_lock() - Cleanup failed lock acquisition
+ * @lock: the rt_mutex we were woken on
+ * @waiter: the pre-initialized rt_mutex_waiter
+ *
+ * Clean up the failed lock acquisition as per rt_mutex_wait_proxy_lock().
+ *
+ * Returns:
+ * true - did the cleanup, we done.
+ * false - we acquired the lock after rt_mutex_wait_proxy_lock() returned,
+ * caller should disregards its return value.
+ *
+ * Special API call for PI-futex support
+ */
+bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock,
+ struct rt_mutex_waiter *waiter)
+{
+ bool cleanup = false;
+
+ raw_spin_lock_irq(&lock->wait_lock);
+ /*
+ * If we acquired the lock, no cleanup required.
+ */
+ if (rt_mutex_owner(lock) != current) {
+ remove_waiter(lock, waiter);
+ fixup_rt_mutex_waiters(lock);
+ cleanup = true;
+ }
+ raw_spin_unlock_irq(&lock->wait_lock);
+
+ return cleanup;
+}
+
--- a/kernel/locking/rtmutex_common.h
+++ b/kernel/locking/rtmutex_common.h
@@ -106,9 +106,11 @@ extern void rt_mutex_proxy_unlock(struct
extern int rt_mutex_start_proxy_lock(struct rt_mutex *lock,
struct rt_mutex_waiter *waiter,
struct task_struct *task);
-extern int rt_mutex_finish_proxy_lock(struct rt_mutex *lock,
- struct hrtimer_sleeper *to,
- struct rt_mutex_waiter *waiter);
+extern int rt_mutex_wait_proxy_lock(struct rt_mutex *lock,
+ struct hrtimer_sleeper *to,
+ struct rt_mutex_waiter *waiter);
+extern bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock,
+ struct rt_mutex_waiter *waiter);
extern int rt_mutex_timed_futex_lock(struct rt_mutex *l, struct hrtimer_sleeper *to);
extern int rt_mutex_futex_trylock(struct rt_mutex *l);
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH -v5 00/14] the saga of FUTEX_UNLOCK_PI wobbles continues Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
[PATCH -v5 02/14] futex: Add missing error handling to FUTEX_REQUEUE_PI Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
[PATCH -v5 04/14] futex: Use smp_store_release() in mark_wake_futex() Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
[PATCH -v5 10/14] futex: Pull rt_mutex_futex_unlock() out from under hb->lock Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
Re: [PATCH -v5 10/14] futex: Pull rt_mutex_futex_unlock() out from under hb->lock Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 19:20 +0100
Re: [PATCH -v5 10/14] futex: Pull rt_mutex_futex_unlock() out from under hb->lock Peter Zijlstra <peterz@infradead.org> - 2017-03-07 19:50 +0100
[PATCH -v5 14/14] futex: futex_unlock_pi() determinism Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
Re: [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 15:40 +0100
Re: [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Peter Zijlstra <peterz@infradead.org> - 2017-03-07 19:50 +0100
Re: [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Peter Zijlstra <peterz@infradead.org> - 2017-03-13 10:30 +0100
Re: [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Thomas Gleixner <tglx@linutronix.de> - 2017-03-13 15:30 +0100
Re: [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Peter Zijlstra <peterz@infradead.org> - 2017-03-13 16:20 +0100
[PATCH -v5 11/14] futex,rt_mutex: Introduce rt_mutex_init_waiter() Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
[PATCH -v5 07/14] futex: Change locking rules Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:10 +0100
Re: [PATCH -v5 07/14] futex: Change locking rules Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 14:50 +0100
Re: [PATCH -v5 07/14] futex: Change locking rules Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-03-07 18:20 +0100
Re: [PATCH -v5 07/14] futex: Change locking rules Peter Zijlstra <peterz@infradead.org> - 2017-03-07 19:10 +0100
[PATCH -v5 06/14] futex,rt_mutex: Provide futex specific rt_mutex API Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:50 +0100
[PATCH -v5 05/14] futex: Remove rt_mutex_deadlock_account_*() Peter Zijlstra <peterz@infradead.org> - 2017-03-04 11:50 +0100
[PATCH -v5 08/14] futex: Cleanup refcounting Peter Zijlstra <peterz@infradead.org> - 2017-03-04 12:50 +0100
[PATCH -v5 03/14] futex: Cleanup variable names for futex_top_waiter() Peter Zijlstra <peterz@infradead.org> - 2017-03-04 12:50 +0100
[PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() Peter Zijlstra <peterz@infradead.org> - 2017-03-04 12:50 +0100
Re: [PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 15:50 +0100
Re: [PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() Thomas Gleixner <tglx@linutronix.de> - 2017-03-07 19:50 +0100
Re: [PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() Peter Zijlstra <peterz@infradead.org> - 2017-03-07 19:50 +0100
[PATCH] futex: move debug_rt_mutex_free_waiter() further down Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-03-08 16:40 +0100
Re: [PATCH] futex: move debug_rt_mutex_free_waiter() further down Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-03-08 16:40 +0100
Re: [PATCH] futex: move debug_rt_mutex_free_waiter() further down Steven Rostedt <rostedt@goodmis.org> - 2017-03-08 17:40 +0100
Re: [PATCH] futex: move debug_rt_mutex_free_waiter() further down Steven Rostedt <rostedt@goodmis.org> - 2017-03-08 17:30 +0100
Re: [PATCH] futex: move debug_rt_mutex_free_waiter() further down Peter Zijlstra <peterz@infradead.org> - 2017-03-13 10:20 +0100
csiph-web