Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1491280

[PATCH -v2 8/9] rtmutex: Fix PI chain order integrity

Show all headers | View raw

rt_mutex_waiter::prio is a copy of task_struct::prio which is updated
during the PI chain walk, such that the PI chain order isn't messed up
by (asynchronous) task state updates.

Currently rt_mutex_waiter_less() uses task state for deadline tasks;
this is broken, since the task state can, as said above, change
asynchronously, causing the RB tree order to change without actual
tree update -> FAIL.

Fix this by also copying the deadline into the rt_mutex_waiter state
and updating it along with its prio field.

Ideally we would also force PI chain updates whenever DL tasks update
their deadline parameter, but for first approximation this is less
broken than it was.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/locking/rtmutex.c        |   29 +++++++++++++++++++++++++++--
 kernel/locking/rtmutex_common.h |    1 +
 2 files changed, 28 insertions(+), 2 deletions(-)

--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -172,8 +172,7 @@ rt_mutex_waiter_less(struct rt_mutex_wai
 	 * then right waiter has a dl_prio() too.
 	 */
 	if (dl_prio(left->prio))
-		return dl_time_before(left->task->dl.deadline,
-				      right->task->dl.deadline);
+		return dl_time_before(left->deadline, right->deadline);
 
 	return 0;
 }
@@ -584,7 +583,26 @@ static int rt_mutex_adjust_prio_chain(st
 
 	/* [7] Requeue the waiter in the lock waiter tree. */
 	rt_mutex_dequeue(lock, waiter);
+
+	/*
+	 * Update the waiter prio fields now that we're dequeued.
+	 *
+	 * These values can have changed through either:
+	 *
+	 *   sys_sched_set_scheduler() / sys_sched_setattr()
+	 *
+	 * or
+	 *
+	 *   DL CBS enforcement advancing the effective deadline.
+	 *
+	 * Even though pi_waiters also uses these fields, and that tree is only
+	 * updated in [11], we can do this here, since we hold [L], which
+	 * serializes all pi_waiters access and rb_erase() does not care about
+	 * the values of the node being removed.
+	 */
 	waiter->prio = task->prio;
+	waiter->deadline = task->dl.deadline;
+
 	rt_mutex_enqueue(lock, waiter);
 
 	/* [8] Release the task */
@@ -711,6 +729,8 @@ static int rt_mutex_adjust_prio_chain(st
 static int try_to_take_rt_mutex(struct rt_mutex *lock, struct task_struct *task,
 				struct rt_mutex_waiter *waiter)
 {
+	lockdep_assert_held(&lock->wait_lock);
+
 	/*
 	 * Before testing whether we can acquire @lock, we set the
 	 * RT_MUTEX_HAS_WAITERS bit in @lock->owner. This forces all
@@ -838,6 +858,8 @@ static int task_blocks_on_rt_mutex(struc
 	struct rt_mutex *next_lock;
 	int chain_walk = 0, res;
 
+	lockdep_assert_held(&lock->wait_lock);
+
 	/*
 	 * Early deadlock detection. We really don't want the task to
 	 * enqueue on itself just to untangle the mess later. It's not
@@ -855,6 +877,7 @@ static int task_blocks_on_rt_mutex(struc
 	waiter->task = task;
 	waiter->lock = lock;
 	waiter->prio = task->prio;
+	waiter->deadline = task->dl.deadline;
 
 	/* Get the top priority waiter on the lock */
 	if (rt_mutex_has_waiters(lock))
@@ -972,6 +995,8 @@ static void remove_waiter(struct rt_mute
 	struct task_struct *owner = rt_mutex_owner(lock);
 	struct rt_mutex *next_lock;
 
+	lockdep_assert_held(&lock->wait_lock);
+
 	raw_spin_lock(&current->pi_lock);
 	rt_mutex_dequeue(lock, waiter);
 	current->pi_blocked_on = NULL;
--- a/kernel/locking/rtmutex_common.h
+++ b/kernel/locking/rtmutex_common.h
@@ -33,6 +33,7 @@ struct rt_mutex_waiter {
 	struct rt_mutex		*deadlock_lock;
 #endif
 	int prio;
+	u64 deadline;
 };
 
 /*

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH -v2 0/9] PI and assorted failings Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
  [PATCH -v2 5/9] rtmutex: Clean up Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
    Re: [PATCH -v2 5/9] rtmutex: Clean up Steven Rostedt <rostedt@goodmis.org> - 2016-09-26 18:20 +0200
      Re: [PATCH -v2 5/9] rtmutex: Clean up Thomas Gleixner <tglx@linutronix.de> - 2016-09-29 17:00 +0200
  [PATCH -v2 3/9] sched/deadline/rtmutex: Dont miss the dl_runtime/dl_period update Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
    Re: [PATCH -v2 3/9] sched/deadline/rtmutex: Dont miss the  dl_runtime/dl_period update Steven Rostedt <rostedt@goodmis.org> - 2016-09-26 18:10 +0200
    Re: [PATCH -v2 3/9] sched/deadline/rtmutex: Dont miss the  dl_runtime/dl_period update Thomas Gleixner <tglx@linutronix.de> - 2016-09-29 17:00 +0200
  [PATCH -v2 8/9] rtmutex: Fix PI chain order integrity Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
  [PATCH -v2 6/9] sched/rtmutex: Refactor rt_mutex_setprio() Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
    Re: [PATCH -v2 6/9] sched/rtmutex: Refactor rt_mutex_setprio() Steven Rostedt <rostedt@goodmis.org> - 2016-09-26 19:00 +0200
  [PATCH -v2 7/9] sched,tracing: Update trace_sched_pi_setprio() Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
    Re: [PATCH -v2 7/9] sched,tracing: Update trace_sched_pi_setprio() Steven Rostedt <rostedt@goodmis.org> - 2016-09-26 19:10 +0200
      Re: [PATCH -v2 7/9] sched,tracing: Update trace_sched_pi_setprio() Peter Zijlstra <peterz@infradead.org> - 2016-09-27 09:50 +0200
  [PATCH -v2 4/9] rtmutex: Remove rt_mutex_fastunlock() Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200
    Re: [PATCH -v2 4/9] rtmutex: Remove rt_mutex_fastunlock() Thomas Gleixner <tglx@linutronix.de> - 2016-09-29 17:00 +0200
  [PATCH -v2 9/9] rtmutex: Fix more prio comparisons Peter Zijlstra <peterz@infradead.org> - 2016-09-26 14:50 +0200

csiph-web