Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1461030 > unrolled thread

[PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu

Started byWanpeng Li <kernellwp@gmail.com>
First post2016-08-12 11:30 +0200
Last post2016-08-19 16:00 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu Wanpeng Li <kernellwp@gmail.com> - 2016-08-12 11:30 +0200
    Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick  dependency for offline cpu Wanpeng Li <kernellwp@gmail.com> - 2016-08-16 04:10 +0200
    Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick  dependency for offline cpu Peter Zijlstra <peterz@infradead.org> - 2016-08-19 15:30 +0200
      Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick  dependency for offline cpu Wanpeng Li <kernellwp@gmail.com> - 2016-08-19 16:00 +0200

#1461030 — [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu

FromWanpeng Li <kernellwp@gmail.com>
Date2016-08-12 11:30 +0200
Subject[PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu
Message-ID<s5gWS-8h2-7@gated-at.bofh.it>
From: Wanpeng Li <wanpeng.li@hotmail.com>

The dl task will be replenished after dl task timer fire and start a 
new period. It will be enqueued and to re-evaluate its dependency on 
the tick in order to restart it. However, if cpu is hot-unplug, 
irq_work_queue will splash since the target cpu is offline.

As a result:

    WARNING: CPU: 2 PID: 0 at kernel/irq_work.c:69 irq_work_queue_on+0xad/0xe0
    Call Trace:
     dump_stack+0x99/0xd0
     __warn+0xd1/0xf0
     warn_slowpath_null+0x1d/0x20
     irq_work_queue_on+0xad/0xe0
     tick_nohz_full_kick_cpu+0x44/0x50
     tick_nohz_dep_set_cpu+0x74/0xb0
     enqueue_task_dl+0x226/0x480
     activate_task+0x5c/0xa0
     dl_task_timer+0x19b/0x2c0
     ? push_dl_task.part.31+0x190/0x190
  
This can be triggered by hot-unplug the full dynticks cpu which dl 
task is running on. 

We enqueue the dl task on the offline CPU, because we need to do 
replenish for start_dl_timer(). So, as Juri pointed out, we would 
need to do is calling replenish_dl_entity() directly, instead of 
enqueue_task_dl(). pi_se shouldn't be a problem as the task shouldn't 
be boosted if it was throttled.

This patch fix it by just replenish dl entity to avoid the intention 
to re-evaluate tick dependency if the cpu is offline.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@arm.com>
Cc: Luca Abeni <luca.abeni@unitn.it>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
v2 -> v3:
 * move rq->online check under CONFIG_SMP
v1 -> v2:
 * replenish dl entity

 kernel/sched/deadline.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index d091f4a..ce0fb00 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -641,6 +641,11 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
 		goto unlock;
 	}
 
+#ifdef CONFIG_SMP
+	if (unlikely(!rq->online))
+		goto offline;
+#endif
+
 	enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
 	if (dl_task(rq->curr))
 		check_preempt_curr_dl(rq, p, 0);
@@ -648,6 +653,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
 		resched_curr(rq);
 
 #ifdef CONFIG_SMP
+offline:
 	/*
 	 * Perform balancing operations here; after the replenishments.  We
 	 * cannot drop rq->lock before this, otherwise the assertion in
@@ -659,6 +665,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
 	 * XXX figure out if select_task_rq_dl() deals with offline cpus.
 	 */
 	if (unlikely(!rq->online)) {
+		replenish_dl_entity(dl_se, dl_se);
 		lockdep_unpin_lock(&rq->lock, rf.cookie);
 		rq = dl_task_offline_migration(rq, p);
 		rf.cookie = lockdep_pin_lock(&rq->lock);
-- 
1.9.1

[toc] | [next] | [standalone]

#1463345 — Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu

FromWanpeng Li <kernellwp@gmail.com>
Date2016-08-16 04:10 +0200
SubjectRe: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu
Message-ID<s6BZf-5nu-3@gated-at.bofh.it>
In reply to#1461030
Ping Juri, Frederic, could I get your Acked?
2016-08-12 17:24 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
>
> The dl task will be replenished after dl task timer fire and start a
> new period. It will be enqueued and to re-evaluate its dependency on
> the tick in order to restart it. However, if cpu is hot-unplug,
> irq_work_queue will splash since the target cpu is offline.
>
> As a result:
>
>     WARNING: CPU: 2 PID: 0 at kernel/irq_work.c:69 irq_work_queue_on+0xad/0xe0
>     Call Trace:
>      dump_stack+0x99/0xd0
>      __warn+0xd1/0xf0
>      warn_slowpath_null+0x1d/0x20
>      irq_work_queue_on+0xad/0xe0
>      tick_nohz_full_kick_cpu+0x44/0x50
>      tick_nohz_dep_set_cpu+0x74/0xb0
>      enqueue_task_dl+0x226/0x480
>      activate_task+0x5c/0xa0
>      dl_task_timer+0x19b/0x2c0
>      ? push_dl_task.part.31+0x190/0x190
>
> This can be triggered by hot-unplug the full dynticks cpu which dl
> task is running on.
>
> We enqueue the dl task on the offline CPU, because we need to do
> replenish for start_dl_timer(). So, as Juri pointed out, we would
> need to do is calling replenish_dl_entity() directly, instead of
> enqueue_task_dl(). pi_se shouldn't be a problem as the task shouldn't
> be boosted if it was throttled.
>
> This patch fix it by just replenish dl entity to avoid the intention
> to re-evaluate tick dependency if the cpu is offline.
>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@arm.com>
> Cc: Luca Abeni <luca.abeni@unitn.it>
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
> ---
> v2 -> v3:
>  * move rq->online check under CONFIG_SMP
> v1 -> v2:
>  * replenish dl entity
>
>  kernel/sched/deadline.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index d091f4a..ce0fb00 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -641,6 +641,11 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>                 goto unlock;
>         }
>
> +#ifdef CONFIG_SMP
> +       if (unlikely(!rq->online))
> +               goto offline;
> +#endif
> +
>         enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
>         if (dl_task(rq->curr))
>                 check_preempt_curr_dl(rq, p, 0);
> @@ -648,6 +653,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>                 resched_curr(rq);
>
>  #ifdef CONFIG_SMP
> +offline:
>         /*
>          * Perform balancing operations here; after the replenishments.  We
>          * cannot drop rq->lock before this, otherwise the assertion in
> @@ -659,6 +665,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>          * XXX figure out if select_task_rq_dl() deals with offline cpus.
>          */
>         if (unlikely(!rq->online)) {
> +               replenish_dl_entity(dl_se, dl_se);
>                 lockdep_unpin_lock(&rq->lock, rf.cookie);
>                 rq = dl_task_offline_migration(rq, p);
>                 rf.cookie = lockdep_pin_lock(&rq->lock);
> --
> 1.9.1
>



-- 
Regards,
Wanpeng Li

[toc] | [prev] | [next] | [standalone]

#1466408 — Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu

FromPeter Zijlstra <peterz@infradead.org>
Date2016-08-19 15:30 +0200
SubjectRe: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu
Message-ID<s7S1Y-68X-43@gated-at.bofh.it>
In reply to#1461030
On Fri, Aug 12, 2016 at 05:24:03PM +0800, Wanpeng Li wrote:

> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index d091f4a..ce0fb00 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -641,6 +641,11 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>  		goto unlock;
>  	}
>  
> +#ifdef CONFIG_SMP
> +	if (unlikely(!rq->online))
> +		goto offline;
> +#endif
> +
>  	enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
>  	if (dl_task(rq->curr))
>  		check_preempt_curr_dl(rq, p, 0);
> @@ -648,6 +653,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>  		resched_curr(rq);
>  
>  #ifdef CONFIG_SMP
> +offline:
>  	/*
>  	 * Perform balancing operations here; after the replenishments.  We
>  	 * cannot drop rq->lock before this, otherwise the assertion in
> @@ -659,6 +665,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>  	 * XXX figure out if select_task_rq_dl() deals with offline cpus.
>  	 */
>  	if (unlikely(!rq->online)) {
> +		replenish_dl_entity(dl_se, dl_se);
>  		lockdep_unpin_lock(&rq->lock, rf.cookie);
>  		rq = dl_task_offline_migration(rq, p);

So I don't like this, even if it magically works. With this we end up
calling dl_task_offline_migration() -> deactivate_task() while the task
isn't on the runqueue at all.

>  		rf.cookie = lockdep_pin_lock(&rq->lock);
> -- 
> 1.9.1
>

[toc] | [prev] | [next] | [standalone]

#1466444 — Re: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu

FromWanpeng Li <kernellwp@gmail.com>
Date2016-08-19 16:00 +0200
SubjectRe: [PATCH v3] sched/deadline: Fix the intention to re-evalute tick dependency for offline cpu
Message-ID<s7SuZ-6kb-23@gated-at.bofh.it>
In reply to#1466408
2016-08-19 21:25 GMT+08:00 Peter Zijlstra <peterz@infradead.org>:
> On Fri, Aug 12, 2016 at 05:24:03PM +0800, Wanpeng Li wrote:
>
>> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
>> index d091f4a..ce0fb00 100644
>> --- a/kernel/sched/deadline.c
>> +++ b/kernel/sched/deadline.c
>> @@ -641,6 +641,11 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>>               goto unlock;
>>       }
>>
>> +#ifdef CONFIG_SMP
>> +     if (unlikely(!rq->online))
>> +             goto offline;
>> +#endif
>> +
>>       enqueue_task_dl(rq, p, ENQUEUE_REPLENISH);
>>       if (dl_task(rq->curr))
>>               check_preempt_curr_dl(rq, p, 0);
>> @@ -648,6 +653,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>>               resched_curr(rq);
>>
>>  #ifdef CONFIG_SMP
>> +offline:
>>       /*
>>        * Perform balancing operations here; after the replenishments.  We
>>        * cannot drop rq->lock before this, otherwise the assertion in
>> @@ -659,6 +665,7 @@ static enum hrtimer_restart dl_task_timer(struct hrtimer *timer)
>>        * XXX figure out if select_task_rq_dl() deals with offline cpus.
>>        */
>>       if (unlikely(!rq->online)) {
>> +             replenish_dl_entity(dl_se, dl_se);
>>               lockdep_unpin_lock(&rq->lock, rf.cookie);
>>               rq = dl_task_offline_migration(rq, p);
>
> So I don't like this, even if it magically works. With this we end up
> calling dl_task_offline_migration() -> deactivate_task() while the task
> isn't on the runqueue at all.

So how about v1, it also works :), https://lkml.org/lkml/2016/8/10/898

Regards,
Wanpeng Li

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web