Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1654066 > unrolled thread

Re: [RFC PATCH 1/2] sched/rt: add utilization tracking

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC PATCH 1/2] sched/rt: add utilization tracking Peter Zijlstra <peterz@infradead.org> - 2017-05-31 11:50 +0200
    Re: [RFC PATCH 1/2] sched/rt: add utilization tracking Peter Zijlstra <peterz@infradead.org> - 2017-05-31 12:40 +0200
      Re: [RFC PATCH 1/2] sched/rt: add utilization tracking Juri Lelli <juri.lelli@arm.com> - 2017-05-31 12:50 +0200
        Re: [RFC PATCH 1/2] sched/rt: add utilization tracking Peter Zijlstra <peterz@infradead.org> - 2017-05-31 13:40 +0200
    Re: [RFC PATCH 1/2] sched/rt: add utilization tracking Vincent Guittot <vincent.guittot@linaro.org> - 2017-05-31 13:30 +0200

#1654066 — Re: [RFC PATCH 1/2] sched/rt: add utilization tracking

On Wed, May 24, 2017 at 11:00:51AM +0200, Vincent Guittot wrote:
> schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs
> tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks
> are preempted by rt tasks and in this case util_avg reflects the remaining
> capacity that is used by cfs tasks but not what cfs tasks want to use. In such
> case, schedutil can select a lower OPP when cfs task runs whereas the CPU is
> overloaded. In order to have a more accurate view of the utilization of the
> CPU, we track the utilization that is used by RT tasks.
> DL tasks are not taken into account as they have their own utilization
> tracking mecanism.

Well, the DL tracking is fairly pessimistic; it assumes all DL tasks
will consume their total budget, which will rarely, if ever, happen.

So I suspect it might well be worth it to also track DL activity for the
purpose of compensating CFS.

In fact, I don't think you particularly care about RT here, as anything
!CFS that preempts it, including those interrupts you mentioned. Which
gets us back to what rt_avg is.

> We don't use rt_avg which doesn't have the same dynamic as PELT and which
> can include IRQ time that are also accounted in cfs task utilization

Well, if rt_avg includes IRQ time, then that IRQ time is not part of
the task clock.

> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> ---
> 
> If the changes are reasonnable, it might worth moving the PELT function in a
> dedicated pelt.c file and the ugly
> extern int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running);
> in a pelt.h header
> 
> 
>  kernel/sched/fair.c  | 21 +++++++++++++++++++++
>  kernel/sched/rt.c    |  9 +++++++++
>  kernel/sched/sched.h |  3 +++
>  3 files changed, 33 insertions(+)

Also, and I didn't check this, it is important that the windows are
aligned if you want to sum the values.

[toc] | [next] | [standalone]

#1654110

On Wed, May 31, 2017 at 11:40:47AM +0200, Peter Zijlstra wrote:
> On Wed, May 24, 2017 at 11:00:51AM +0200, Vincent Guittot wrote:
> > schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs
> > tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks
> > are preempted by rt tasks and in this case util_avg reflects the remaining
> > capacity that is used by cfs tasks but not what cfs tasks want to use. In such
> > case, schedutil can select a lower OPP when cfs task runs whereas the CPU is
> > overloaded. In order to have a more accurate view of the utilization of the
> > CPU, we track the utilization that is used by RT tasks.
> > DL tasks are not taken into account as they have their own utilization
> > tracking mecanism.
> 
> Well, the DL tracking is fairly pessimistic; it assumes all DL tasks
> will consume their total budget, which will rarely, if ever, happen.
> 
> So I suspect it might well be worth it to also track DL activity for the
> purpose of compensating CFS.

Again, it seems I have this CPPC/HWP crud firmly stuck in my brain.
Because I was thinking:

	min_freq = dl_util
	avg_freq = dl_avg + rt_avg + cfs_util


But given we don't actually have that split... meh.

> In fact, I don't think you particularly care about RT here, as anything
> !CFS that preempts it, including those interrupts you mentioned. Which
> gets us back to what rt_avg is.
> 
> > We don't use rt_avg which doesn't have the same dynamic as PELT and which
> > can include IRQ time that are also accounted in cfs task utilization
> 
> Well, if rt_avg includes IRQ time, then that IRQ time is not part of
> the task clock.

[toc] | [prev] | [next] | [standalone]

#1654116

On 31/05/17 12:30, Peter Zijlstra wrote:
> On Wed, May 31, 2017 at 11:40:47AM +0200, Peter Zijlstra wrote:
> > On Wed, May 24, 2017 at 11:00:51AM +0200, Vincent Guittot wrote:
> > > schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs
> > > tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks
> > > are preempted by rt tasks and in this case util_avg reflects the remaining
> > > capacity that is used by cfs tasks but not what cfs tasks want to use. In such
> > > case, schedutil can select a lower OPP when cfs task runs whereas the CPU is
> > > overloaded. In order to have a more accurate view of the utilization of the
> > > CPU, we track the utilization that is used by RT tasks.
> > > DL tasks are not taken into account as they have their own utilization
> > > tracking mecanism.
> > 
> > Well, the DL tracking is fairly pessimistic; it assumes all DL tasks
> > will consume their total budget, which will rarely, if ever, happen.
> > 
> > So I suspect it might well be worth it to also track DL activity for the
> > purpose of compensating CFS.
> 
> Again, it seems I have this CPPC/HWP crud firmly stuck in my brain.
> Because I was thinking:
> 
> 	min_freq = dl_util
> 	avg_freq = dl_avg + rt_avg + cfs_util
> 
> 
> But given we don't actually have that split... meh.
> 

Right, interesting. So, I guess the question is: should we have it? :)

IMHO, it makes sense and seems to benefit mobile use-cases I'm looking
at.

rt_avg though it also seems to build up very slowly (at least with
default configs). I'm experimenting with Vincent proposal and it looks
better (w.r.t. using rt_avg). Also summing up signals that behave
similarly doesn't seem the wrong thing to do.

[toc] | [prev] | [next] | [standalone]

#1654148

On Wed, May 31, 2017 at 11:41:11AM +0100, Juri Lelli wrote:
> On 31/05/17 12:30, Peter Zijlstra wrote:

> > Again, it seems I have this CPPC/HWP crud firmly stuck in my brain.
> > Because I was thinking:
> > 
> > 	min_freq = dl_util
> > 	avg_freq = dl_avg + rt_avg + cfs_util
> > 
> > 
> > But given we don't actually have that split... meh.
> > 
> 
> Right, interesting. So, I guess the question is: should we have it? :)

I think cpufreq should certainly look at moving into that direction, but
that's up to Rafael of course..

[toc] | [prev] | [next] | [standalone]

#1654137

On 31 May 2017 at 11:40, Peter Zijlstra <peterz@infradead.org> wrote:
> On Wed, May 24, 2017 at 11:00:51AM +0200, Vincent Guittot wrote:
>> schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs
>> tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks
>> are preempted by rt tasks and in this case util_avg reflects the remaining
>> capacity that is used by cfs tasks but not what cfs tasks want to use. In such
>> case, schedutil can select a lower OPP when cfs task runs whereas the CPU is
>> overloaded. In order to have a more accurate view of the utilization of the
>> CPU, we track the utilization that is used by RT tasks.
>> DL tasks are not taken into account as they have their own utilization
>> tracking mecanism.
>
> Well, the DL tracking is fairly pessimistic; it assumes all DL tasks
> will consume their total budget, which will rarely, if ever, happen.
>
> So I suspect it might well be worth it to also track DL activity for the
> purpose of compensating CFS.
>
> In fact, I don't think you particularly care about RT here, as anything
> !CFS that preempts it, including those interrupts you mentioned. Which
> gets us back to what rt_avg is.
>
>> We don't use rt_avg which doesn't have the same dynamic as PELT and which
>> can include IRQ time that are also accounted in cfs task utilization
>
> Well, if rt_avg includes IRQ time, then that IRQ time is not part of
> the task clock.

ah yes you're right.
I haven't noticed irq time was removed from the clock used for accounting PELT

>
>> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
>> ---
>>
>> If the changes are reasonnable, it might worth moving the PELT function in a
>> dedicated pelt.c file and the ugly
>> extern int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running);
>> in a pelt.h header
>>
>>
>>  kernel/sched/fair.c  | 21 +++++++++++++++++++++
>>  kernel/sched/rt.c    |  9 +++++++++
>>  kernel/sched/sched.h |  3 +++
>>  3 files changed, 33 insertions(+)
>
> Also, and I didn't check this, it is important that the windows are
> aligned if you want to sum the values.

yes. good point

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web