Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1342364 > unrolled thread

Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted bandwidth

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted  bandwidth Peter Zijlstra <peterz@infradead.org> - 2016-02-24 20:20 +0100
    Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted  bandwidth luca abeni <luca.abeni@unitn.it> - 2016-02-24 22:50 +0100
      Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted  bandwidth Peter Zijlstra <peterz@infradead.org> - 2016-02-25 09:00 +0100
    Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted  bandwidth Juri Lelli <juri.lelli@arm.com> - 2016-02-25 11:10 +0100
      Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted  bandwidth Peter Zijlstra <peterz@infradead.org> - 2016-02-25 11:30 +0100

#1342364 — Re: [PATCH 1/2] sched/deadline: add per rq tracking of admitted bandwidth

On Fri, Feb 12, 2016 at 06:05:30PM +0100, Peter Zijlstra wrote:
> Having two separate means of accounting this also feels more fragile
> than one would want.
> 
> Let me think a bit about this.

I think there's a fundamental problem that makes the whole notion of
per-rq accounting 'impossible'.

On hot-unplug we only migrate runnable tasks, all blocked tasks remain
on the dead cpu. This would very much include their bandwidth
requirements.

This means that between a hot-unplug and the moment that _all_ those
blocked tasks have ran at least once, the sum of online bandwidth
doesn't match and we can get into admission trouble (same for GRUB,
which can also use per-rq bw like this).

The main problem is that there is no real way to find blocked tasks;
currently the only way is to iterate _all_ tasks and filter on
task_cpu().

We could of course add a blocked tree/list for deadline tasks, to
explicitly keep track of all these; this would allow migrating blocked
tasks on hotplug and avoid the real ugly I think. But I've not tried
yet.

[toc] | [next] | [standalone]

#1342461

Hi,

On Wed, 24 Feb 2016 20:17:52 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Fri, Feb 12, 2016 at 06:05:30PM +0100, Peter Zijlstra wrote:
> > Having two separate means of accounting this also feels more fragile
> > than one would want.
> > 
> > Let me think a bit about this.
> 
> I think there's a fundamental problem that makes the whole notion of
> per-rq accounting 'impossible'.
> 
> On hot-unplug we only migrate runnable tasks, all blocked tasks remain
> on the dead cpu. This would very much include their bandwidth
> requirements.
> 
> This means that between a hot-unplug and the moment that _all_ those
> blocked tasks have ran at least once, the sum of online bandwidth
> doesn't match and we can get into admission trouble (same for GRUB,
> which can also use per-rq bw like this).

After Juri's patch and emails, I tried to think about the CPU
hot-(un)plugging issues, and to check if/how they affect GRUB
reclaiming...

I arrived to the conclusion that for GRUB this is not a problem (but,
as usual, I might be wrong): GRUB just needs to track the per-runqueue
active/inactive utilization, and is not badly affected by the fact that
inactive utilization is migrated "too late" (when a task wakes up
instead of when the CPU goes offline). This is because GRUB does not
care about "global" utilization, but considers the various runqueues in
isolation (there is a flavor of the m-grub algorithm that uses global
inactive utilization, but it is not implemented by the patches I
submitted).
In other words: Juri's patch uses per-runqueue utilizations to re-build
the global utilization, while GRUB does not care if the sum of the
"active utilizations" match with the utilization used for admission
control.

I still have to check some details, and to run some more tests with CPU
hot-(un)plug (and this is why I did not send a v2 of the reclaiming RFC
yet)... In particular, I need to check what happens if the "inactive
timer" fires when the CPU on which the task was running is already
offline.



			Thanks,
				Luca

[toc] | [prev] | [next] | [standalone]

#1342928

On Wed, Feb 24, 2016 at 10:46:43PM +0100, luca abeni wrote:
> 
> I arrived to the conclusion that for GRUB this is not a problem (but,
> as usual, I might be wrong): GRUB just needs to track the per-runqueue
> active/inactive utilization,

Ah! indeed, my bad.

[toc] | [prev] | [next] | [standalone]

#1343074

Hi Peter,

On 24/02/16 20:17, Peter Zijlstra wrote:
> On Fri, Feb 12, 2016 at 06:05:30PM +0100, Peter Zijlstra wrote:
> > Having two separate means of accounting this also feels more fragile
> > than one would want.
> > 
> > Let me think a bit about this.
> 
> I think there's a fundamental problem that makes the whole notion of
> per-rq accounting 'impossible'.
>
> On hot-unplug we only migrate runnable tasks, all blocked tasks remain
> on the dead cpu. This would very much include their bandwidth
> requirements.
> 
> This means that between a hot-unplug and the moment that _all_ those
> blocked tasks have ran at least once, the sum of online bandwidth
> doesn't match and we can get into admission trouble (same for GRUB,
> which can also use per-rq bw like this).
> 
> The main problem is that there is no real way to find blocked tasks;
> currently the only way is to iterate _all_ tasks and filter on
> task_cpu().
> 
> We could of course add a blocked tree/list for deadline tasks, to
> explicitly keep track of all these; this would allow migrating blocked
> tasks on hotplug and avoid the real ugly I think. But I've not tried
> yet.
> 

Argh, this makes lot of sense to me. I've actually pondered a tree/list
solution, but then decided to try the cumulative approach because it
looked nicer. But it contains holes, I'm afraid. As Luca already said,
GRUB shouldn't have these problems though.

I'll try and see what introducting a list of blocked/throttled deadline
tasks means, considering also the interaction with cpusets and such.
Maybe it's simpler than it seems.

I'm not sure this will come anytime soon, unfortunately. I'm almost 100%
on the sched-freq/schedutil discussion these days.

Anyway, do you also think that what we want to solve the root domain
issue is something based on rq_online/offline and per-rq information?
Everything else that I tried or thought of was broken/more horrible. :-/

Best,

- Juri

[toc] | [prev] | [next] | [standalone]

#1343088

On Thu, Feb 25, 2016 at 10:07:06AM +0000, Juri Lelli wrote:
> Argh, this makes lot of sense to me. I've actually pondered a tree/list
> solution, but then decided to try the cumulative approach because it
> looked nicer. But it contains holes, I'm afraid. As Luca already said,
> GRUB shouldn't have these problems though.
> 
> I'll try and see what introducting a list of blocked/throttled deadline
> tasks means, considering also the interaction with cpusets and such.
> Maybe it's simpler than it seems.
> 
> I'm not sure this will come anytime soon, unfortunately. I'm almost 100%
> on the sched-freq/schedutil discussion these days.

Just skip sleep and write them when its dark outside :-)

> Anyway, do you also think that what we want to solve the root domain
> issue is something based on rq_online/offline and per-rq information?
> Everything else that I tried or thought of was broken/more horrible. :-/

I was still trying to get my head around this, the above was my
suggestion to the per-rq state, but I've not thought hard on alternative
approaches to the root_domain issue.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web