Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1405243 > unrolled thread

[PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide Morten Rasmussen <morten.rasmussen@arm.com> - 2016-05-23 13:00 +0200
    Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Mike Galbraith <mgalbraith@suse.de> - 2016-05-23 13:20 +0200
      Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Morten Rasmussen <morten.rasmussen@arm.com> - 2016-05-23 14:00 +0200
        Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Mike Galbraith <mgalbraith@suse.de> - 2016-05-23 15:10 +0200
          Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Morten Rasmussen <morten.rasmussen@arm.com> - 2016-05-23 16:20 +0200
            Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Mike Galbraith <mgalbraith@suse.de> - 2016-05-23 17:50 +0200
              Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Yuyang Du <yuyang.du@intel.com> - 2016-05-24 09:20 +0200
        Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in  wake_wide Yuyang Du <yuyang.du@intel.com> - 2016-05-24 09:10 +0200

#1405243 — [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

wake_wide() is based on task wakee_flips of the waker and the wakee to
decide whether an affine wakeup is desirable. On lightly loaded systems
the waker is frequently the idle task (pid=0) which can accumulate a lot
of wakee_flips in that scenario. It makes little sense to prevent affine
wakeups on an idle cpu due to the idle task wakee_flips, so it makes
more sense to ignore them in wake_wide().

cc: Ingo Molnar <mingo@redhat.com>
cc: Peter Zijlstra <peterz@infradead.org>

Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c49e25a..0fe3020 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5007,6 +5007,10 @@ static int wake_wide(struct task_struct *p)
 	unsigned int slave = p->wakee_flips;
 	int factor = this_cpu_read(sd_llc_size);
 
+	/* Don't let the idle task prevent affine wakeups */
+	if (is_idle_task(current))
+		return 0;
+
 	if (master < slave)
 		swap(master, slave);
 	if (slave < factor || master < slave * factor)
-- 
1.9.1

[toc] | [next] | [standalone]

#1405268 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, 2016-05-23 at 11:58 +0100, Morten Rasmussen wrote:
> wake_wide() is based on task wakee_flips of the waker and the wakee to
> decide whether an affine wakeup is desirable. On lightly loaded systems
> the waker is frequently the idle task (pid=0) which can accumulate a lot
> of wakee_flips in that scenario. It makes little sense to prevent affine
> wakeups on an idle cpu due to the idle task wakee_flips, so it makes
> more sense to ignore them in wake_wide().

You sure?  What's the difference between a task flipping enough to
warrant spreading the load, and an interrupt source doing the same? 
 I've both witnessed firsthand, and received user confirmation of this
very thing improving utilization.

> cc: Ingo Molnar <mingo@redhat.com>
> cc: Peter Zijlstra <peterz@infradead.org>
> 
> Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
> ---
>  kernel/sched/fair.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index c49e25a..0fe3020 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5007,6 +5007,10 @@ static int wake_wide(struct task_struct *p)
>  	unsigned int slave = p->wakee_flips;
>  	int factor = this_cpu_read(sd_llc_size);
>  
> +	/* Don't let the idle task prevent affine wakeups */
> +	if (is_idle_task(current))
> +		return 0;
> +
>  	if (master < slave)
>  		swap(master, slave);
>  	if (slave < factor || master < slave * factor)

[toc] | [prev] | [next] | [standalone]

#1405300 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, May 23, 2016 at 01:12:07PM +0200, Mike Galbraith wrote:
> On Mon, 2016-05-23 at 11:58 +0100, Morten Rasmussen wrote:
> > wake_wide() is based on task wakee_flips of the waker and the wakee to
> > decide whether an affine wakeup is desirable. On lightly loaded systems
> > the waker is frequently the idle task (pid=0) which can accumulate a lot
> > of wakee_flips in that scenario. It makes little sense to prevent affine
> > wakeups on an idle cpu due to the idle task wakee_flips, so it makes
> > more sense to ignore them in wake_wide().
> 
> You sure?  What's the difference between a task flipping enough to
> warrant spreading the load, and an interrupt source doing the same? 
>  I've both witnessed firsthand, and received user confirmation of this
> very thing improving utilization.

Right, I didn't consider the interrupt source scenario, my fault.

The problem then seems to be distinguishing truly idle and busy doing
interrupts. The issue that I observe is that wake_wide() likes pushing
tasks around in lightly scenarios which isn't desirable for power
management. Selecting the same cpu again may potentially let others
reach deeper C-state.

With that in mind I will if I can do better. Suggestions are welcome :-)

> 
> > cc: Ingo Molnar <mingo@redhat.com>
> > cc: Peter Zijlstra <peterz@infradead.org>
> > 
> > Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
> > ---
> >  kernel/sched/fair.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index c49e25a..0fe3020 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -5007,6 +5007,10 @@ static int wake_wide(struct task_struct *p)
> >  	unsigned int slave = p->wakee_flips;
> >  	int factor = this_cpu_read(sd_llc_size);
> >  
> > +	/* Don't let the idle task prevent affine wakeups */
> > +	if (is_idle_task(current))
> > +		return 0;
> > +
> >  	if (master < slave)
> >  		swap(master, slave);
> >  	if (slave < factor || master < slave * factor)

[toc] | [prev] | [next] | [standalone]

#1405352 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, 2016-05-23 at 13:00 +0100, Morten Rasmussen wrote:

> The problem then seems to be distinguishing truly idle and busy doing
> interrupts. The issue that I observe is that wake_wide() likes pushing
> tasks around in lightly scenarios which isn't desirable for power
> management. Selecting the same cpu again may potentially let others
> reach deeper C-state.
> 
> With that in mind I will if I can do better. Suggestions are welcome :-)

None here.  For big boxen that are highly idle, you'd likely want to
shut down nodes and consolidate load, but otoh, all that slows response
to burst, which I hate.  I prefer race to idle, let power gating do its
job.  If I had a server farm with enough capacity vs load variability
to worry about, I suspect I'd become highly interested in routing.

	-Mike

[toc] | [prev] | [next] | [standalone]

#1405406 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, May 23, 2016 at 03:00:46PM +0200, Mike Galbraith wrote:
> On Mon, 2016-05-23 at 13:00 +0100, Morten Rasmussen wrote:
> 
> > The problem then seems to be distinguishing truly idle and busy doing
> > interrupts. The issue that I observe is that wake_wide() likes pushing
> > tasks around in lightly scenarios which isn't desirable for power
> > management. Selecting the same cpu again may potentially let others
> > reach deeper C-state.
> > 
> > With that in mind I will if I can do better. Suggestions are welcome :-)
> 
> None here.  For big boxen that are highly idle, you'd likely want to
> shut down nodes and consolidate load, but otoh, all that slows response
> to burst, which I hate.  I prefer race to idle, let power gating do its
> job.  If I had a server farm with enough capacity vs load variability
> to worry about, I suspect I'd become highly interested in routing.

I don't disagree for systems of that scale, but at the other end of the
spectrum it is a single SoC we are trying squeeze the best possible
mileage out of. That implies optimizing for power gating to reach deeper
C-states when possible by consolidating idle-time and grouping
idle cpus. Migrating task unnecessarily isn't helping us in achieving
that, unfortunately :-(

[toc] | [prev] | [next] | [standalone]

#1405468 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, 2016-05-23 at 15:10 +0100, Morten Rasmussen wrote:
> On Mon, May 23, 2016 at 03:00:46PM +0200, Mike Galbraith wrote:
> > On Mon, 2016-05-23 at 13:00 +0100, Morten Rasmussen wrote:
> > 
> > > The problem then seems to be distinguishing truly idle and busy doing
> > > interrupts. The issue that I observe is that wake_wide() likes pushing
> > > tasks around in lightly scenarios which isn't desirable for power
> > > management. Selecting the same cpu again may potentially let others
> > > reach deeper C-state.
> > > 
> > > With that in mind I will if I can do better. Suggestions are welcome :-)
> > 
> > None here.  For big boxen that are highly idle, you'd likely want to
> > shut down nodes and consolidate load, but otoh, all that slows response
> > to burst, which I hate.  I prefer race to idle, let power gating do its
> > job.  If I had a server farm with enough capacity vs load variability
> > to worry about, I suspect I'd become highly interested in routing.
> 
> I don't disagree for systems of that scale, but at the other end of the
> spectrum it is a single SoC we are trying squeeze the best possible
> mileage out of. That implies optimizing for power gating to reach deeper
> C-states when possible by consolidating idle-time and grouping
> idle cpus. Migrating task unnecessarily isn't helping us in achieving
> that, unfortunately :-(

Yup, the goals are pretty much mutually exclusive.  For your goal, you
want more of an allocator like behavior, where stacking of tasks is bad
only once there's too much overlap (ie latency, defining is hard), and
allocation always has the same order (expand rightward or such for the
general case, adding little/big complexity for arm).  For mine, current
behavior is good, avoid stacking like the plague.

	-Mike

[toc] | [prev] | [next] | [standalone]

#1405905 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, May 23, 2016 at 05:42:20PM +0200, Mike Galbraith wrote:
> On Mon, 2016-05-23 at 15:10 +0100, Morten Rasmussen wrote:
> > On Mon, May 23, 2016 at 03:00:46PM +0200, Mike Galbraith wrote:
> > > On Mon, 2016-05-23 at 13:00 +0100, Morten Rasmussen wrote:
> > > 
> > > > The problem then seems to be distinguishing truly idle and busy doing
> > > > interrupts. The issue that I observe is that wake_wide() likes pushing
> > > > tasks around in lightly scenarios which isn't desirable for power
> > > > management. Selecting the same cpu again may potentially let others
> > > > reach deeper C-state.
> > > > 
> > > > With that in mind I will if I can do better. Suggestions are welcome :-)
> > > 
> > > None here.  For big boxen that are highly idle, you'd likely want to
> > > shut down nodes and consolidate load, but otoh, all that slows response
> > > to burst, which I hate.  I prefer race to idle, let power gating do its
> > > job.  If I had a server farm with enough capacity vs load variability
> > > to worry about, I suspect I'd become highly interested in routing.
> > 
> > I don't disagree for systems of that scale, but at the other end of the
> > spectrum it is a single SoC we are trying squeeze the best possible
> > mileage out of. That implies optimizing for power gating to reach deeper
> > C-states when possible by consolidating idle-time and grouping
> > idle cpus. Migrating task unnecessarily isn't helping us in achieving
> > that, unfortunately :-(
> 
> Yup, the goals are pretty much mutually exclusive.  For your goal, you
> want more of an allocator like behavior, where stacking of tasks is bad
> only once there's too much overlap (ie latency, defining is hard), and
> allocation always has the same order (expand rightward or such for the
> general case, adding little/big complexity for arm).  For mine, current
> behavior is good, avoid stacking like the plague.

I'd be happy to have a switch to either one goal.

[toc] | [prev] | [next] | [standalone]

#1405895 — Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

On Mon, May 23, 2016 at 01:00:10PM +0100, Morten Rasmussen wrote:
> On Mon, May 23, 2016 at 01:12:07PM +0200, Mike Galbraith wrote:
> > On Mon, 2016-05-23 at 11:58 +0100, Morten Rasmussen wrote:
> > > wake_wide() is based on task wakee_flips of the waker and the wakee to
> > > decide whether an affine wakeup is desirable. On lightly loaded systems
> > > the waker is frequently the idle task (pid=0) which can accumulate a lot
> > > of wakee_flips in that scenario. It makes little sense to prevent affine
> > > wakeups on an idle cpu due to the idle task wakee_flips, so it makes
> > > more sense to ignore them in wake_wide().
> > 
> > You sure?  What's the difference between a task flipping enough to
> > warrant spreading the load, and an interrupt source doing the same? 
> >  I've both witnessed firsthand, and received user confirmation of this
> > very thing improving utilization.
> 
> Right, I didn't consider the interrupt source scenario, my fault.
> 
> The problem then seems to be distinguishing truly idle and busy doing
> interrupts. The issue that I observe is that wake_wide() likes pushing
> tasks around in lightly scenarios which isn't desirable for power
> management. Selecting the same cpu again may potentially let others
> reach deeper C-state.
> 
> With that in mind I will if I can do better. Suggestions are welcome :-)
 
On mobile, the factor is as small as 2 to 4, may easily be exceeded,
so decay at HZ may be too slow.

> > 
> > > cc: Ingo Molnar <mingo@redhat.com>
> > > cc: Peter Zijlstra <peterz@infradead.org>
> > > 
> > > Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
> > > ---
> > >  kernel/sched/fair.c | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > > 
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index c49e25a..0fe3020 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -5007,6 +5007,10 @@ static int wake_wide(struct task_struct *p)
> > >  	unsigned int slave = p->wakee_flips;
> > >  	int factor = this_cpu_read(sd_llc_size);
> > >  
> > > +	/* Don't let the idle task prevent affine wakeups */
> > > +	if (is_idle_task(current))
> > > +		return 0;
> > > +
> > >  	if (master < slave)
> > >  		swap(master, slave);
> > >  	if (slave < factor || master < slave * factor)

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web