Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1647849 > unrolled thread

[PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

Back to article view | Back to linux.kernel

Contents

  [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection Juri Lelli <juri.lelli@arm.com> - 2017-05-23 11:00 +0200
    [PATCH RFC 1/8] sched/cpufreq_schedutil: make use of DEADLINE utilization signal Juri Lelli <juri.lelli@arm.com> - 2017-05-23 11:00 +0200
    [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals Juri Lelli <juri.lelli@arm.com> - 2017-05-23 11:00 +0200
      Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization  signals Peter Zijlstra <peterz@infradead.org> - 2017-05-23 21:10 +0200
        Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization  signals Juri Lelli <juri.lelli@arm.com> - 2017-05-24 11:10 +0200
      Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization  signals Peter Zijlstra <peterz@infradead.org> - 2017-05-23 21:40 +0200
        Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2017-05-24 01:40 +0200
          Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization  signals Peter Zijlstra <peterz@infradead.org> - 2017-05-24 09:10 +0200
            Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization  signals Juri Lelli <juri.lelli@arm.com> - 2017-05-24 11:10 +0200
    Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Peter Zijlstra <peterz@infradead.org> - 2017-05-23 22:40 +0200
    Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Peter Zijlstra <peterz@infradead.org> - 2017-05-24 00:00 +0200
      Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Juri Lelli <juri.lelli@arm.com> - 2017-05-24 11:30 +0200
        Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Peter Zijlstra <peterz@infradead.org> - 2017-05-24 11:50 +0200
          Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Juri Lelli <juri.lelli@arm.com> - 2017-05-24 12:00 +0200
            Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Peter Zijlstra <peterz@infradead.org> - 2017-05-24 13:40 +0200
        Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Luca Abeni <luca.abeni@santannapisa.it> - 2017-05-24 12:10 +0200
          Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP  selection Peter Zijlstra <peterz@infradead.org> - 2017-05-24 13:40 +0200

#1647849 — [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

Hi,

this RFC set implements frequency/cpu invariance and OPP selection for
SCHED_DEADLINE. The set has been slightly tested on a Juno platform. The
current incarnation of the patches stems both from previous RFD[1] review
comments and discussion at OSPM-summit[2], during which we seemed to agree
that:

 - we probably want to use running_bw (instead of this_bw), as it is less
   pessimistic (we should save more energy)
 - special kworker hack seems acceptable as a mid term solution to foster
   further SCHED_DEADLINE/schedutil development/adoption

A point that is still very much up for discussion (more that the others :) is
how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
grub_reclaim(), as the function already scales their reservation runtime
considering other reservations and maximum bandwidth a CPU has to offer.
However, for normal !RECLAIM tasks multiple things can be implemented which
seem to make sense:

 - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
   by AC
 - go to max as soon as a normal task in enqueued: this because dimensioning of
   parameters is usually done at max OPP/biggest CPU and normal task assume
   that this is always the condition when they run
 - scale runtime acconding to current frequency and max CPU capacity: this is
   what this set is currently implementing

Opinions?

The set is based on tip/sched/core as of today (a9e7f6544b9c) plus some
schedutil fixes coming from linux-pm/linux-next and Luca's "CPU reclaiming for
SCHED_DEADLINE" [3].

Patches high level description:

 o [01-02]/08 add the necessary links to start accounting DEADLINE contribution
              to OPP selection 
 o 03/08      it's a temporary solution to make possible (on ARM) to change
              frequency for DEADLINE tasks (that would possibly delay the SCHED_FIFO
              worker kthread); proper solution would be to be able to issue frequency
              transition from an atomic ctx
 o [04-05]/08 it's a schedutil change that copes with the fact that DEADLINE
              doesn't require periodic OPP selection triggering point
 o [06-07]/08 make arch_scale_{freq,cpu}_capacity() function available on !CONFIG_SMP
              configurations too
 o 08/08      implements frequency/cpu invariance for tasks' reservation
              parameters; which basically means that we implement GRUB-PA [4]

Changes w.r.t. RFD:

 - use grub_reclaim for RECLAIM and scale freq/cpu for !RECLAIM
 - discard CFS contribution only, after TICK_NSEC
 - added patches 06 and 07 to fix !CONFIG_SMP builds

Please have a look. Feedback and comments are, as usual, more than welcome.

In case you would like to test this out:

 git://linux-arm.org/linux-jl.git upstream/deadline/freq-rfc

Best,

- Juri

[1] http://marc.info/?l=linux-kernel&m=149036457909119&w=2
[2] http://retis.sssup.it/ospm-summit/program.html
    https://lwn.net/Articles/721573/
[3] http://marc.info/?l=linux-kernel&m=149513848804404
[4] C. Scordino, G. Lipari, A Resource Reservation Algorithm for Power-Aware
    Scheduling of Periodic and Aperiodic Real-Time Tasks, IEEE Transactions
    on Computers, December 2006.

Juri Lelli (8):
  sched/cpufreq_schedutil: make use of DEADLINE utilization signal
  sched/deadline: move cpu frequency selection triggering points
  sched/cpufreq_schedutil: make worker kthread be SCHED_DEADLINE
  sched/cpufreq_schedutil: split utilization signals
  sched/cpufreq_schedutil: always consider all CPUs when deciding next
    freq
  sched/sched.h: remove sd arch_scale_freq_capacity parameter
  sched/sched.h: move arch_scale_{freq,cpu}_capacity outside CONFIG_SMP
  sched/deadline: make bandwidth enforcement scale-invariant

 include/linux/sched.h            |  1 +
 include/linux/sched/cpufreq.h    |  2 --
 include/linux/sched/topology.h   | 12 ++++----
 include/uapi/linux/sched.h       |  1 +
 kernel/sched/core.c              | 19 ++++++++++--
 kernel/sched/cpufreq_schedutil.c | 62 ++++++++++++++++++++++++----------------
 kernel/sched/deadline.c          | 39 ++++++++++++++++++++-----
 kernel/sched/fair.c              |  4 +--
 kernel/sched/sched.h             | 27 +++++++++++++----
 9 files changed, 116 insertions(+), 51 deletions(-)

-- 
2.11.0

[toc] | [next] | [standalone]

#1647850 — [PATCH RFC 1/8] sched/cpufreq_schedutil: make use of DEADLINE utilization signal

SCHED_DEADLINE tracks active utilization signal with a per dl_rq
variable named running_bw.

Make use of that to drive cpu frequency selection: add up FAIR and
DEADLINE contribution to get the required CPU capacity to handle both
requirements (while RT still selects max frequency).

Co-authored-by: Claudio Scordino <claudio@evidence.eu.com>
Signed-off-by: Juri Lelli <juri.lelli@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Luca Abeni <luca.abeni@santannapisa.it>
---
 include/linux/sched/cpufreq.h    |  2 --
 kernel/sched/cpufreq_schedutil.c | 13 ++++++-------
 2 files changed, 6 insertions(+), 9 deletions(-)

diff --git a/include/linux/sched/cpufreq.h b/include/linux/sched/cpufreq.h
index d2be2ccbb372..39640bb3a8ee 100644
--- a/include/linux/sched/cpufreq.h
+++ b/include/linux/sched/cpufreq.h
@@ -11,8 +11,6 @@
 #define SCHED_CPUFREQ_DL	(1U << 1)
 #define SCHED_CPUFREQ_IOWAIT	(1U << 2)
 
-#define SCHED_CPUFREQ_RT_DL	(SCHED_CPUFREQ_RT | SCHED_CPUFREQ_DL)
-
 #ifdef CONFIG_CPU_FREQ
 struct update_util_data {
        void (*func)(struct update_util_data *data, u64 time, unsigned int flags);
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index 622eed1b7658..7f1913e265d1 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -160,12 +160,11 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy,
 static void sugov_get_util(unsigned long *util, unsigned long *max)
 {
 	struct rq *rq = this_rq();
-	unsigned long cfs_max;
+	unsigned long dl_util = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
 
-	cfs_max = arch_scale_cpu_capacity(NULL, smp_processor_id());
+	*max = arch_scale_cpu_capacity(NULL, smp_processor_id());
 
-	*util = min(rq->cfs.avg.util_avg, cfs_max);
-	*max = cfs_max;
+	*util = min(rq->cfs.avg.util_avg + dl_util, *max);
 }
 
 static void sugov_set_iowait_boost(struct sugov_cpu *sg_cpu, u64 time,
@@ -229,7 +228,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time,
 
 	busy = sugov_cpu_is_busy(sg_cpu);
 
-	if (flags & SCHED_CPUFREQ_RT_DL) {
+	if (flags & SCHED_CPUFREQ_RT) {
 		next_f = policy->cpuinfo.max_freq;
 	} else {
 		sugov_get_util(&util, &max);
@@ -269,7 +268,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time)
 			j_sg_cpu->iowait_boost = 0;
 			continue;
 		}
-		if (j_sg_cpu->flags & SCHED_CPUFREQ_RT_DL)
+		if (j_sg_cpu->flags & SCHED_CPUFREQ_RT)
 			return policy->cpuinfo.max_freq;
 
 		j_util = j_sg_cpu->util;
@@ -305,7 +304,7 @@ static void sugov_update_shared(struct update_util_data *hook, u64 time,
 	sg_cpu->last_update = time;
 
 	if (sugov_should_update_freq(sg_policy, time)) {
-		if (flags & SCHED_CPUFREQ_RT_DL)
+		if (flags & SCHED_CPUFREQ_RT)
 			next_f = sg_policy->policy->cpuinfo.max_freq;
 		else
 			next_f = sugov_next_freq_shared(sg_cpu, time);
-- 
2.11.0

[toc] | [prev] | [next] | [standalone]

#1647852 — [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

To be able to treat utilization signals of different scheduling classes
in different ways (e.g., CFS signal might be stale while DEADLINE signal
is never stale by design) we need to split sugov_cpu::util signal in two:
util_cfs and util_dl.

This patch does that by also changing sugov_get_util() parameter list.
After this change aggregation of the different signals has to be performed
by sugov_get_util() users (so that they can decide what to do with the
different signals).

Suggested-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Juri Lelli <juri.lelli@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Luca Abeni <luca.abeni@santannapisa.it>
Cc: Claudio Scordino <claudio@evidence.eu.com>
---
 kernel/sched/cpufreq_schedutil.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index 1508109c7f19..f930cec4c3d4 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -58,7 +58,8 @@ struct sugov_cpu {
 	u64 last_update;
 
 	/* The fields below are only needed when sharing a policy. */
-	unsigned long util;
+	unsigned long util_cfs;
+	unsigned long util_dl;
 	unsigned long max;
 	unsigned int flags;
 
@@ -157,14 +158,13 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy,
 	return cpufreq_driver_resolve_freq(policy, freq);
 }
 
-static void sugov_get_util(unsigned long *util, unsigned long *max)
+static void sugov_get_util(struct sugov_cpu *sg_cpu)
 {
 	struct rq *rq = this_rq();
-	unsigned long dl_util = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
 
-	*max = arch_scale_cpu_capacity(NULL, smp_processor_id());
-
-	*util = min(rq->cfs.avg.util_avg + dl_util, *max);
+	sg_cpu->max = arch_scale_cpu_capacity(NULL, smp_processor_id());
+	sg_cpu->util_cfs = rq->cfs.avg.util_avg;
+	sg_cpu->util_dl = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
 }
 
 static void sugov_set_iowait_boost(struct sugov_cpu *sg_cpu, u64 time,
@@ -231,7 +231,9 @@ static void sugov_update_single(struct update_util_data *hook, u64 time,
 	if (flags & SCHED_CPUFREQ_RT) {
 		next_f = policy->cpuinfo.max_freq;
 	} else {
-		sugov_get_util(&util, &max);
+		sugov_get_util(sg_cpu);
+		max = sg_cpu->max;
+		util = min(sg_cpu->util_cfs + sg_cpu->util_dl, max);
 		sugov_iowait_boost(sg_cpu, &util, &max);
 		next_f = get_next_freq(sg_policy, util, max);
 		/*
@@ -271,8 +273,8 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time)
 		if (j_sg_cpu->flags & SCHED_CPUFREQ_RT)
 			return policy->cpuinfo.max_freq;
 
-		j_util = j_sg_cpu->util;
 		j_max = j_sg_cpu->max;
+		j_util = min(j_sg_cpu->util_cfs + j_sg_cpu->util_dl, j_max);
 		if (j_util * max > j_max * util) {
 			util = j_util;
 			max = j_max;
@@ -289,15 +291,12 @@ static void sugov_update_shared(struct update_util_data *hook, u64 time,
 {
 	struct sugov_cpu *sg_cpu = container_of(hook, struct sugov_cpu, update_util);
 	struct sugov_policy *sg_policy = sg_cpu->sg_policy;
-	unsigned long util, max;
 	unsigned int next_f;
 
-	sugov_get_util(&util, &max);
 
 	raw_spin_lock(&sg_policy->update_lock);
 
-	sg_cpu->util = util;
-	sg_cpu->max = max;
+	sugov_get_util(sg_cpu);
 	sg_cpu->flags = flags;
 
 	sugov_set_iowait_boost(sg_cpu, time, flags);
-- 
2.11.0

[toc] | [prev] | [next] | [standalone]

#1648353 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> @@ -157,14 +158,13 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy,
>  	return cpufreq_driver_resolve_freq(policy, freq);
>  }
>  
> -static void sugov_get_util(unsigned long *util, unsigned long *max)
> +static void sugov_get_util(struct sugov_cpu *sg_cpu)
>  {
>  	struct rq *rq = this_rq();
> -	unsigned long dl_util = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
>  
> -	*max = arch_scale_cpu_capacity(NULL, smp_processor_id());
> -
> -	*util = min(rq->cfs.avg.util_avg + dl_util, *max);
> +	sg_cpu->max = arch_scale_cpu_capacity(NULL, smp_processor_id());
> +	sg_cpu->util_cfs = rq->cfs.avg.util_avg;
> +	sg_cpu->util_dl = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
>  }

Luca just introduced a nice BW_SHIFT for that '20' thing.

[toc] | [prev] | [next] | [standalone]

#1649358 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

Hi,

On 23/05/17 21:04, Peter Zijlstra wrote:
> On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> > @@ -157,14 +158,13 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy,
> >  	return cpufreq_driver_resolve_freq(policy, freq);
> >  }
> >  
> > -static void sugov_get_util(unsigned long *util, unsigned long *max)
> > +static void sugov_get_util(struct sugov_cpu *sg_cpu)
> >  {
> >  	struct rq *rq = this_rq();
> > -	unsigned long dl_util = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
> >  
> > -	*max = arch_scale_cpu_capacity(NULL, smp_processor_id());
> > -
> > -	*util = min(rq->cfs.avg.util_avg + dl_util, *max);
> > +	sg_cpu->max = arch_scale_cpu_capacity(NULL, smp_processor_id());
> > +	sg_cpu->util_cfs = rq->cfs.avg.util_avg;
> > +	sg_cpu->util_dl = (rq->dl.running_bw * SCHED_CAPACITY_SCALE) >> 20;
> >  }
> 
> Luca just introduced a nice BW_SHIFT for that '20' thing.

Right, will use that.

Thanks,

- Juri

[toc] | [prev] | [next] | [standalone]

#1648370 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> To be able to treat utilization signals of different scheduling classes
> in different ways (e.g., CFS signal might be stale while DEADLINE signal
> is never stale by design) we need to split sugov_cpu::util signal in two:
> util_cfs and util_dl.
> 
> This patch does that by also changing sugov_get_util() parameter list.
> After this change aggregation of the different signals has to be performed
> by sugov_get_util() users (so that they can decide what to do with the
> different signals).

So what I don't see this patch doing; and I don't remember if cpufreq is
ready for this at all, is set the util_dl as min/guaranteed freq and
util_cfs+util_dl as requested freq.

[toc] | [prev] | [next] | [standalone]

#1649002 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

On Tuesday, May 23, 2017 09:29:27 PM Peter Zijlstra wrote:
> On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> > To be able to treat utilization signals of different scheduling classes
> > in different ways (e.g., CFS signal might be stale while DEADLINE signal
> > is never stale by design) we need to split sugov_cpu::util signal in two:
> > util_cfs and util_dl.
> > 
> > This patch does that by also changing sugov_get_util() parameter list.
> > After this change aggregation of the different signals has to be performed
> > by sugov_get_util() users (so that they can decide what to do with the
> > different signals).
> 
> So what I don't see this patch doing; and I don't remember if cpufreq is
> ready for this at all, is set the util_dl as min/guaranteed freq and
> util_cfs+util_dl as requested freq.

I'm totally unsure what you mean here.

cpufreq doesn't have a "guaranteed frequency" concept of any sort right now.

[toc] | [prev] | [next] | [standalone]

#1649169 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

On Wed, May 24, 2017 at 01:30:36AM +0200, Rafael J. Wysocki wrote:
> On Tuesday, May 23, 2017 09:29:27 PM Peter Zijlstra wrote:
> > On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> > > To be able to treat utilization signals of different scheduling classes
> > > in different ways (e.g., CFS signal might be stale while DEADLINE signal
> > > is never stale by design) we need to split sugov_cpu::util signal in two:
> > > util_cfs and util_dl.
> > > 
> > > This patch does that by also changing sugov_get_util() parameter list.
> > > After this change aggregation of the different signals has to be performed
> > > by sugov_get_util() users (so that they can decide what to do with the
> > > different signals).
> > 
> > So what I don't see this patch doing; and I don't remember if cpufreq is
> > ready for this at all, is set the util_dl as min/guaranteed freq and
> > util_cfs+util_dl as requested freq.
> 
> I'm totally unsure what you mean here.

I was thinking of the CPPC/HWP stuff, where you can set different
frequencies with different levels of guarantees.

We'd want to set util_dl as the minimum (guaranteed) performance, and
util_dl + util_cfs as the desired performance level.

> cpufreq doesn't have a "guaranteed frequency" concept of any sort right now.

I was afraid of that ;-) I think we want a comment in the code stating
that this is the desired goal though. Then once cpufreq is ready to deal
with it we can change it..

[toc] | [prev] | [next] | [standalone]

#1649349 — Re: [PATCH RFC 4/8] sched/cpufreq_schedutil: split utilization signals

Hi,

On 24/05/17 09:01, Peter Zijlstra wrote:
> On Wed, May 24, 2017 at 01:30:36AM +0200, Rafael J. Wysocki wrote:
> > On Tuesday, May 23, 2017 09:29:27 PM Peter Zijlstra wrote:
> > > On Tue, May 23, 2017 at 09:53:47AM +0100, Juri Lelli wrote:
> > > > To be able to treat utilization signals of different scheduling classes
> > > > in different ways (e.g., CFS signal might be stale while DEADLINE signal
> > > > is never stale by design) we need to split sugov_cpu::util signal in two:
> > > > util_cfs and util_dl.
> > > > 
> > > > This patch does that by also changing sugov_get_util() parameter list.
> > > > After this change aggregation of the different signals has to be performed
> > > > by sugov_get_util() users (so that they can decide what to do with the
> > > > different signals).
> > > 
> > > So what I don't see this patch doing; and I don't remember if cpufreq is
> > > ready for this at all, is set the util_dl as min/guaranteed freq and
> > > util_cfs+util_dl as requested freq.
> > 
> > I'm totally unsure what you mean here.
> 
> I was thinking of the CPPC/HWP stuff, where you can set different
> frequencies with different levels of guarantees.
> 
> We'd want to set util_dl as the minimum (guaranteed) performance, and
> util_dl + util_cfs as the desired performance level.
> 
> > cpufreq doesn't have a "guaranteed frequency" concept of any sort right now.
> 
> I was afraid of that ;-) I think we want a comment in the code stating
> that this is the desired goal though. Then once cpufreq is ready to deal
> with it we can change it..

Sure, I can add that in next version.

Thanks,

- Juri

[toc] | [prev] | [next] | [standalone]

#1648549 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Tue, May 23, 2017 at 10:23:21PM +0200, Peter Zijlstra wrote:
> On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:
> 
> > A point that is still very much up for discussion (more that the others :) is
> > how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
> > grub_reclaim(), as the function already scales their reservation runtime
> > considering other reservations and maximum bandwidth a CPU has to offer.
> > However, for normal !RECLAIM tasks multiple things can be implemented which
> > seem to make sense:
> > 
> >  - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
> >    by AC
> >  - go to max as soon as a normal task in enqueued: this because dimensioning of
> >    parameters is usually done at max OPP/biggest CPU and normal task assume
> >    that this is always the condition when they run
> >  - scale runtime acconding to current frequency and max CPU capacity: this is
> >    what this set is currently implementing
> > 
> > Opinions?
> 
> 
> So I'm terribly confused...
> 
> By using the active bandwidth to select frequency we effectively
> reduce idle time (to 0 if we had infinite granular frequency steps and
> no margins).

When all DL tasks consume their full reservation.

> So !RECLAIM works as expected. They get the time they reserved, since
> that was taken into account by active bandwidth.
> 
> And RECLAIM works, since that only promises to (re)distribute idle time,
> and if there is none that is an easy task.

And if they don't, there will thus be some idle time to redistribute and
that, again, still works as expected.

[toc] | [prev] | [next] | [standalone]

#1648873 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:

> A point that is still very much up for discussion (more that the others :) is
> how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
> grub_reclaim(), as the function already scales their reservation runtime
> considering other reservations and maximum bandwidth a CPU has to offer.
> However, for normal !RECLAIM tasks multiple things can be implemented which
> seem to make sense:
> 
>  - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
>    by AC
>  - go to max as soon as a normal task in enqueued: this because dimensioning of
>    parameters is usually done at max OPP/biggest CPU and normal task assume
>    that this is always the condition when they run
>  - scale runtime acconding to current frequency and max CPU capacity: this is
>    what this set is currently implementing
> 
> Opinions?


So I'm terribly confused...

By using the active bandwidth to select frequency we effectively
reduce idle time (to 0 if we had infinite granular frequency steps and
no margins).

So !RECLAIM works as expected. They get the time they reserved, since
that was taken into account by active bandwidth.

And RECLAIM works, since that only promises to (re)distribute idle time,
and if there is none that is an easy task.


Where is the problem?

[toc] | [prev] | [next] | [standalone]

#1649369 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

Hi,

On 23/05/17 22:23, Peter Zijlstra wrote:
> On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:
> 
> > A point that is still very much up for discussion (more that the others :) is
> > how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
> > grub_reclaim(), as the function already scales their reservation runtime
> > considering other reservations and maximum bandwidth a CPU has to offer.
> > However, for normal !RECLAIM tasks multiple things can be implemented which
> > seem to make sense:
> > 
> >  - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
> >    by AC
> >  - go to max as soon as a normal task in enqueued: this because dimensioning of
> >    parameters is usually done at max OPP/biggest CPU and normal task assume
> >    that this is always the condition when they run
> >  - scale runtime acconding to current frequency and max CPU capacity: this is
> >    what this set is currently implementing
> > 
> > Opinions?
> 
> 
> So I'm terribly confused...
> 
> By using the active bandwidth to select frequency we effectively
> reduce idle time (to 0 if we had infinite granular frequency steps and
> no margins).
> 
> So !RECLAIM works as expected. They get the time they reserved, since
> that was taken into account by active bandwidth.
> 

This was my impression as well, but Luca (and please Luca correct me if
I misunderstood your point) argued (in an off-line discussion ahead of
this posting) that !reclaim tasks might not be interested in reclaiming
*at all*. Since scaling frequency down is another way of effectively
reclaiming unused bandwidth (the other being sharing unused bandwidth
among reservations while keeping frequency at max), !reclaim tasks could
not be interested in frequency scaling (my first point above) or require
frequency to be always at max (second point above).

Does this help claryfing a bit? :)

This said however, I'd personally be inclined to go with option 3 above,
which is what this set is currently implementing.

> And RECLAIM works, since that only promises to (re)distribute idle time,
> and if there is none that is an easy task.
> 

Right.

Thanks,

- Juri

[toc] | [prev] | [next] | [standalone]

#1649384 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Wed, May 24, 2017 at 10:25:05AM +0100, Juri Lelli wrote:
> Hi,
> 
> On 23/05/17 22:23, Peter Zijlstra wrote:
> > On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:
> > 
> > > A point that is still very much up for discussion (more that the others :) is
> > > how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
> > > grub_reclaim(), as the function already scales their reservation runtime
> > > considering other reservations and maximum bandwidth a CPU has to offer.
> > > However, for normal !RECLAIM tasks multiple things can be implemented which
> > > seem to make sense:
> > > 
> > >  - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
> > >    by AC
> > >  - go to max as soon as a normal task in enqueued: this because dimensioning of
> > >    parameters is usually done at max OPP/biggest CPU and normal task assume
> > >    that this is always the condition when they run
> > >  - scale runtime acconding to current frequency and max CPU capacity: this is
> > >    what this set is currently implementing
> > > 
> > > Opinions?
> > 
> > 
> > So I'm terribly confused...
> > 
> > By using the active bandwidth to select frequency we effectively
> > reduce idle time (to 0 if we had infinite granular frequency steps and
> > no margins).
> > 
> > So !RECLAIM works as expected. They get the time they reserved, since
> > that was taken into account by active bandwidth.
> > 
> 
> This was my impression as well, but Luca (and please Luca correct me if
> I misunderstood your point) argued (in an off-line discussion ahead of
> this posting) that !reclaim tasks might not be interested in reclaiming
> *at all*. Since scaling frequency down is another way of effectively
> reclaiming unused bandwidth (the other being sharing unused bandwidth
> among reservations while keeping frequency at max), !reclaim tasks could
> not be interested in frequency scaling (my first point above) or require
> frequency to be always at max (second point above).
> 
> Does this help claryfing a bit? :)

No ;-) As you said, confusion++.

A !RECLAIM task doesn't care (cannot care, should not care etc..) about
any bandwidth not allocated to itself. Therefore it should/must/etc..
not have any opinion on what we do with 'spare' bandwidth.

[toc] | [prev] | [next] | [standalone]

#1649401 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On 24/05/17 11:41, Peter Zijlstra wrote:
> On Wed, May 24, 2017 at 10:25:05AM +0100, Juri Lelli wrote:
> > Hi,
> > 
> > On 23/05/17 22:23, Peter Zijlstra wrote:
> > > On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:
> > > 
> > > > A point that is still very much up for discussion (more that the others :) is
> > > > how we implement frequency/cpu scaling. SCHED_FLAG_RECLAIM tasks only need
> > > > grub_reclaim(), as the function already scales their reservation runtime
> > > > considering other reservations and maximum bandwidth a CPU has to offer.
> > > > However, for normal !RECLAIM tasks multiple things can be implemented which
> > > > seem to make sense:
> > > > 
> > > >  - don't scale at all: normal tasks will only get a % of CPU _time_ as granted
> > > >    by AC
> > > >  - go to max as soon as a normal task in enqueued: this because dimensioning of
> > > >    parameters is usually done at max OPP/biggest CPU and normal task assume
> > > >    that this is always the condition when they run
> > > >  - scale runtime acconding to current frequency and max CPU capacity: this is
> > > >    what this set is currently implementing
> > > > 
> > > > Opinions?
> > > 
> > > 
> > > So I'm terribly confused...
> > > 
> > > By using the active bandwidth to select frequency we effectively
> > > reduce idle time (to 0 if we had infinite granular frequency steps and
> > > no margins).
> > > 
> > > So !RECLAIM works as expected. They get the time they reserved, since
> > > that was taken into account by active bandwidth.
> > > 
> > 
> > This was my impression as well, but Luca (and please Luca correct me if
> > I misunderstood your point) argued (in an off-line discussion ahead of
> > this posting) that !reclaim tasks might not be interested in reclaiming
> > *at all*. Since scaling frequency down is another way of effectively
> > reclaiming unused bandwidth (the other being sharing unused bandwidth
> > among reservations while keeping frequency at max), !reclaim tasks could
> > not be interested in frequency scaling (my first point above) or require
> > frequency to be always at max (second point above).
> > 
> > Does this help claryfing a bit? :)
> 
> No ;-) As you said, confusion++.
> 
> A !RECLAIM task doesn't care (cannot care, should not care etc..) about
> any bandwidth not allocated to itself. Therefore it should/must/etc..
> not have any opinion on what we do with 'spare' bandwidth.
> 

Agreed. However, problem seems to be that

 - in my opinion (current implementation) this translated into scaling
   runtime considering current freq and cpu-max-capacity; and this is
   required when frequency scaling is enabled and we still want to meet
   a task's guaranteed bandwidth
 
 - Luca seemed instead to be inclined to say that, if we scale runtime
   for !reclaim tasks, such tasks are basically allowed to run for more
   time (when frequency is lower than max) by using some of the
   bandwidth not allocated to themselves

[toc] | [prev] | [next] | [standalone]

#1649503 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Wed, May 24, 2017 at 10:50:53AM +0100, Juri Lelli wrote:

> Agreed. However, problem seems to be that
> 
>  - in my opinion (current implementation) this translated into scaling
>    runtime considering current freq and cpu-max-capacity; and this is
>    required when frequency scaling is enabled and we still want to meet
>    a task's guaranteed bandwidth

Just so. The bandwidth they request is based on instructions/work. We
need to get a certain amount of instructions sorted. Nobody cares we get
an exact 10% at random frequency if they loose they finger because we
didn't get that final instruction out that stops the saw blade.

>  - Luca seemed instead to be inclined to say that, if we scale runtime
>    for !reclaim tasks, such tasks are basically allowed to run for more
>    time (when frequency is lower than max) by using some of the
>    bandwidth not allocated to themselves

Yes, that's a wrong view :-) We don't care about 'time', we care about
getting the instruction stream / work completed.

[toc] | [prev] | [next] | [standalone]

#1649408 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Wed, 24 May 2017 10:25:05 +0100
Juri Lelli <juri.lelli@arm.com> wrote:

> Hi,
> 
> On 23/05/17 22:23, Peter Zijlstra wrote:
> > On Tue, May 23, 2017 at 09:53:43AM +0100, Juri Lelli wrote:
> >   
> > > A point that is still very much up for discussion (more that the
> > > others :) is how we implement frequency/cpu scaling.
> > > SCHED_FLAG_RECLAIM tasks only need grub_reclaim(), as the
> > > function already scales their reservation runtime considering
> > > other reservations and maximum bandwidth a CPU has to offer.
> > > However, for normal !RECLAIM tasks multiple things can be
> > > implemented which seem to make sense:
> > > 
> > >  - don't scale at all: normal tasks will only get a % of CPU
> > > _time_ as granted by AC
> > >  - go to max as soon as a normal task in enqueued: this because
> > > dimensioning of parameters is usually done at max OPP/biggest CPU
> > > and normal task assume that this is always the condition when
> > > they run
> > >  - scale runtime acconding to current frequency and max CPU
> > > capacity: this is what this set is currently implementing
> > > 
> > > Opinions?  
> > 
> > 
> > So I'm terribly confused...
> > 
> > By using the active bandwidth to select frequency we effectively
> > reduce idle time (to 0 if we had infinite granular frequency steps
> > and no margins).
> > 
> > So !RECLAIM works as expected. They get the time they reserved,
> > since that was taken into account by active bandwidth.
> >   
> 
> This was my impression as well, but Luca (and please Luca correct me
> if I misunderstood your point) argued (in an off-line discussion
> ahead of this posting) that !reclaim tasks might not be interested in
> reclaiming *at all*.

Well, I also admitted that I am almost completely ignorant about many
people's requirements...

What I know is that there are some people using SCHED_DEADLINE to make
sure that a task can make progress (executing with a "high priority")
without consuming more than a specified fraction of CPU time... So,
they for example schedule a CPU-hungry task with runtime=10ms and
period=100ms to make sure that the task can execute every 100ms (giving
the impression of a "fluid progress") without stealing more than 10% of
CPU time to other tasks.

In this case, if the CPU frequency change the goal is still to
"reserve" 10% of CPU time (not more, even if the CPU is slower) to the
task. So, no runtime rescaling (or reclaiming) is required in this case.


My proposal was that if a task is not interested in a fixed
runtime / fraction of CPU time but wants to adapt the runtime when the
CPU frequency scales, then it can select the RECLAIMING flag.

But of course there might be different requirements or other use-cases.



			Luca

> Since scaling frequency down is another way of
> effectively reclaiming unused bandwidth (the other being sharing
> unused bandwidth among reservations while keeping frequency at
> max), !reclaim tasks could not be interested in frequency scaling (my
> first point above) or require frequency to be always at max (second
> point above).
> 
> Does this help claryfing a bit? :)
> 
> This said however, I'd personally be inclined to go with option 3
> above, which is what this set is currently implementing.
> 
> > And RECLAIM works, since that only promises to (re)distribute idle
> > time, and if there is none that is an easy task.
> >   
> 
> Right.
> 
> Thanks,
> 
> - Juri

[toc] | [prev] | [next] | [standalone]

#1649508 — Re: [PATCH RFC 0/8] SCHED_DEADLINE freq/cpu invariance and OPP selection

On Wed, May 24, 2017 at 12:01:51PM +0200, Luca Abeni wrote:
> > > So I'm terribly confused...
> > > 
> > > By using the active bandwidth to select frequency we effectively
> > > reduce idle time (to 0 if we had infinite granular frequency steps
> > > and no margins).
> > > 
> > > So !RECLAIM works as expected. They get the time they reserved,
> > > since that was taken into account by active bandwidth.


> Well, I also admitted that I am almost completely ignorant about many
> people's requirements...
> 
> What I know is that there are some people using SCHED_DEADLINE to make
> sure that a task can make progress (executing with a "high priority")
> without consuming more than a specified fraction of CPU time... So,
> they for example schedule a CPU-hungry task with runtime=10ms and
> period=100ms to make sure that the task can execute every 100ms (giving
> the impression of a "fluid progress") without stealing more than 10% of
> CPU time to other tasks.
> 
> In this case, if the CPU frequency change the goal is still to
> "reserve" 10% of CPU time (not more, even if the CPU is slower) to the
> task. So, no runtime rescaling (or reclaiming) is required in this case.
> 
> 
> My proposal was that if a task is not interested in a fixed
> runtime / fraction of CPU time but wants to adapt the runtime when the
> CPU frequency scales, then it can select the RECLAIMING flag.

I think these people are doing it wrong :-)

Firstly, the runtime budget is a WCET. This very much means it is
subject to CPU frequency; after all, when the CPU runs slower, that same
amount of work takes longer. So being subject to cpufreq is the natural
state and should not require a special marker.

Secondly, if you want a steady progress of 10%, I don't see the problem
with giving them more at slower frequency, they get the 'same' amount of
'work' done without bothering other people.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web