Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1261077 > unrolled thread

[RFC PATCH 0/3] CFS idle injection

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-03 01:20 +0100
    [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-03 01:20 +0100
      Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-03 14:40 +0100
        Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-03 15:20 +0100
        Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-03 17:50 +0100
          Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 11:20 +0100
            Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Arjan van de Ven <arjan@linux.intel.com> - 2015-11-05 15:30 +0100
              Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 15:40 +0100
                Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 15:50 +0100
                Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Arjan van de Ven <arjan@linux.intel.com> - 2015-11-05 16:30 +0100
                  Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 18:00 +0100
                    Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Thomas Gleixner <tglx@linutronix.de> - 2015-11-05 20:00 +0100
                  Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Thomas Gleixner <tglx@linutronix.de> - 2015-11-05 19:50 +0100
                Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-05 16:40 +0100
                  Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Arjan van de Ven <arjan@linux.intel.com> - 2015-11-05 17:10 +0100
            Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-05 20:30 +0100
              Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-05 20:40 +0100
        Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 15:10 +0100
          Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-06 00:50 +0100
            Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-06 08:50 +0100
              Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-07 01:00 +0100
                Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-10 01:30 +0100
    [RFC PATCH 2/3] timer: relax tick stop in idle entry Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-03 01:20 +0100
    Re: [RFC PATCH 0/3] CFS idle injection Eduardo Valentin <edubezval@gmail.com> - 2015-11-04 07:10 +0100
      Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-04 18:00 +0100
        Re: [RFC PATCH 0/3] CFS idle injection Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> - 2015-11-04 18:10 +0100
          Re: [RFC PATCH 0/3] CFS idle injection Eduardo Valentin <edubezval@gmail.com> - 2015-11-04 19:50 +0100
        Re: [RFC PATCH 0/3] CFS idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-05 11:20 +0100
          Re: [RFC PATCH 0/3] CFS idle injection Punit Agrawal <punit.agrawal@arm.com> - 2015-11-06 18:00 +0100
            Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-06 22:00 +0100
              Re: [RFC PATCH 0/3] CFS idle injection Punit Agrawal <punit.agrawal@arm.com> - 2015-11-09 13:00 +0100
                Re: [RFC PATCH 0/3] CFS idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-09 15:20 +0100
                  Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-09 15:50 +0100
                  Re: [RFC PATCH 0/3] CFS idle injection Juri Lelli <juri.lelli@gmail.com> - 2015-11-10 11:10 +0100
                    Re: [RFC PATCH 0/3] CFS idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-10 11:40 +0100
                      Re: [RFC PATCH 0/3] CFS idle injection Juri Lelli <juri.lelli@arm.com> - 2015-11-10 12:00 +0100
                Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-09 15:40 +0100
          Re: [RFC PATCH 0/3] CFS idle injection Dietmar Eggemann <dietmar.eggemann@arm.com> - 2015-11-06 19:40 +0100
            Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-06 20:20 +0100
              Re: [RFC PATCH 0/3] CFS idle injection Dietmar Eggemann <dietmar.eggemann@arm.com> - 2015-11-06 23:00 +0100
                Re: [RFC PATCH 0/3] CFS idle injection Jacob Pan <jacob.jun.pan@linux.intel.com> - 2015-11-09 22:30 +0100
                  Re: [RFC PATCH 0/3] CFS idle injection Peter Zijlstra <peterz@infradead.org> - 2015-11-09 22:50 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#1264661 — Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection

On Fri, 6 Nov 2015 08:45:10 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Thu, Nov 05, 2015 at 03:36:25PM -0800, Jacob Pan wrote:
> 
> > I did some testing with the code below, it shows random
> > [  150.442597] NOHZ: local_softirq_pending 02
> > [  153.032673] NOHZ: local_softirq_pending 202
> > [  153.203785] NOHZ: local_softirq_pending 202
> > [  153.206486] NOHZ: local_softirq_pending 282
> > I recalled that was why i checked for local_softirq_pending in the
> > initial patch, still trying to find out how we can avoid that. These
> > also causes non stop sched ticks in the inner idle loop.
> 
> Check the softirq stuff before calling throttle ?

yes, played with it but it seems there are other cases causing pending
softirq in idle in addition to throttle. I still haven't figure it out,
this problem only shows up in heavy irq, network load. e.g. compile
kernel over NFS. Debugging.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266117 — Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection

On Fri, 6 Nov 2015 15:49:29 -0800
Jacob Pan <jacob.jun.pan@linux.intel.com> wrote:

> > Check the softirq stuff before calling throttle ?  
> 
> yes, played with it but it seems there are other cases causing pending
> softirq in idle in addition to throttle. I still haven't figure it
> out, this problem only shows up in heavy irq, network load. e.g.
> compile kernel over NFS. Debugging.
ok, I added a check for softirq_pending and a retry. seems to work. Now
idle injection will allow softirq and softirqd to run. The caveat is
that during that injection period, if softirqd does not run for the
entire duration, other normal tasks would also run during forced idle.
But just for that period. I guess we have to strike for the right
balance for QoS and overhead. For most workload, pending softirq is
rare so the tasks slip under softirqd are also rare. Will send out V2
soon.

Thanks,

Jacob
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1261081 — [RFC PATCH 2/3] timer: relax tick stop in idle entry

Upon entering idle, we can turn off tick if the next timeout
is exactly one tick away. Otherwise, we could enter inner idle loop
with tick still enabled, without resched set, the tick will continue
during idle therefore less optimal in terms of energy savings.

Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
 kernel/time/tick-sched.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 7c7ec45..bcadaab 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -606,7 +606,7 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 	 * restart it proper.
 	 */
 	delta = next_tick - basemono;
-	if (delta <= (u64)TICK_NSEC) {
+	if (delta < (u64)TICK_NSEC) {
 		tick.tv64 = 0;
 		if (!ts->tick_stopped)
 			goto out;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1262088

Hello Jacob,

On Mon, Nov 02, 2015 at 04:10:25PM -0800, Jacob Pan wrote:
> Hi Peter and all,
> 
> A while ago, we had discussion about how powerclamp is broken in the
> sense of turning off idle ticks in the forced idle period.
> https://lkml.org/lkml/2014/12/18/369
> 
> It was suggested to replace the current kthread play idle loop with a
> timer based runqueue throttling scheme. I finally got around to implement
> this and code is much simpler. I also have good test results in terms of
> efficiency, scalability, etc.
> http://events.linuxfoundation.org/sites/events/files/slides/LinuxCon_Japan_2015_idle_injection1_0.pdf
> slide #18+ shows the data on client and server.
> 
> I have two choices for this code:
> 1) be part of existing powerclamp driver but require exporting some
>    sched APIs.
> 2) be part of sched since the genernal rule applies when it comes down
>    to sycnhronized idle time for best power savings.
> 
> The patches below are for #2. There is a known problem with LOW RES timer
> mode that I am working on. But I am hoping to get review earlier.
> 

I also like #2 too. Specially now that it is not limited to a specific
platform. One question though, could you still keep the cooling device
support of it? In some systems, it might make sense to enable / disable
idle injections based on temperature.

Was there any particular reason you dropped the cooling device support?

BR,

Eduardo Valentin


> We are entering a very power limited environment on client side, frequency
> scaling can only be efficient at certain range. e.g. on SKL, upto ~900MHz,
> anything below, it is increasingly more efficient to do C-states insertion
> if coordinated.
> 
> Looking forward, there are use case beyond thermal/power capping. I think
> we can consolidate ballanced partial busy workload that are evenly
> distributed among CPUs.
> 
> Please let me know what you think.
> 
> Thanks,
> 
> 
> Jacob Pan (3):
>   ktime: add a roundup function
>   timer: relax tick stop in idle entry
>   sched: introduce synchronized idle injection
> 
>  include/linux/ktime.h        |  10 ++
>  include/linux/sched.h        |  12 ++
>  include/linux/sched/sysctl.h |   5 +
>  include/trace/events/sched.h |  23 +++
>  init/Kconfig                 |   8 +
>  kernel/sched/fair.c          | 345 +++++++++++++++++++++++++++++++++++++++++++
>  kernel/sched/sched.h         |   3 +
>  kernel/sysctl.c              |  20 +++
>  kernel/time/tick-sched.c     |   2 +-
>  9 files changed, 427 insertions(+), 1 deletion(-)
> 
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1262438

On Tue, 3 Nov 2015 22:06:55 -0800
Eduardo Valentin <edubezval@gmail.com> wrote:

> Hello Jacob,
> 
> On Mon, Nov 02, 2015 at 04:10:25PM -0800, Jacob Pan wrote:
> > Hi Peter and all,
> > 
> > A while ago, we had discussion about how powerclamp is broken in the
> > sense of turning off idle ticks in the forced idle period.
> > https://lkml.org/lkml/2014/12/18/369
> > 
> > It was suggested to replace the current kthread play idle loop with
> > a timer based runqueue throttling scheme. I finally got around to
> > implement this and code is much simpler. I also have good test
> > results in terms of efficiency, scalability, etc.
> > http://events.linuxfoundation.org/sites/events/files/slides/LinuxCon_Japan_2015_idle_injection1_0.pdf
> > slide #18+ shows the data on client and server.
> > 
> > I have two choices for this code:
> > 1) be part of existing powerclamp driver but require exporting some
> >    sched APIs.
> > 2) be part of sched since the genernal rule applies when it comes
> > down to sycnhronized idle time for best power savings.
> > 
> > The patches below are for #2. There is a known problem with LOW RES
> > timer mode that I am working on. But I am hoping to get review
> > earlier.
> > 
> 
> I also like #2 too. Specially now that it is not limited to a specific
> platform. One question though, could you still keep the cooling device
> support of it? In some systems, it might make sense to enable /
> disable idle injections based on temperature.
> 
One of the key difference between 1 and 2 is that #2 is open loop
control, since we don't have CPU c-states info baked into scheduler. To
close the loop, perhaps we can export some internal APIs to the thermal
subsystem then the thermal governors can pick the condition to inject
idle.
> Was there any particular reason you dropped the cooling device
> support?
> 
I did sysctl instead of thermal sysfs to conform the rest of the sched
tuning knobs. We could also have a proxy cooling device to call
internal APIs mentioned above.

Another reason is that, I intend to extend beyond thermal. Where we can
consolidate/sync idle work in semi-active and balanced workload.

Thanks for the suggestions,

Jacob
> BR,
> 
> Eduardo Valentin
> 
> 
> > We are entering a very power limited environment on client side,
> > frequency scaling can only be efficient at certain range. e.g. on
> > SKL, upto ~900MHz, anything below, it is increasingly more
> > efficient to do C-states insertion if coordinated.
> > 
> > Looking forward, there are use case beyond thermal/power capping. I
> > think we can consolidate ballanced partial busy workload that are
> > evenly distributed among CPUs.
> > 
> > Please let me know what you think.
> > 
> > Thanks,
> > 
> > 
> > Jacob Pan (3):
> >   ktime: add a roundup function
> >   timer: relax tick stop in idle entry
> >   sched: introduce synchronized idle injection
> > 
> >  include/linux/ktime.h        |  10 ++
> >  include/linux/sched.h        |  12 ++
> >  include/linux/sched/sysctl.h |   5 +
> >  include/trace/events/sched.h |  23 +++
> >  init/Kconfig                 |   8 +
> >  kernel/sched/fair.c          | 345
> > +++++++++++++++++++++++++++++++++++++++++++
> > kernel/sched/sched.h         |   3 + kernel/sysctl.c
> > |  20 +++ kernel/time/tick-sched.c     |   2 +-
> >  9 files changed, 427 insertions(+), 1 deletion(-)
> > 
> > -- 
> > 1.9.1
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/

[Jacob Pan]
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1262444

On Wed, 2015-11-04 at 08:58 -0800, Jacob Pan wrote:
> On Tue, 3 Nov 2015 22:06:55 -0800
> Eduardo Valentin <edubezval@gmail.com> wrote:
> 
> > Hello Jacob,
> > 
> > On Mon, Nov 02, 2015 at 04:10:25PM -0800, Jacob Pan wrote:
> > > Hi Peter and all,
> > > 
> > > A while ago, we had discussion about how powerclamp is broken in the
> > > sense of turning off idle ticks in the forced idle period.
> > > https://lkml.org/lkml/2014/12/18/369
> > > 
> > > It was suggested to replace the current kthread play idle loop with
> > > a timer based runqueue throttling scheme. I finally got around to
> > > implement this and code is much simpler. I also have good test
> > > results in terms of efficiency, scalability, etc.
> > > http://events.linuxfoundation.org/sites/events/files/slides/LinuxCon_Japan_2015_idle_injection1_0.pdf
> > > slide #18+ shows the data on client and server.
> > > 
> > > I have two choices for this code:
> > > 1) be part of existing powerclamp driver but require exporting some
> > >    sched APIs.
> > > 2) be part of sched since the genernal rule applies when it comes
> > > down to sycnhronized idle time for best power savings.
> > > 
> > > The patches below are for #2. There is a known problem with LOW RES
> > > timer mode that I am working on. But I am hoping to get review
> > > earlier.
> > > 
> > 
> > I also like #2 too. Specially now that it is not limited to a specific
> > platform. One question though, could you still keep the cooling device
> > support of it? In some systems, it might make sense to enable /
> > disable idle injections based on temperature.
> > 
> One of the key difference between 1 and 2 is that #2 is open loop
> control, since we don't have CPU c-states info baked into scheduler. To
> close the loop, perhaps we can export some internal APIs to the thermal
> subsystem then the thermal governors can pick the condition to inject
> idle.
> > Was there any particular reason you dropped the cooling device
> > support?
> > 
> I did sysctl instead of thermal sysfs to conform the rest of the sched
> tuning knobs. We could also have a proxy cooling device to call
> internal APIs mentioned above.
I think we should have cooling device as we are already using this
cooling device. Once it pass RFC stage,I think we should consider add
this.
Thanks,
Srinivas
> 
> Another reason is that, I intend to extend beyond thermal. Where we can
> consolidate/sync idle work in semi-active and balanced workload.
> 
> Thanks for the suggestions,
> 
> Jacob
> > BR,
> > 
> > Eduardo Valentin
> > 
> > 
> > > We are entering a very power limited environment on client side,
> > > frequency scaling can only be efficient at certain range. e.g. on
> > > SKL, upto ~900MHz, anything below, it is increasingly more
> > > efficient to do C-states insertion if coordinated.
> > > 
> > > Looking forward, there are use case beyond thermal/power capping. I
> > > think we can consolidate ballanced partial busy workload that are
> > > evenly distributed among CPUs.
> > > 
> > > Please let me know what you think.
> > > 
> > > Thanks,
> > > 
> > > 
> > > Jacob Pan (3):
> > >   ktime: add a roundup function
> > >   timer: relax tick stop in idle entry
> > >   sched: introduce synchronized idle injection
> > > 
> > >  include/linux/ktime.h        |  10 ++
> > >  include/linux/sched.h        |  12 ++
> > >  include/linux/sched/sysctl.h |   5 +
> > >  include/trace/events/sched.h |  23 +++
> > >  init/Kconfig                 |   8 +
> > >  kernel/sched/fair.c          | 345
> > > +++++++++++++++++++++++++++++++++++++++++++
> > > kernel/sched/sched.h         |   3 + kernel/sysctl.c
> > > |  20 +++ kernel/time/tick-sched.c     |   2 +-
> > >  9 files changed, 427 insertions(+), 1 deletion(-)
> > > 
> > > -- 
> > > 1.9.1
> > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe
> > > linux-kernel" in the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > Please read the FAQ at  http://www.tux.org/lkml/
> 
> [Jacob Pan]


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1262503

Hello Jacob, Srinivas,

On Wed, Nov 04, 2015 at 09:05:52AM -0800, Srinivas Pandruvada wrote:
> On Wed, 2015-11-04 at 08:58 -0800, Jacob Pan wrote:

<cut>
> > > > I have two choices for this code:
> > > > 1) be part of existing powerclamp driver but require exporting some
> > > >    sched APIs.
> > > > 2) be part of sched since the genernal rule applies when it comes
> > > > down to sycnhronized idle time for best power savings.
> > > > 
> > > > The patches below are for #2. There is a known problem with LOW RES
> > > > timer mode that I am working on. But I am hoping to get review
> > > > earlier.
> > > > 
> > > 
> > > I also like #2 too. Specially now that it is not limited to a specific
> > > platform. One question though, could you still keep the cooling device
> > > support of it? In some systems, it might make sense to enable /
> > > disable idle injections based on temperature.
> > > 
> > One of the key difference between 1 and 2 is that #2 is open loop
> > control, since we don't have CPU c-states info baked into scheduler. To
> > close the loop, perhaps we can export some internal APIs to the thermal
> > subsystem then the thermal governors can pick the condition to inject
> > idle.


Jacob,

I also like this direction. Having the proper APIs exported, creating a
cooling device that use them would be  natural path. Then, one could
create a thermal zone plugging a governor and the idle injection cooling
device that uses the exported APIs.

> > > Was there any particular reason you dropped the cooling device
> > > support?
> > > 
> > I did sysctl instead of thermal sysfs to conform the rest of the sched
> > tuning knobs. We could also have a proxy cooling device to call
> > internal APIs mentioned above.

Agreed here then.


> I think we should have cooling device as we are already using this
> cooling device. Once it pass RFC stage,I think we should consider add
> this.

Srinivas, 
Yes, that seens to be a good path to follow. Thanks.


> Thanks,
> Srinivas
> > 
> > Another reason is that, I intend to extend beyond thermal. Where we can
> > consolidate/sync idle work in semi-active and balanced workload.

I see. 

BR,

Eduardo Valentin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1263064

People, trim your emails!

On Wed, Nov 04, 2015 at 08:58:30AM -0800, Jacob Pan wrote:

> > I also like #2 too. Specially now that it is not limited to a specific
> > platform. One question though, could you still keep the cooling device
> > support of it? In some systems, it might make sense to enable /
> > disable idle injections based on temperature.

> One of the key difference between 1 and 2 is that #2 is open loop
> control, since we don't have CPU c-states info baked into scheduler. 

_yet_, there's people working on that. The whole power aware scheduling
stuff needs that.
 
> To close the loop, perhaps we can export some internal APIs to the
> thermal subsystem then the thermal governors can pick the condition to
> inject idle.

I would much rather that all be part of the power aware stuff, such that
the scheduler itself is aware of thermal limits and can migrate load
away if needed.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1264160

Peter Zijlstra <peterz@infradead.org> writes:

> People, trim your emails!
>
> On Wed, Nov 04, 2015 at 08:58:30AM -0800, Jacob Pan wrote:
>
>> > I also like #2 too. Specially now that it is not limited to a specific
>> > platform. One question though, could you still keep the cooling device
>> > support of it? In some systems, it might make sense to enable /
>> > disable idle injections based on temperature.
>
>> One of the key difference between 1 and 2 is that #2 is open loop
>> control, since we don't have CPU c-states info baked into scheduler. 
>
> _yet_, there's people working on that. The whole power aware scheduling
> stuff needs that.
>  
>> To close the loop, perhaps we can export some internal APIs to the
>> thermal subsystem then the thermal governors can pick the condition to
>> inject idle.
>
> I would much rather that all be part of the power aware stuff, such that
> the scheduler itself is aware of thermal limits and can migrate load
> away if needed.

I was wondering if we could use cpu capacity as the interface between
the thermal sub-system and the scheduler. This would be better than
dealing with frequency caps and idle injection percentages directly in
the scheduler.

We've been playing with making the scheduler respect capacity caps due
to thermal constraints and have tasks migrated away to less capped
cores.

It would be great if in addition to the frequency caps, we could add
idle injection to the arsenal. This would allow building policies on top
such as -

* pure idle injection where frequency capping is unsuitable (or
  unavailable)
* a smooth continuum of capacities using a combination of frequency and
  capacity capping
* idle injection once frequencies have been capped to the lowest
  feasible values (as suggested in the cover letter)

One question about the implementation in these patches - should the
implementation hook into pick_next_task in core instead of CFS? Higher
priority tasks might get in the way of idle injection.

> -- To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1264531

On Fri, 06 Nov 2015 16:50:15 +0000
Punit Agrawal <punit.agrawal@arm.com> wrote:

> * idle injection once frequencies have been capped to the lowest
>   feasible values (as suggested in the cover letter)
> 
actually, I was suggesting to start considering idle injection once
frequency capped to the energy efficient point, which can be much
higher than the lowest frequency. The idea being, deep idle power is
negligible compared to running power which allows near linear
power-perf scaling for balanced workload.
Below energy efficient frequency, continuous lowering frequency may
lose disproportion performance vs. power. i.e. worse than linear.

> One question about the implementation in these patches - should the
> implementation hook into pick_next_task in core instead of CFS? Higher
> priority tasks might get in the way of idle injection.
My take is that RT and throttling will never go well together since they
are conflicting in principle.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1265599

Jacob Pan <jacob.jun.pan@linux.intel.com> writes:

> On Fri, 06 Nov 2015 16:50:15 +0000
> Punit Agrawal <punit.agrawal@arm.com> wrote:
>
>> * idle injection once frequencies have been capped to the lowest
>>   feasible values (as suggested in the cover letter)
>> 
> actually, I was suggesting to start considering idle injection once
> frequency capped to the energy efficient point, which can be much
> higher than the lowest frequency. The idea being, deep idle power is
> negligible compared to running power which allows near linear
> power-perf scaling for balanced workload.
> Below energy efficient frequency, continuous lowering frequency may
> lose disproportion performance vs. power. i.e. worse than linear.
>

I agree. I was making that assumption that with the ability to inject
idle states, there wouldn't be a need to expose the inefficient
frequency states.

Do you still see a reason to do that?

>> One question about the implementation in these patches - should the
>> implementation hook into pick_next_task in core instead of CFS? Higher
>> priority tasks might get in the way of idle injection.
> My take is that RT and throttling will never go well together since they
> are conflicting in principle.

I am not sure I follow. If RT (or other higher priority classes) can't
be throttled then the CPUs are not able to contribute towards
constraining power consumption and hence temperature.

This is especially true in certain platforms where tasks belong to the
RT class to maintain user experience, e.g., audio and video. 

> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1265704

On Mon, Nov 09, 2015 at 11:56:51AM +0000, Punit Agrawal wrote:
> Jacob Pan <jacob.jun.pan@linux.intel.com> writes:
> > My take is that RT and throttling will never go well together since they
> > are conflicting in principle.
> 
> I am not sure I follow. If RT (or other higher priority classes) can't
> be throttled then the CPUs are not able to contribute towards
> constraining power consumption and hence temperature.
> 
> This is especially true in certain platforms where tasks belong to the
> RT class to maintain user experience, e.g., audio and video. 

Audio/Video playback generally doesn't take a _lot_ of time these days.
What is important though is _when_ it happens.

And media playback typically already has a very well defined and stable
cadence (24Hz or whatnot).  What you want is for your idle injector to
sync up with that, not disrupt it.

For other workloads, missing a deadline is about as bad as destroying
the chip, complete system shutdown might be safer than getting delayed.
(The very tired scenario of a saw, a laser and your finger; you want to
shut down the entire machine rather than just cut off your finger.)



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1265731

On Mon, 9 Nov 2015 15:15:34 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Mon, Nov 09, 2015 at 11:56:51AM +0000, Punit Agrawal wrote:
> > Jacob Pan <jacob.jun.pan@linux.intel.com> writes:
> > > My take is that RT and throttling will never go well together
> > > since they are conflicting in principle.
> > 
> > I am not sure I follow. If RT (or other higher priority classes)
> > can't be throttled then the CPUs are not able to contribute towards
> > constraining power consumption and hence temperature.
> > 
> > This is especially true in certain platforms where tasks belong to
> > the RT class to maintain user experience, e.g., audio and video. 
> 
> Audio/Video playback generally doesn't take a _lot_ of time these
> days. What is important though is _when_ it happens.
> 
> And media playback typically already has a very well defined and
> stable cadence (24Hz or whatnot).  What you want is for your idle
> injector to sync up with that, not disrupt it.
> 
Agreed, i have tested idle injection on video playback (mostly one cpu
busy, no sync with gpu), it does not do well to improve energy
efficiency. With the video playback being offloaded, there is no
thermal condition either. So outside the scope of this first patchst
trying to solve. The ability to sync with external pattern, could be the
next step. kind of like pll in hw :).

> For other workloads, missing a deadline is about as bad as destroying
> the chip, complete system shutdown might be safer than getting
> delayed. (The very tired scenario of a saw, a laser and your finger;
> you want to shut down the entire machine rather than just cut off
> your finger.)
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266374

Hi,

On 9 November 2015 at 14:15, Peter Zijlstra <peterz@infradead.org> wrote:
> On Mon, Nov 09, 2015 at 11:56:51AM +0000, Punit Agrawal wrote:
>> Jacob Pan <jacob.jun.pan@linux.intel.com> writes:
>> > My take is that RT and throttling will never go well together since they
>> > are conflicting in principle.
>>
>> I am not sure I follow. If RT (or other higher priority classes) can't
>> be throttled then the CPUs are not able to contribute towards
>> constraining power consumption and hence temperature.
>>
>> This is especially true in certain platforms where tasks belong to the
>> RT class to maintain user experience, e.g., audio and video.
>
> Audio/Video playback generally doesn't take a _lot_ of time these days.
> What is important though is _when_ it happens.
>
> And media playback typically already has a very well defined and stable
> cadence (24Hz or whatnot).  What you want is for your idle injector to
> sync up with that, not disrupt it.
>

Do you think that using SCHED_DEADLINE here would be completely
foolish? I mean, we would have the duty_cycle/period thing for free, it
would be know to the scheduler (as to maybe address Thomas' concerns)
and we could think to make idle injection part of system analysis (for
the soft-RT use cases).

Thanks,

- Juri

> For other workloads, missing a deadline is about as bad as destroying
> the chip, complete system shutdown might be safer than getting delayed.
> (The very tired scenario of a saw, a laser and your finger; you want to
> shut down the entire machine rather than just cut off your finger.)
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266389

On Tue, Nov 10, 2015 at 10:07:35AM +0000, Juri Lelli wrote:
> Do you think that using SCHED_DEADLINE here would be completely
> foolish? I mean, we would have the duty_cycle/period thing for free, it
> would be know to the scheduler (as to maybe address Thomas' concerns)
> and we could think to make idle injection part of system analysis (for
> the soft-RT use cases).

DEADLINE would be awesome, but I think we need work on two fronts before
we can really sell it as the awesome that it is ;-)

 - greedy and or statistical bounds
 - !priv

The first is such that we can better deal with the erratic nature of
media decode without going full worst case on it, and the second just
makes it so much more accessible.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266399

On 11/10/15, Peter Zijlstra wrote:
> On Tue, Nov 10, 2015 at 10:07:35AM +0000, Juri Lelli wrote:
> > Do you think that using SCHED_DEADLINE here would be completely
> > foolish? I mean, we would have the duty_cycle/period thing for free, it
> > would be know to the scheduler (as to maybe address Thomas' concerns)
> > and we could think to make idle injection part of system analysis (for
> > the soft-RT use cases).
> 
> DEADLINE would be awesome, but I think we need work on two fronts before
> we can really sell it as the awesome that it is ;-)
> 
>  - greedy and or statistical bounds
>  - !priv
> 
> The first is such that we can better deal with the erratic nature of
> media decode without going full worst case on it, and the second just
> makes it so much more accessible.
> 

Right. For the first point I think we just need to make our off-line
calculation right, not that we need to modify implementation. On the
second point we need more work yes. Also, another thing that is missing
is frequency (uarch) scaling for reservations parameters, something alike
what we are doing for CFS; this last point might be solved sooner :-).

Thanks,

- Juri

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1265718

On Mon, 09 Nov 2015 11:56:51 +0000
Punit Agrawal <punit.agrawal@arm.com> wrote:

> > actually, I was suggesting to start considering idle injection once
> > frequency capped to the energy efficient point, which can be much
> > higher than the lowest frequency. The idea being, deep idle power is
> > negligible compared to running power which allows near linear
> > power-perf scaling for balanced workload.
> > Below energy efficient frequency, continuous lowering frequency may
> > lose disproportion performance vs. power. i.e. worse than linear.
> >  
> 
> I agree. I was making that assumption that with the ability to inject
> idle states, there wouldn't be a need to expose the inefficient
> frequency states.
> 
> Do you still see a reason to do that?
yes, but it is up to a governor or management sw to decide when to to
pick what mechanism. there may be certain workload scale better with
frequency change. e.g. unbalanced workload, we don't want to inject
idle to all cpus if just one is busy. but it is also unlikely to run
into thermal issue in this case.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1264211

On 05/11/15 10:12, Peter Zijlstra wrote:
> 
> People, trim your emails!
> 
> On Wed, Nov 04, 2015 at 08:58:30AM -0800, Jacob Pan wrote:
> 
>>> I also like #2 too. Specially now that it is not limited to a specific
>>> platform. One question though, could you still keep the cooling device
>>> support of it? In some systems, it might make sense to enable /
>>> disable idle injections based on temperature.
> 
>> One of the key difference between 1 and 2 is that #2 is open loop
>> control, since we don't have CPU c-states info baked into scheduler. 
> 
> _yet_, there's people working on that. The whole power aware scheduling
> stuff needs that.

Isn't the idle state information (rq->idle_state) already used in
find_idlest_cpu()?

What we use in energy aware scheduling is quite similar but since we're
interested in the index information of the c-state (to access the right
element of the idle_state vectors of the energy model, we added
rq->idle_state_idx.

[...]

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1264244

On Fri, 6 Nov 2015 18:30:01 +0000
Dietmar Eggemann <dietmar.eggemann@arm.com> wrote:

> On 05/11/15 10:12, Peter Zijlstra wrote:
> > 
> > People, trim your emails!
> > 
> > On Wed, Nov 04, 2015 at 08:58:30AM -0800, Jacob Pan wrote:
> > 
> >>> I also like #2 too. Specially now that it is not limited to a
> >>> specific platform. One question though, could you still keep the
> >>> cooling device support of it? In some systems, it might make
> >>> sense to enable / disable idle injections based on temperature.
> > 
> >> One of the key difference between 1 and 2 is that #2 is open loop
> >> control, since we don't have CPU c-states info baked into
> >> scheduler. 
> > 
> > _yet_, there's people working on that. The whole power aware
> > scheduling stuff needs that.
> 
> Isn't the idle state information (rq->idle_state) already used in
> find_idlest_cpu()?
> 
> What we use in energy aware scheduling is quite similar but since
> we're interested in the index information of the c-state (to access
> the right element of the idle_state vectors of the energy model, we
> added rq->idle_state_idx.
> 
what i am interested is not per cpu idle state but rather at the package
level or domain. It must be an indication for the overlapped idle time.
Usually has to come from HW counters.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1264585

On 11/06/2015 07:10 PM, Jacob Pan wrote:
> On Fri, 6 Nov 2015 18:30:01 +0000
> Dietmar Eggemann <dietmar.eggemann@arm.com> wrote:
>
>> On 05/11/15 10:12, Peter Zijlstra wrote:
>>>
>>> People, trim your emails!
>>>
>>> On Wed, Nov 04, 2015 at 08:58:30AM -0800, Jacob Pan wrote:
>>>
>>>>> I also like #2 too. Specially now that it is not limited to a
>>>>> specific platform. One question though, could you still keep the
>>>>> cooling device support of it? In some systems, it might make
>>>>> sense to enable / disable idle injections based on temperature.
>>>
>>>> One of the key difference between 1 and 2 is that #2 is open loop
>>>> control, since we don't have CPU c-states info baked into
>>>> scheduler.
>>>
>>> _yet_, there's people working on that. The whole power aware
>>> scheduling stuff needs that.
>>
>> Isn't the idle state information (rq->idle_state) already used in
>> find_idlest_cpu()?
>>
>> What we use in energy aware scheduling is quite similar but since
>> we're interested in the index information of the c-state (to access
>> the right element of the idle_state vectors of the energy model, we
>> added rq->idle_state_idx.
>>
> what i am interested is not per cpu idle state but rather at the package
> level or domain. It must be an indication for the overlapped idle time.
> Usually has to come from HW counters.

I see. We have a similar problem with the Energy Model (EM) on cluster 
level (sched domain level DIE). We iterate over the cpus of a sched 
group and declare the shallowest cpu idle state as the cluster idle 
state to index our EM. On a typical ARM system we have (active, WFI, 
cpu-off and cluster-off). But I guess for you the idle state index is 
only for core idle states and you can't draw any conclusions from this 
for the package idle states.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel

csiph-web