Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1399024

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

Show all headers | View raw

On Wed, May 11, 2016 at 12:55:56PM +0100, Matt Fleming wrote:
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -7842,13 +7842,13 @@ static inline void set_cpu_sd_state_busy
> >  	int cpu = smp_processor_id();
> >  
> >  	rcu_read_lock();
> > -	sd = rcu_dereference(per_cpu(sd_busy, cpu));
> > +	sd = rcu_dereference(per_cpu(sd_llc, cpu));
> >  
> >  	if (!sd || !sd->nohz_idle)
> >  		goto unlock;
> >  	sd->nohz_idle = 0;
> >  
> > -	atomic_inc(&sd->groups->sgc->nr_busy_cpus);
> > +	atomic_inc(&sd->shared->nr_busy_cpus);
> >  unlock:
> >  	rcu_read_unlock();
> >  }
> 
> This breaks my POWER7 box which presumably doesn't have SD_SHARE_PKG_RESOURCES,
> 

Hmm, PPC folks; what does your topology look like?

Currently your sched_domain_topology, as per arch/powerpc/kernel/smp.c
seems to suggest your cores do not share cache at all.

https://en.wikipedia.org/wiki/POWER7 seems to agree and states

  "4 MB L3 cache per C1 core"

And http://www-03.ibm.com/systems/resources/systems_power_software_i_perfmgmt_underthehood.pdf
also explicitly draws pictures with the L3 per core.

_however_, that same document describes L3 inter-core fill and lateral
cast-out, which sounds like the L3s work together to form a node wide
caching system.

Do we want to model this co-operative L3 slices thing as a sort of
node-wide LLC for the purpose of the scheduler ?

While we should definitely fix the assumption that an LLC exists (and I
need to look at why it isn't set to the core domain instead as well),
the scheduler does try and scale things by 'assuming' LLC := node.

It does this for NOHZ, and these here patches under discussion would be
doing the same for idle-core state.

Would this make sense for power, or should we somehow think of something
else?

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[RFC][PATCH 0/7] sched: select_idle_siblings rewrite Peter Zijlstra <peterz@infradead.org> - 2016-05-09 13:00 +0200
  [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-09 13:00 +0200
    Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Matt Fleming <matt@codeblueprint.co.uk> - 2016-05-11 14:00 +0200
      Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-11 14:40 +0200
        Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-11 20:20 +0200
        Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-11 20:30 +0200
          Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Michael Neuling <mikey@neuling.org> - 2016-05-12 04:10 +0200
            Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-12 07:10 +0200
              Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Michael Neuling <mikey@neuling.org> - 2016-05-12 13:10 +0200
                Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-12 13:40 +0200
      Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-11 19:40 +0200
        Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with  sched_domain_shared Matt Fleming <matt@codeblueprint.co.uk> - 2016-05-11 20:10 +0200
  [RFC][PATCH 1/7] sched: Remove unused @cpu argument from destroy_sched_domain*() Peter Zijlstra <peterz@infradead.org> - 2016-05-09 13:00 +0200
  [RFC][PATCH 3/7] sched: Introduce struct sched_domain_shared Peter Zijlstra <peterz@infradead.org> - 2016-05-09 13:00 +0200
  [RFC][PATCH 6/7] sched: Optimize SCHED_SMT Peter Zijlstra <peterz@infradead.org> - 2016-05-09 13:00 +0200
  Re: [RFC][PATCH 0/7] sched: select_idle_siblings rewrite Chris Mason <clm@fb.com> - 2016-05-10 03:00 +0200
  Re: [RFC][PATCH 0/7] sched: select_idle_siblings rewrite Chris Mason <clm@fb.com> - 2016-05-11 16:30 +0200

csiph-web