Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1440435 > unrolled thread

Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root domain

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root  domain Peter Zijlstra <peterz@infradead.org> - 2016-07-11 12:20 +0200
    Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root  domain Dietmar Eggemann <dietmar.eggemann@arm.com> - 2016-07-11 18:20 +0200
      Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root  domain Peter Zijlstra <peterz@infradead.org> - 2016-07-12 13:50 +0200
        Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root  domain Dietmar Eggemann <dietmar.eggemann@arm.com> - 2016-07-13 13:30 +0200

#1440435 — Re: [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root domain

On Wed, Jun 22, 2016 at 06:03:17PM +0100, Morten Rasmussen wrote:
> @@ -6905,11 +6906,19 @@ static int build_sched_domains(const struct cpumask *cpu_map,
>  	/* Attach the domains */
>  	rcu_read_lock();
>  	for_each_cpu(i, cpu_map) {
> +		rq = cpu_rq(i);
>  		sd = *per_cpu_ptr(d.sd, i);
>  		cpu_attach_domain(sd, d.rd, i);
> +
> +		if (rq->cpu_capacity_orig > rq->rd->max_cpu_capacity)
> +			rq->rd->max_cpu_capacity = rq->cpu_capacity_orig;
>  	}

Should you not set that _before_ cpu_attach_domain(), such that the
state is up-to-date when its published?

Also, since its lockless, should we not use {READ,WRITE}_ONCE() with it?

>  	rcu_read_unlock();
>  
> +	if (rq)
> +		pr_info("span: %*pbl (max cpu_capacity = %lu)\n",
> +			cpumask_pr_args(cpu_map), rq->rd->max_cpu_capacity);
> +

While a single statement, it is multi line, please add brackets.

>  	ret = 0;
>  error:

[toc] | [next] | [standalone]

#1440740

On 11/07/16 11:18, Peter Zijlstra wrote:
> On Wed, Jun 22, 2016 at 06:03:17PM +0100, Morten Rasmussen wrote:
>> @@ -6905,11 +6906,19 @@ static int build_sched_domains(const struct cpumask *cpu_map,
>>  	/* Attach the domains */
>>  	rcu_read_lock();
>>  	for_each_cpu(i, cpu_map) {
>> +		rq = cpu_rq(i);
>>  		sd = *per_cpu_ptr(d.sd, i);
>>  		cpu_attach_domain(sd, d.rd, i);
>> +
>> +		if (rq->cpu_capacity_orig > rq->rd->max_cpu_capacity)
>> +			rq->rd->max_cpu_capacity = rq->cpu_capacity_orig;
>>  	}
> 
> Should you not set that _before_ cpu_attach_domain(), such that the
> state is up-to-date when its published?

yes, much better.

> Also, since its lockless, should we not use {READ,WRITE}_ONCE() with it?

You mean for rq->rd->max_cpu_capacity ? IMHO, there is a data dependency
between the read and the write and the code only runs on one cpu.

I assume here that this is related to item 2 'Overlapping loads and
stores within a particular CPU ...' in GUARANTEES of
doc/Documentation/memory-barriers.txt.

Do I miss something?

>>  	rcu_read_unlock();
>>  
>> +	if (rq)
>> +		pr_info("span: %*pbl (max cpu_capacity = %lu)\n",
>> +			cpumask_pr_args(cpu_map), rq->rd->max_cpu_capacity);
>> +
> 
> While a single statement, it is multi line, please add brackets.

OK.

> 
>>  	ret = 0;
>>  error:

[toc] | [prev] | [next] | [standalone]

#1441279

On Mon, Jul 11, 2016 at 05:16:06PM +0100, Dietmar Eggemann wrote:
> On 11/07/16 11:18, Peter Zijlstra wrote:
> > On Wed, Jun 22, 2016 at 06:03:17PM +0100, Morten Rasmussen wrote:
> >> @@ -6905,11 +6906,19 @@ static int build_sched_domains(const struct cpumask *cpu_map,
> >>  	/* Attach the domains */
> >>  	rcu_read_lock();
> >>  	for_each_cpu(i, cpu_map) {
> >> +		rq = cpu_rq(i);
> >>  		sd = *per_cpu_ptr(d.sd, i);
> >>  		cpu_attach_domain(sd, d.rd, i);
> >> +
> >> +		if (rq->cpu_capacity_orig > rq->rd->max_cpu_capacity)
> >> +			rq->rd->max_cpu_capacity = rq->cpu_capacity_orig;
> >>  	}
> > 
> > Should you not set that _before_ cpu_attach_domain(), such that the
> > state is up-to-date when its published?
> 
> yes, much better.
> 
> > Also, since its lockless, should we not use {READ,WRITE}_ONCE() with it?
> 
> You mean for rq->rd->max_cpu_capacity ? IMHO, there is a data dependency
> between the read and the write and the code only runs on one cpu.
> 
> I assume here that this is related to item 2 'Overlapping loads and
> stores within a particular CPU ...' in GUARANTEES of
> doc/Documentation/memory-barriers.txt.
> 
> Do I miss something?

Well, the value 'rd->max_cpu_capacity' is read by all CPUs attached to
the root_domain, right? So CPUs already attached can observe this change
when we update the value, we want them to observe either the old or the
new max value, not a random mix of bytes.

{READ,WRITE}_ONCE() ensure whole word load/store, iow they avoid
load/store-tearing.

[toc] | [prev] | [next] | [standalone]

#1442355

On 12/07/16 12:42, Peter Zijlstra wrote:
> On Mon, Jul 11, 2016 at 05:16:06PM +0100, Dietmar Eggemann wrote:
>> On 11/07/16 11:18, Peter Zijlstra wrote:
>>> On Wed, Jun 22, 2016 at 06:03:17PM +0100, Morten Rasmussen wrote:
>>>> @@ -6905,11 +6906,19 @@ static int build_sched_domains(const struct cpumask *cpu_map,
>>>>  	/* Attach the domains */
>>>>  	rcu_read_lock();
>>>>  	for_each_cpu(i, cpu_map) {
>>>> +		rq = cpu_rq(i);
>>>>  		sd = *per_cpu_ptr(d.sd, i);
>>>>  		cpu_attach_domain(sd, d.rd, i);
>>>> +
>>>> +		if (rq->cpu_capacity_orig > rq->rd->max_cpu_capacity)
>>>> +			rq->rd->max_cpu_capacity = rq->cpu_capacity_orig;
>>>>  	}
>>>
>>> Should you not set that _before_ cpu_attach_domain(), such that the
>>> state is up-to-date when its published?
>>
>> yes, much better.
>>
>>> Also, since its lockless, should we not use {READ,WRITE}_ONCE() with it?
>>
>> You mean for rq->rd->max_cpu_capacity ? IMHO, there is a data dependency
>> between the read and the write and the code only runs on one cpu.
>>
>> I assume here that this is related to item 2 'Overlapping loads and
>> stores within a particular CPU ...' in GUARANTEES of
>> doc/Documentation/memory-barriers.txt.
>>
>> Do I miss something?
> 
> Well, the value 'rd->max_cpu_capacity' is read by all CPUs attached to
> the root_domain, right? So CPUs already attached can observe this change
> when we update the value, we want them to observe either the old or the
> new max value, not a random mix of bytes.
> 
> {READ,WRITE}_ONCE() ensure whole word load/store, iow they avoid
> load/store-tearing.
> 

OK, thanks, will add them.

So this maps to the point '(*) For aligned memory locations whose size
allows them to be accessed with a single memory-reference instruction,
prevents "load tearing" and "store tearing," ...' under 'The READ_ONCE()
and WRITE_ONCE() functions can prevent any number of optimizations ...'
section in the 'COMPILER BARRIER' paragraph in
Documentation/memory-barriers.txt.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web