Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1708486 > unrolled thread

Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to  improve performance on some archs Peter Zijlstra <peterz@infradead.org> - 2017-08-10 14:00 +0200
    Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve  performance on some archs Waiman Long <longman@redhat.com> - 2017-08-10 15:30 +0200
      Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve  performance on some archs Waiman Long <longman@redhat.com> - 2017-08-10 16:00 +0200
        Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to  improve performance on some archs Peter Zijlstra <peterz@infradead.org> - 2017-08-10 18:20 +0200
          Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve  performance on some archs Waiman Long <longman@redhat.com> - 2017-08-10 18:30 +0200
            Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve  performance on some archs Waiman Long <longman@redhat.com> - 2017-08-10 20:20 +0200
              Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to  improve performance on some archs Peter Zijlstra <peterz@infradead.org> - 2017-08-11 11:10 +0200

#1708486 — Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
> 
>   # of thread     w/o patch    with patch      % Change
>   -----------     ---------    ----------      --------
>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%

Waiman, could you run those numbers again but with the below 'fixed' ?

> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
>  	 * observe its next->locked value and advance itself.
>  	 *
>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
> +	 *
> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
> +	 * must be ordered before the read of pn->state in the cmpxchg()
> +	 * below for the code to work correctly. However, this is not
> +	 * guaranteed on all architectures when the cmpxchg() call fails.
> +	 * Both x86 and PPC can provide that guarantee, but other
> +	 * architectures not necessarily.
>  	 */

	smp_mb();

>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
>  		return;

Ideally this Power CPU can optimize back-to-back SYNC instructions, but
who knows...

[toc] | [next] | [standalone]

#1708627 — Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

On 08/10/2017 07:50 AM, Peter Zijlstra wrote:
> On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
>>   # of thread     w/o patch    with patch      % Change
>>   -----------     ---------    ----------      --------
>>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
>>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
>>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%
> Waiman, could you run those numbers again but with the below 'fixed' ?
>
>> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
>>  	 * observe its next->locked value and advance itself.
>>  	 *
>>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
>> +	 *
>> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
>> +	 * must be ordered before the read of pn->state in the cmpxchg()
>> +	 * below for the code to work correctly. However, this is not
>> +	 * guaranteed on all architectures when the cmpxchg() call fails.
>> +	 * Both x86 and PPC can provide that guarantee, but other
>> +	 * architectures not necessarily.
>>  	 */
> 	smp_mb();
>
>>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
>>  		return;
> Ideally this Power CPU can optimize back-to-back SYNC instructions, but
> who knows...

Yes, I can run the numbers again. However, the changes here is in the
slowpath. My current patch optimizes the fast path only and my original
test doesn't stress the slowpath at all, I think. I will have to make
some changes to stress the slowpath.

Cheers,
Longman

[toc] | [prev] | [next] | [standalone]

#1708652 — Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

On 08/10/2017 09:27 AM, Waiman Long wrote:
> On 08/10/2017 07:50 AM, Peter Zijlstra wrote:
>> On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
>>>   # of thread     w/o patch    with patch      % Change
>>>   -----------     ---------    ----------      --------
>>>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
>>>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
>>>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%
>> Waiman, could you run those numbers again but with the below 'fixed' ?
>>
>>> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
>>>  	 * observe its next->locked value and advance itself.
>>>  	 *
>>>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
>>> +	 *
>>> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
>>> +	 * must be ordered before the read of pn->state in the cmpxchg()
>>> +	 * below for the code to work correctly. However, this is not
>>> +	 * guaranteed on all architectures when the cmpxchg() call fails.
>>> +	 * Both x86 and PPC can provide that guarantee, but other
>>> +	 * architectures not necessarily.
>>>  	 */
>> 	smp_mb();
>>
>>>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
>>>  		return;
>> Ideally this Power CPU can optimize back-to-back SYNC instructions, but
>> who knows...
> Yes, I can run the numbers again. However, the changes here is in the
> slowpath. My current patch optimizes the fast path only and my original
> test doesn't stress the slowpath at all, I think. I will have to make
> some changes to stress the slowpath.

Looking at past emails, I remember why I put the comment there. Putting
an smp_mb() here will definitely has an negative performance impact on
x86. So I put in the comment here to remind me that the current code may
not work for ARM64.

To fix that, my current thought is to have a cmpxchg variant that
guarantees ordering for both success and failure, for example,
cmpxchg_ordered(). In that way, we only need to insert the barrier for
architectures that need it. That will be a separate patch instead of
integrating into this one.

Cheers,
Longman

[toc] | [prev] | [next] | [standalone]

#1708793

On Thu, Aug 10, 2017 at 09:58:57AM -0400, Waiman Long wrote:
> On 08/10/2017 09:27 AM, Waiman Long wrote:
> > On 08/10/2017 07:50 AM, Peter Zijlstra wrote:
> >> On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
> >>>   # of thread     w/o patch    with patch      % Change
> >>>   -----------     ---------    ----------      --------
> >>>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
> >>>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
> >>>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%
> >> Waiman, could you run those numbers again but with the below 'fixed' ?
> >>
> >>> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
> >>>  	 * observe its next->locked value and advance itself.
> >>>  	 *
> >>>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
> >>> +	 *
> >>> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
> >>> +	 * must be ordered before the read of pn->state in the cmpxchg()
> >>> +	 * below for the code to work correctly. However, this is not
> >>> +	 * guaranteed on all architectures when the cmpxchg() call fails.
> >>> +	 * Both x86 and PPC can provide that guarantee, but other
> >>> +	 * architectures not necessarily.
> >>>  	 */
> >> 	smp_mb();
> >>
> >>>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
> >>>  		return;
> >> Ideally this Power CPU can optimize back-to-back SYNC instructions, but
> >> who knows...
> > Yes, I can run the numbers again. However, the changes here is in the
> > slowpath. My current patch optimizes the fast path only and my original
> > test doesn't stress the slowpath at all, I think. I will have to make
> > some changes to stress the slowpath.
> 
> Looking at past emails, I remember why I put the comment there. Putting
> an smp_mb() here will definitely has an negative performance impact on
> x86. So I put in the comment here to remind me that the current code may
> not work for ARM64.
> 
> To fix that, my current thought is to have a cmpxchg variant that
> guarantees ordering for both success and failure, for example,
> cmpxchg_ordered(). In that way, we only need to insert the barrier for
> architectures that need it. That will be a separate patch instead of
> integrating into this one.

Might as well do an explicit:

	smp_mb__before_atomic()
	cmpxchg_relaxed()
	smp_mb__after_atomic()

I suppose and not introduce new primitives.

[toc] | [prev] | [next] | [standalone]

#1708816 — Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

On 08/10/2017 12:15 PM, Peter Zijlstra wrote:
> On Thu, Aug 10, 2017 at 09:58:57AM -0400, Waiman Long wrote:
>> On 08/10/2017 09:27 AM, Waiman Long wrote:
>>> On 08/10/2017 07:50 AM, Peter Zijlstra wrote:
>>>> On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
>>>>>   # of thread     w/o patch    with patch      % Change
>>>>>   -----------     ---------    ----------      --------
>>>>>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
>>>>>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
>>>>>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%
>>>> Waiman, could you run those numbers again but with the below 'fixed' ?
>>>>
>>>>> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
>>>>>  	 * observe its next->locked value and advance itself.
>>>>>  	 *
>>>>>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
>>>>> +	 *
>>>>> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
>>>>> +	 * must be ordered before the read of pn->state in the cmpxchg()
>>>>> +	 * below for the code to work correctly. However, this is not
>>>>> +	 * guaranteed on all architectures when the cmpxchg() call fails.
>>>>> +	 * Both x86 and PPC can provide that guarantee, but other
>>>>> +	 * architectures not necessarily.
>>>>>  	 */
>>>> 	smp_mb();
>>>>
>>>>>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
>>>>>  		return;
>>>> Ideally this Power CPU can optimize back-to-back SYNC instructions, but
>>>> who knows...
>>> Yes, I can run the numbers again. However, the changes here is in the
>>> slowpath. My current patch optimizes the fast path only and my original
>>> test doesn't stress the slowpath at all, I think. I will have to make
>>> some changes to stress the slowpath.
>> Looking at past emails, I remember why I put the comment there. Putting
>> an smp_mb() here will definitely has an negative performance impact on
>> x86. So I put in the comment here to remind me that the current code may
>> not work for ARM64.
>>
>> To fix that, my current thought is to have a cmpxchg variant that
>> guarantees ordering for both success and failure, for example,
>> cmpxchg_ordered(). In that way, we only need to insert the barrier for
>> architectures that need it. That will be a separate patch instead of
>> integrating into this one.
> Might as well do an explicit:
>
> 	smp_mb__before_atomic()
> 	cmpxchg_relaxed()
> 	smp_mb__after_atomic()
>
> I suppose and not introduce new primitives.


Right. I think that will work without impacting current x86 performance.
Will update my patch accordingly.

Thanks,
Longman

[toc] | [prev] | [next] | [standalone]

#1708949 — Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

On 08/10/2017 12:22 PM, Waiman Long wrote:
> On 08/10/2017 12:15 PM, Peter Zijlstra wrote:
>> On Thu, Aug 10, 2017 at 09:58:57AM -0400, Waiman Long wrote:
>>> On 08/10/2017 09:27 AM, Waiman Long wrote:
>>>> On 08/10/2017 07:50 AM, Peter Zijlstra wrote:
>>>>> On Wed, May 24, 2017 at 09:38:28AM -0400, Waiman Long wrote:
>>>>>>   # of thread     w/o patch    with patch      % Change
>>>>>>   -----------     ---------    ----------      --------
>>>>>>        4         4053.3 Mop/s  4223.7 Mop/s     +4.2%
>>>>>>        8         3310.4 Mop/s  3406.0 Mop/s     +2.9%
>>>>>>       12         2576.4 Mop/s  2674.6 Mop/s     +3.8%
>>>>> Waiman, could you run those numbers again but with the below 'fixed' ?
>>>>>
>>>>>> @@ -361,6 +361,13 @@ static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node)
>>>>>>  	 * observe its next->locked value and advance itself.
>>>>>>  	 *
>>>>>>  	 * Matches with smp_store_mb() and cmpxchg() in pv_wait_node()
>>>>>> +	 *
>>>>>> +	 * The write to next->locked in arch_mcs_spin_unlock_contended()
>>>>>> +	 * must be ordered before the read of pn->state in the cmpxchg()
>>>>>> +	 * below for the code to work correctly. However, this is not
>>>>>> +	 * guaranteed on all architectures when the cmpxchg() call fails.
>>>>>> +	 * Both x86 and PPC can provide that guarantee, but other
>>>>>> +	 * architectures not necessarily.
>>>>>>  	 */
>>>>> 	smp_mb();
>>>>>
>>>>>>  	if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted)
>>>>>>  		return;
>>>>> Ideally this Power CPU can optimize back-to-back SYNC instructions, but
>>>>> who knows...
>>>> Yes, I can run the numbers again. However, the changes here is in the
>>>> slowpath. My current patch optimizes the fast path only and my original
>>>> test doesn't stress the slowpath at all, I think. I will have to make
>>>> some changes to stress the slowpath.
>>> Looking at past emails, I remember why I put the comment there. Putting
>>> an smp_mb() here will definitely has an negative performance impact on
>>> x86. So I put in the comment here to remind me that the current code may
>>> not work for ARM64.
>>>
>>> To fix that, my current thought is to have a cmpxchg variant that
>>> guarantees ordering for both success and failure, for example,
>>> cmpxchg_ordered(). In that way, we only need to insert the barrier for
>>> architectures that need it. That will be a separate patch instead of
>>> integrating into this one.
>> Might as well do an explicit:
>>
>> 	smp_mb__before_atomic()
>> 	cmpxchg_relaxed()
>> 	smp_mb__after_atomic()
>>
>> I suppose and not introduce new primitives.

I think we don't need smp_mb__after_atomic(). The read has to be fully
ordered, but the write part may not need it as the control dependency of
the old value should guard against incorrect action. Right?

Cheers,
Longman

[toc] | [prev] | [next] | [standalone]

#1709390

On Thu, Aug 10, 2017 at 02:18:30PM -0400, Waiman Long wrote:
> On 08/10/2017 12:22 PM, Waiman Long wrote:
> > On 08/10/2017 12:15 PM, Peter Zijlstra wrote:

> >> Might as well do an explicit:
> >>
> >> 	smp_mb__before_atomic()
> >> 	cmpxchg_relaxed()
> >> 	smp_mb__after_atomic()
> >>
> >> I suppose and not introduce new primitives.
> 
> I think we don't need smp_mb__after_atomic(). The read has to be fully
> ordered, but the write part may not need it as the control dependency of
> the old value should guard against incorrect action. Right?

You'd think that, but IIRC there was something funny about using the SC
return flag for control dependencies. Will?

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web