Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1464277 > unrolled thread

Re: [PATCH v2] locking/mutex: Prevent lock starvation when spinning is enabled

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v2] locking/mutex: Prevent lock starvation when spinning  is enabled Wanpeng Li <kernellwp@gmail.com> - 2016-08-17 03:50 +0200
    Re: [PATCH v2] locking/mutex: Prevent lock starvation when spinning  is enabled Jason Low <jason.low2@hpe.com> - 2016-08-17 20:40 +0200
      Re: [PATCH v2] locking/mutex: Prevent lock starvation when spinning  is enabled Wanpeng Li <kernellwp@gmail.com> - 2016-08-18 02:40 +0200

#1464277 — Re: [PATCH v2] locking/mutex: Prevent lock starvation when spinning is enabled

2016-08-11 2:44 GMT+08:00 Jason Low <jason.low2@hpe.com>:
> Imre reported an issue where threads are getting starved when trying
> to acquire a mutex. Threads acquiring a mutex can get arbitrarily delayed
> sleeping on a mutex because other threads can continually steal the lock
> in the fastpath and/or through optimistic spinning.
>
> Waiman has developed patches that allow waiters to return to optimistic
> spinning, thus reducing the probability that starvation occurs. However,
> Imre still sees this starvation problem in the workloads when optimistic
> spinning is disabled.
>
> This patch adds an additional boolean to the mutex that gets used in
> the CONFIG_SMP && !CONFIG_MUTEX_SPIN_ON_OWNER cases. The flag signifies
> whether or not other threads need to yield to a waiter and gets set
> when a waiter spends too much time waiting for the mutex. The threshold
> is currently set to 16 wakeups, and once the wakeup threshold is exceeded,
> other threads must yield to the top waiter. The flag gets cleared
> immediately after the top waiter acquires the mutex.

There is a subtle difference between this patch and Waiman's. Waiman's
patch will boost any waiter-spinner which is woken up, however, this
patch will boost the top waiter once the number of any waiter-spinners
woken up reaches the threshold. We can't get any benefit if the
resource holder which top waiter is waiting for still not release the
resource.

Regards,
Wanpeng Li

[toc] | [next] | [standalone]

#1464713

Hi Wanpeng,

On Wed, 2016-08-17 at 09:41 +0800, Wanpeng Li wrote:
> 2016-08-11 2:44 GMT+08:00 Jason Low <jason.low2@hpe.com>:
> > Imre reported an issue where threads are getting starved when trying
> > to acquire a mutex. Threads acquiring a mutex can get arbitrarily delayed
> > sleeping on a mutex because other threads can continually steal the lock
> > in the fastpath and/or through optimistic spinning.
> >
> > Waiman has developed patches that allow waiters to return to optimistic
> > spinning, thus reducing the probability that starvation occurs. However,
> > Imre still sees this starvation problem in the workloads when optimistic
> > spinning is disabled.
> >
> > This patch adds an additional boolean to the mutex that gets used in
> > the CONFIG_SMP && !CONFIG_MUTEX_SPIN_ON_OWNER cases. The flag signifies
> > whether or not other threads need to yield to a waiter and gets set
> > when a waiter spends too much time waiting for the mutex. The threshold
> > is currently set to 16 wakeups, and once the wakeup threshold is exceeded,
> > other threads must yield to the top waiter. The flag gets cleared
> > immediately after the top waiter acquires the mutex.
> 
> There is a subtle difference between this patch and Waiman's. Waiman's
> patch will boost any waiter-spinner which is woken up, however, this
> patch will boost the top waiter once the number of any waiter-spinners
> woken up reaches the threshold.

Correct, since when spinning is disabled, we still want to generally
allow other threads to steal the lock even if there are waiters in order
to keep performance good, and only yield the lock when a waiter is
getting 'starved'.

> We can't get any benefit if the
> resource holder which top waiter is waiting for still not release the
> resource.

If the resource holder does not release the resource, that sounds like
an issue with the lock holder.

Unless you're referring to how this doesn't provide immediate benefit to
the top waiter, in which case, I think that is okay since the goal of
the patch is to prevent starvation. We tried disabling 'lock stealing'
anytime there are waiters and that proved to reduce performance by quite
a bit in some workloads.

[toc] | [prev] | [next] | [standalone]

#1464859

2016-08-18 2:30 GMT+08:00 Jason Low <jason.low2@hpe.com>:
> Hi Wanpeng,
>
> On Wed, 2016-08-17 at 09:41 +0800, Wanpeng Li wrote:
>> 2016-08-11 2:44 GMT+08:00 Jason Low <jason.low2@hpe.com>:
>> > Imre reported an issue where threads are getting starved when trying
>> > to acquire a mutex. Threads acquiring a mutex can get arbitrarily delayed
>> > sleeping on a mutex because other threads can continually steal the lock
>> > in the fastpath and/or through optimistic spinning.
>> >
>> > Waiman has developed patches that allow waiters to return to optimistic
>> > spinning, thus reducing the probability that starvation occurs. However,
>> > Imre still sees this starvation problem in the workloads when optimistic
>> > spinning is disabled.
>> >
>> > This patch adds an additional boolean to the mutex that gets used in
>> > the CONFIG_SMP && !CONFIG_MUTEX_SPIN_ON_OWNER cases. The flag signifies
>> > whether or not other threads need to yield to a waiter and gets set
>> > when a waiter spends too much time waiting for the mutex. The threshold
>> > is currently set to 16 wakeups, and once the wakeup threshold is exceeded,
>> > other threads must yield to the top waiter. The flag gets cleared
>> > immediately after the top waiter acquires the mutex.
>>
>> There is a subtle difference between this patch and Waiman's. Waiman's
>> patch will boost any waiter-spinner which is woken up, however, this
>> patch will boost the top waiter once the number of any waiter-spinners
>> woken up reaches the threshold.
>
> Correct, since when spinning is disabled, we still want to generally
> allow other threads to steal the lock even if there are waiters in order
> to keep performance good, and only yield the lock when a waiter is
> getting 'starved'.
>
>> We can't get any benefit if the
>> resource holder which top waiter is waiting for still not release the
>> resource.
>
> If the resource holder does not release the resource, that sounds like
> an issue with the lock holder.
>
> Unless you're referring to how this doesn't provide immediate benefit to
> the top waiter,

Yes.

> in which case, I think that is okay since the goal of
> the patch is to prevent starvation. We tried disabling 'lock stealing'
> anytime there are waiters and that proved to reduce performance by quite
> a bit in some workloads.

Thanks for the clarification. :)

Regards,
Wanpeng Li

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web