Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1741435 > unrolled thread

Re: [RFC] futex: hashbucket as list of futex instead of waiters

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC] futex: hashbucket as list of futex instead of waiters Peter Zijlstra <peterz@infradead.org> - 2017-09-28 13:00 +0200
    Re: [RFC] futex: hashbucket as list of futex instead of waiters Thomas Gleixner <tglx@linutronix.de> - 2017-09-28 14:20 +0200
    Re: [RFC] futex: hashbucket as list of futex instead of waiters Gerd Gerats <gerd.gerats.lkml@gmx.de> - 2017-09-29 00:50 +0200
      Re: [RFC] futex: hashbucket as list of futex instead of waiters Peter Zijlstra <peterz@infradead.org> - 2017-09-29 10:40 +0200

#1741435 — Re: [RFC] futex: hashbucket as list of futex instead of waiters

On Sun, Sep 10, 2017 at 11:41:58PM +0200, Gerd Gerats wrote:
> When using futex as a condition variable, for example: to manage a
> threadpool, there may be a lot of threads inside the futex_wait to sleep on
> this futex. The futex_hash_bucket consists therefore of many struct futex_q
> for the same futex.
> 
> On bad luck another futex, used as mutex, hashed into the same bucket.
> Every futex_wake on this mutex, has to scan the whole chain of above waiter
> to find the struct futex_q for this mutex. For non-unusual threadpool sizes
> of more than 20, this should be a considerable effort.
> 
> I therefore suggest to include in the hash-bucketchain only one struct
> futex_q per futex and to queue additional waiter in an extrachain at the
> 'top' futex_q entry. Thus different futex are isolated from each other, the
> cost of a hash collision is reduced.

So I don't dislike that idea.. however

> To show the idea, I added a sample patch. Here, the plist is exchanged for
> a futex-specific implementation. kernel/pring.h is certainly not not the
> right place.

So I suppose the purpose of that plist in futex is to enable waking up
the highest prio waiter, but with the advent of SCHED_DEADLINE that no
longer works.

I think Thomas resisted going the RB-tree route earlier..

[toc] | [next] | [standalone]

#1741482

On Thu, 28 Sep 2017, Peter Zijlstra wrote:
> On Sun, Sep 10, 2017 at 11:41:58PM +0200, Gerd Gerats wrote:
> > When using futex as a condition variable, for example: to manage a
> > threadpool, there may be a lot of threads inside the futex_wait to sleep on
> > this futex. The futex_hash_bucket consists therefore of many struct futex_q
> > for the same futex.
> > 
> > On bad luck another futex, used as mutex, hashed into the same bucket.
> > Every futex_wake on this mutex, has to scan the whole chain of above waiter
> > to find the struct futex_q for this mutex. For non-unusual threadpool sizes
> > of more than 20, this should be a considerable effort.
> > 
> > I therefore suggest to include in the hash-bucketchain only one struct
> > futex_q per futex and to queue additional waiter in an extrachain at the
> > 'top' futex_q entry. Thus different futex are isolated from each other, the
> > cost of a hash collision is reduced.
> 
> So I don't dislike that idea.. however
> 
> > To show the idea, I added a sample patch. Here, the plist is exchanged for
> > a futex-specific implementation. kernel/pring.h is certainly not not the
> > right place.
> 
> So I suppose the purpose of that plist in futex is to enable waking up
> the highest prio waiter, but with the advent of SCHED_DEADLINE that no
> longer works.
> 
> I think Thomas resisted going the RB-tree route earlier..

The only problem I see with the RB tree is the performance overhead and
people are constantly complaining about futex performance anyway. But it
might be worth a try.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1741839

On Thu, Sep 28, 2017 at 12:58:14PM +0200, Peter Zijlstra wrote:
> On Sun, Sep 10, 2017 at 11:41:58PM +0200, Gerd Gerats wrote:
> > When using futex as a condition variable, for example: to manage a
> > threadpool, there may be a lot of threads inside the futex_wait to sleep on
> > this futex. The futex_hash_bucket consists therefore of many struct futex_q
> > for the same futex.
> > 
> > On bad luck another futex, used as mutex, hashed into the same bucket.
> > Every futex_wake on this mutex, has to scan the whole chain of above waiter
> > to find the struct futex_q for this mutex. For non-unusual threadpool sizes
> > of more than 20, this should be a considerable effort.
> > 
> > I therefore suggest to include in the hash-bucketchain only one struct
> > futex_q per futex and to queue additional waiter in an extrachain at the
> > 'top' futex_q entry. Thus different futex are isolated from each other, the
> > cost of a hash collision is reduced.
> 
> So I don't dislike that idea.. however
> 
> > To show the idea, I added a sample patch. Here, the plist is exchanged for
> > a futex-specific implementation. kernel/pring.h is certainly not not the
> > right place.
> 
> So I suppose the purpose of that plist in futex is to enable waking up
> the highest prio waiter, but with the advent of SCHED_DEADLINE that no
> longer works.

I do not understand, plist is the original data structure. I assume, it works
together with SCHED_DEADLINE. Or is there something on the way not pressent in
4.13.

pring replaces this data structure, retaining prio sorting rules, by a
head-less ring. This ring contains all waiter for one futex_key, ordered the
same way as by plist.

Thanx, Gerd

[toc] | [prev] | [next] | [standalone]

#1741985

On Fri, Sep 29, 2017 at 12:39:32AM +0200, Gerd Gerats wrote:
> > 
> > So I suppose the purpose of that plist in futex is to enable waking up
> > the highest prio waiter, but with the advent of SCHED_DEADLINE that no
> > longer works.
> 
> I do not understand, plist is the original data structure. I assume, it works
> together with SCHED_DEADLINE.

plist does not work well with DEADLINE, all deadline tasks will get
lumped together at prio -1; prio as a single integer doesn't really work
for DEADLINE.

We've converted the RT-mutes over to RB-trees a fair while ago; see
commit:

  fb00aca47440 ("rtmutex: Turn the plist into an rb-tree")

> Or is there something on the way not pressent in 4.13.

Thomas held off on converting futex to rb-trees because, as he said
elsewhere in this thread, he's worried about the performance impact on
futexes.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web