Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1641389

Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically

Show all headers | View raw

Hi,

On 12/05/17 10:25, Steven Rostedt wrote:
> On Fri, 12 May 2017 14:48:45 +0900
> Byungchul Park <byungchul.park@lge.com> wrote:
> 
> > cpudl.elements is an instance that should be protected with a spin lock.
> > Without it, the code would be insane.
> 
> And how much contention will this add? Spin locks in the scheduler code
> that are shared among a domain can cause huge latency. This was why I
> worked hard not to add any in the cpupri code.
> 
> 
> > 
> > Current cpudl_find() has problems like,
> > 
> >    1. cpudl.elements[0].cpu might not match with cpudl.elements[0].dl.
> >    2. cpudl.elements[0].dl(u64) might not be referred atomically.
> >    3. Two cpudl_maximum()s might return different values.
> >    4. It's just insane.
> 
> And lockless algorithms usually are insane. But locks come with a huge
> cost. What happens when we have 32 core domains. This can cause
> tremendous contention and makes the entire cpu priority for deadlines
> useless. Might as well rip out the code.
> 

Right. So, rationale for not taking any lock in the find() path (at the
risk of getting bogus values) is that we don't want to pay to much in
terms of contention, when also considering the fact that find_lock_later_
rq() might then release the rq lock, possibly making the search useless
(if things change in the meantime anyway). The update path is instead
guarded by a lock, to ensure consistency.

Experiments on reasonably big machines (48-cores IIRC) showed that the
approach was "good enough", so we looked somewhere else to improve
things (as there are many to improve :). This of course doesn't prevent
us to look at this again now and see if we need to do something about it.

Having numbers about introduced overhead and wrong decisions caused by
the lockless find() path would help a lot understanding what (and can)
be done.

Thanks!

- Juri

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH v4 0/5] Make find_later_rq() choose a closer cpu in topology Byungchul Park <byungchul.park@lge.com> - 2017-05-12 07:50 +0200
  [PATCH v4 3/5] sched/deadline: Change return value of cpudl_find() Byungchul Park <byungchul.park@lge.com> - 2017-05-12 07:50 +0200
  [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Byungchul Park <byungchul.park@lge.com> - 2017-05-12 07:50 +0200
    Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements  atomically Steven Rostedt <rostedt@goodmis.org> - 2017-05-12 16:30 +0200
      Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Juri Lelli <juri.lelli@arm.com> - 2017-05-15 10:40 +0200
        Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Byungchul Park <byungchul.park@lge.com> - 2017-05-16 09:10 +0200
      Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Byungchul Park <byungchul.park@lge.com> - 2017-05-16 09:00 +0200
        Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Juri Lelli <juri.lelli@arm.com> - 2017-05-16 12:40 +0200
          Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements  atomically Steven Rostedt <rostedt@goodmis.org> - 2017-05-16 15:20 +0200
            Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Byungchul Park <byungchul.park@lge.com> - 2017-05-23 03:20 +0200
  [PATCH v4 4/5] sched/deadline: Add support for SD_PREFER_SIBLING on find_later_rq() Byungchul Park <byungchul.park@lge.com> - 2017-05-12 08:00 +0200
  [PATCH v4 5/5] sched/rt: Add support for SD_PREFER_SIBLING on find_lowest_rq() Byungchul Park <byungchul.park@lge.com> - 2017-05-12 08:00 +0200

csiph-web