Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1157780 > unrolled thread

Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

Started byMike Galbraith <umgwanakikbuti@gmail.com>
First post2015-06-03 17:40 +0200
Last post2015-06-03 19:20 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for  BALANCE_WAKE Mike Galbraith <umgwanakikbuti@gmail.com> - 2015-06-03 17:40 +0200
    Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for  BALANCE_WAKE Josef Bacik <jbacik@fb.com> - 2015-06-03 18:00 +0200
      Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for  BALANCE_WAKE Mike Galbraith <umgwanakikbuti@gmail.com> - 2015-06-03 19:00 +0200
        Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for  BALANCE_WAKE Josef Bacik <jbacik@fb.com> - 2015-06-03 19:20 +0200

#1157780 — Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2015-06-03 17:40 +0200
SubjectRe: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE
Message-ID<pxiVQ-5gl-9@gated-at.bofh.it>
On Wed, 2015-06-03 at 16:24 +0200, Peter Zijlstra wrote:
> On Wed, 2015-06-03 at 10:12 -0400, Rik van Riel wrote:
> 
> > There is a policy vs mechanism thing here. Ingo and Peter
> > are worried about the overhead in the mechanism of finding
> > an idle CPU.  Your measurements show that the policy of
> > finding an idle CPU is the correct one.
> 
> For his workload; I'm sure I can find a workload where it hurts.
> 
> In fact, I'm fairly sure Mike knows one from the top of his head, seeing
> how he's the one playing about trying to shrink that idle search :-)

Like anything where scheduling latency doesn't heavily dominate.  Even
if searching were free, bounces aren't, even for the very light.

	-Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1157801

FromJosef Bacik <jbacik@fb.com>
Date2015-06-03 18:00 +0200
Message-ID<pxjfc-5DK-21@gated-at.bofh.it>
In reply to#1157780
On 06/03/2015 11:30 AM, Mike Galbraith wrote:
> On Wed, 2015-06-03 at 16:24 +0200, Peter Zijlstra wrote:
>> On Wed, 2015-06-03 at 10:12 -0400, Rik van Riel wrote:
>>
>>> There is a policy vs mechanism thing here. Ingo and Peter
>>> are worried about the overhead in the mechanism of finding
>>> an idle CPU.  Your measurements show that the policy of
>>> finding an idle CPU is the correct one.
>>
>> For his workload; I'm sure I can find a workload where it hurts.
>>
>> In fact, I'm fairly sure Mike knows one from the top of his head, seeing
>> how he's the one playing about trying to shrink that idle search :-)
>
> Like anything where scheduling latency doesn't heavily dominate.  Even
> if searching were free, bounces aren't, even for the very light.
>

If scheduling latency doesn't hurt then making the search shouldn't 
matter should it?  I get that migrations aren't free, but it seems like 
they can't hurt that much.  This application is huge, it's our 
webserver, we're doing like 400 requests per second on these things, and 
hands down moving stuff to idle cpus is beating the pants off of staying 
on the same cpu.  Is there a specific workload I could build a test for 
that you think this approach would hurt?  Thanks,

Josef

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157854

FromMike Galbraith <umgwanakikbuti@gmail.com>
Date2015-06-03 19:00 +0200
Message-ID<pxkbi-71b-43@gated-at.bofh.it>
In reply to#1157801
On Wed, 2015-06-03 at 11:57 -0400, Josef Bacik wrote:
> On 06/03/2015 11:30 AM, Mike Galbraith wrote:
> > On Wed, 2015-06-03 at 16:24 +0200, Peter Zijlstra wrote:
> >> On Wed, 2015-06-03 at 10:12 -0400, Rik van Riel wrote:
> >>
> >>> There is a policy vs mechanism thing here. Ingo and Peter
> >>> are worried about the overhead in the mechanism of finding
> >>> an idle CPU.  Your measurements show that the policy of
> >>> finding an idle CPU is the correct one.
> >>
> >> For his workload; I'm sure I can find a workload where it hurts.
> >>
> >> In fact, I'm fairly sure Mike knows one from the top of his head, seeing
> >> how he's the one playing about trying to shrink that idle search :-)
> >
> > Like anything where scheduling latency doesn't heavily dominate.  Even
> > if searching were free, bounces aren't, even for the very light.
> >
> 
> If scheduling latency doesn't hurt then making the search shouldn't 
> matter should it?  I get that migrations aren't free, but it seems like 
> they can't hurt that much.

Nah, they don't hurt much :)

commit e0a79f529d5ba2507486d498b25da40911d95cf6
Author: Mike Galbraith <bitbucket@online.de>
Date:   Mon Jan 28 12:19:25 2013 +0100

    sched: Fix select_idle_sibling() bouncing cow syndrome
    
    If the previous CPU is cache affine and idle, select it.
    
    The current implementation simply traverses the sd_llc domain,
    taking the first idle CPU encountered, which walks buddy pairs
    hand in hand over the package, inflicting excruciating pain.
    
    1 tbench pair (worst case) in a 10 core + SMT package:
    
      pre   15.22 MB/sec 1 procs
      post 252.01 MB/sec 1 procs
    

>   This application is huge, it's our 
> webserver, we're doing like 400 requests per second on these things, and 
> hands down moving stuff to idle cpus is beating the pants off of staying 
> on the same cpu.  Is there a specific workload I could build a test for 
> that you think this approach would hurt?  Thanks,

Search cost hurts fast movers, as does dragging even a small footprint
all over the place, as you can see above.

	-Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1157891

FromJosef Bacik <jbacik@fb.com>
Date2015-06-03 19:20 +0200
Message-ID<pxkuB-7Ey-9@gated-at.bofh.it>
In reply to#1157854
On 06/03/2015 12:53 PM, Mike Galbraith wrote:
> On Wed, 2015-06-03 at 11:57 -0400, Josef Bacik wrote:
>> On 06/03/2015 11:30 AM, Mike Galbraith wrote:
>>> On Wed, 2015-06-03 at 16:24 +0200, Peter Zijlstra wrote:
>>>> On Wed, 2015-06-03 at 10:12 -0400, Rik van Riel wrote:
>>>>
>>>>> There is a policy vs mechanism thing here. Ingo and Peter
>>>>> are worried about the overhead in the mechanism of finding
>>>>> an idle CPU.  Your measurements show that the policy of
>>>>> finding an idle CPU is the correct one.
>>>>
>>>> For his workload; I'm sure I can find a workload where it hurts.
>>>>
>>>> In fact, I'm fairly sure Mike knows one from the top of his head, seeing
>>>> how he's the one playing about trying to shrink that idle search :-)
>>>
>>> Like anything where scheduling latency doesn't heavily dominate.  Even
>>> if searching were free, bounces aren't, even for the very light.
>>>
>>
>> If scheduling latency doesn't hurt then making the search shouldn't
>> matter should it?  I get that migrations aren't free, but it seems like
>> they can't hurt that much.
>
> Nah, they don't hurt much :)
>
> commit e0a79f529d5ba2507486d498b25da40911d95cf6
> Author: Mike Galbraith <bitbucket@online.de>
> Date:   Mon Jan 28 12:19:25 2013 +0100
>
>      sched: Fix select_idle_sibling() bouncing cow syndrome
>
>      If the previous CPU is cache affine and idle, select it.
>
>      The current implementation simply traverses the sd_llc domain,
>      taking the first idle CPU encountered, which walks buddy pairs
>      hand in hand over the package, inflicting excruciating pain.
>
>      1 tbench pair (worst case) in a 10 core + SMT package:
>
>        pre   15.22 MB/sec 1 procs
>        post 252.01 MB/sec 1 procs
>
>
>>    This application is huge, it's our
>> webserver, we're doing like 400 requests per second on these things, and
>> hands down moving stuff to idle cpus is beating the pants off of staying
>> on the same cpu.  Is there a specific workload I could build a test for
>> that you think this approach would hurt?  Thanks,
>
> Search cost hurts fast movers, as does dragging even a small footprint
> all over the place, as you can see above.
>

Eesh ok, do you happen to remember how you ran tbench so I can add it to 
my tests here?  In addition to fixing this problem we're also interested 
in tracking performance of new kernels so we don't have to do this "what 
the hell went wrong in the last 6 releases" dance every year, so I'm 
throwing every performance thing we find useful in our test 
infrastructure.  Thanks,

Josef

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web