Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps -11.3% regression

From	Joel Fernandes <joelaf@google.com>
Newsgroups	linux.kernel
Subject	Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps -11.3% regression
Date	2017-09-11 08:40 +0200
Message-ID	<uoqxX-7vR-3@gated-at.bofh.it> (permalink)
References	<uiUfn-nC-1@gated-at.bofh.it> <uoaMx-4sT-3@gated-at.bofh.it> <uodKq-6F7-15@gated-at.bofh.it> <uon73-4Ue-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi Mike,
Thanks a lot for sharing the history of this.

On Sun, Sep 10, 2017 at 7:55 PM, Mike Galbraith <efault@gmx.de> wrote:
> On Sun, 2017-09-10 at 09:53 -0700, Joel Fernandes wrote:
>>
>> Anyone know what in the netperf test triggers use of the sync flag?
>
> homer:..kernel/linux-master # git grep wake_up_interruptible_sync_poll net
> net/core/sock.c:                wake_up_interruptible_sync_poll(&wq->wait, POLLIN | POLLPRI |
> net/core/sock.c:                        wake_up_interruptible_sync_poll(&wq->wait, POLLOUT |
> net/sctp/socket.c:              wake_up_interruptible_sync_poll(&wq->wait, POLLIN |
> net/smc/smc_rx.c:               wake_up_interruptible_sync_poll(&wq->wait, POLLIN | POLLPRI |
> net/tipc/socket.c:              wake_up_interruptible_sync_poll(&wq->wait, POLLOUT |
> net/tipc/socket.c:              wake_up_interruptible_sync_poll(&wq->wait, POLLIN |
> net/unix/af_unix.c:                     wake_up_interruptible_sync_poll(&wq->wait,
> net/unix/af_unix.c:             wake_up_interruptible_sync_poll(&u->peer_wait,
>
> The same as metric tons of other stuff.
>
> Once upon a time, we had avg_overlap to help decide whether to wake
> core affine or not, on top of the wake_affine() imbalance constraint,
> but instrumentation showed it to be too error prone, so it had to die.
>  These days, an affine wakeup generally means cache affine, and the
> sync hint gives you a wee bit more chance of migration near to tasty
> hot data being approved.
>
> The sync hint was born back in the bad old days, when communicating
> tasks not sharing L2 may as well have been talking over two tin cans
> and a limp string.  These days, things are oodles better, but truly
> synchronous stuff could still benefit from core affinity (up to hugely
> for very fast/light stuff) if it weren't for all the caveats that can
> lead to tossing concurrency opportunities out the window.

Cool, thanks. For this test I suspect its the other way? I think the
reason why regresses is that the 'nr_running < 2' check is too weak of
a check to prevent sync in all bad situations ('bad' being pulling a
task to a crowded CPU). Could we maybe be having a situation for this
test where if the blocked load a CPU is high (many tasks recently were
running on it and went to sleep), then the nr_running < 2  is a false
positive and in such a scenario we listened to the sync flag when we
shouldn't have?

To make the load check more meaningful, I am thinking if using
wake_affine()'s balance check is a better thing to do than the
'nr_running < 2' check I used in this patch. Then again, since commit
3fed382b46baac ("sched/numa: Implement NUMA node level wake_affine()",
wake_affine() doesn't do balance check for CPUs within a socket so
probably bringing back something like the *old* wake_affine that
checked load between different CPUs within a socket is needed to avoid
a potentially disastrous sync decision?  The commit I refer to was
added with the reason that select_idle_sibling was selecting cores
anywhere within a socket, but with my patch we're more specifically
selecting the waker's CPU on passing the sync flag. Could you share
your thoughts about this?

I will run some tracing on this netperf test and try to understand the
undesirable behavior better as well,

thanks,

-Joel

Thread

[PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Joel Fernandes <joelaf@google.com> - 2017-08-27 03:10 +0200
  Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-27 07:50 +0200
    Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-27 08:10 +0200
      Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Joel Fernandes <joelaf@google.com> - 2017-08-27 08:40 +0200
        Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-27 09:20 +0200
        Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-27 20:10 +0200
          Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Joel Fernandes <joelaf@google.com> - 2017-08-28 07:30 +0200
            Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-28 08:20 +0200
              Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-28 08:50 +0200
              Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Joel Fernandes <joelaf@google.com> - 2017-08-28 18:30 +0200
                Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-28 19:20 +0200
    Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Joel Fernandes <joelaf@google.com> - 2017-08-27 08:30 +0200
    Re: [PATCH RFC/RFT] sched/fair: Improve the behavior of sync flag Mike Galbraith <efault@gmx.de> - 2017-08-27 09:00 +0200
  Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Joel Fernandes <joelaf@google.com> - 2017-09-10 19:00 +0200
    Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Mike Galbraith <efault@gmx.de> - 2017-09-11 05:00 +0200
      Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Joel Fernandes <joelaf@google.com> - 2017-09-11 08:40 +0200
        Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Mike Galbraith <efault@gmx.de> - 2017-09-11 10:10 +0200
        Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Rik van Riel <riel@redhat.com> - 2017-09-14 18:00 +0200
          Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Mike Galbraith <efault@gmx.de> - 2017-09-15 06:10 +0200
          Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Joel Fernandes <joelaf@google.com> - 2017-09-17 08:50 +0200
            Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Mike Galbraith <efault@gmx.de> - 2017-09-17 18:50 +0200
              Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Joel Fernandes <joelaf@google.com> - 2017-09-17 23:50 +0200
                Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Mike Galbraith <efault@gmx.de> - 2017-09-18 07:40 +0200
                Re: [lkp-robot] [sched/fair] 6d46bd3d97: netperf.Throughput_tps  -11.3% regression Joel Fernandes <joelaf@google.com> - 2017-09-25 01:50 +0200

csiph-web