Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1451747 > unrolled thread

Re: [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression

Started byXin Long <lucien.xin@gmail.com>
First post2016-07-28 09:10 +0200
Last post2016-08-08 04:20 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression Xin Long <lucien.xin@gmail.com> - 2016-07-28 09:10 +0200
    Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2%  regression Aaron Lu <aaron.lu@intel.com> - 2016-08-05 05:40 +0200
      Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression Xin Long <lucien.xin@gmail.com> - 2016-08-05 14:00 +0200
        Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2%  regression Aaron Lu <aaron.lu@intel.com> - 2016-08-08 04:20 +0200

#1451747 — Re: [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression

FromXin Long <lucien.xin@gmail.com>
Date2016-07-28 09:10 +0200
SubjectRe: [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Message-ID<rZNC9-2Rd-1@gated-at.bofh.it>
On Wed, Jul 27, 2016 at 9:54 AM, kernel test robot
<xiaolong.ye@intel.com> wrote:
>
> FYI, we noticed a -37.2% regression of netperf.Throughput_Mbps due to commit:
>
> commit a6c2f792873aff332a4689717c3cd6104f46684c ("sctp: implement prsctp TTL policy")
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
>
> in testcase: netperf
> on test machine: 4 threads Ivy Bridge with 8G memory
> with following parameters:
>
>         ip: ipv4
>         runtime: 300s
>         nr_threads: 200%
>         cluster: cs-localhost
>         send_size: 10K
>         test: SCTP_STREAM_MANY
>         cpufreq_governor: performance
>
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
It doesn't make much sense to me. the codes I added cannot be
triggered without enable any pr policies. and I also did the tests in
my local environment,  the result looks normal to me compare to
prior version.

Recently the sctp performance is not stable,  as during these patches,
netperf cannot get the result, but return ENOTCONN. which may
also affect the testing. anyway we've fixed the -ENOTCONN issue
already in the latest version.

[toc] | [next] | [standalone]


#1456864 — Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression

FromAaron Lu <aaron.lu@intel.com>
Date2016-08-05 05:40 +0200
SubjectRe: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Message-ID<s2E9j-29n-1@gated-at.bofh.it>
In reply to#1451747

[Multipart message — attachments visible in raw view] — view raw

On Thu, Jul 28, 2016 at 03:01:36PM +0800, Xin Long wrote:
> On Wed, Jul 27, 2016 at 9:54 AM, kernel test robot
> <xiaolong.ye@intel.com> wrote:
> >
> > FYI, we noticed a -37.2% regression of netperf.Throughput_Mbps due to commit:
> >
> > commit a6c2f792873aff332a4689717c3cd6104f46684c ("sctp: implement prsctp TTL policy")
> > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
> >
> > in testcase: netperf
> > on test machine: 4 threads Ivy Bridge with 8G memory
> > with following parameters:
> >
> >         ip: ipv4
> >         runtime: 300s
> >         nr_threads: 200%
> >         cluster: cs-localhost
> >         send_size: 10K
> >         test: SCTP_STREAM_MANY
> >         cpufreq_governor: performance
> >
> >
> >
> > Disclaimer:
> > Results have been estimated based on internal Intel analysis and are provided
> > for informational purposes only. Any difference in system hardware or software
> > design or configuration may affect actual performance.
> >
> It doesn't make much sense to me. the codes I added cannot be
> triggered without enable any pr policies. and I also did the tests in

It seems these pr policies has to be turned on by user space, i.e.
netperf in this case?

I checked netperf's source code, it doesn't seem set any option
related to SCTP PR POLICY but I'm new to network code so I could be
wrong or missing something.

> my local environment,  the result looks normal to me compare to
> prior version.

Can you share your number?
We run netperf like this:
netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1
The full log of the run is attached for your reference.

> 
> Recently the sctp performance is not stable,  as during these patches,
> netperf cannot get the result, but return ENOTCONN. which may
> also affect the testing. anyway we've fixed the -ENOTCONN issue
> already in the latest version.

I tested commit 96b585267f55, which is Linus' git tree HEAD on 08/03, I
guess the fix you mentioned should already be in there? But
unfortunately, the throughput of netperf is still at low number(we did
the test 5 times):
$ cat */netperf.json
{
  "netperf.Throughput_Mbps": [
    2470.6974999999998
  ]
}{
  "netperf.Throughput_Mbps": [
    2486.7675
  ]
}{
  "netperf.Throughput_Mbps": [
    2478.945
  ]
}{
  "netperf.Throughput_Mbps": [
    2429.465
  ]
}{
  "netperf.Throughput_Mbps": [
    2476.9150000000004
  ]

Considering what you have said that the patch shouldn't make a
difference, the performance drop is really confusing. Any idea what
could be the cause? Thanks.

Regards,
Aaron

[toc] | [prev] | [next] | [standalone]


#1457050 — Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression

FromXin Long <lucien.xin@gmail.com>
Date2016-08-05 14:00 +0200
SubjectRe: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Message-ID<s2LXc-72j-5@gated-at.bofh.it>
In reply to#1456864
>> It doesn't make much sense to me. the codes I added cannot be
>> triggered without enable any pr policies. and I also did the tests in
>
> It seems these pr policies has to be turned on by user space, i.e.
> netperf in this case?
>
> I checked netperf's source code, it doesn't seem set any option
> related to SCTP PR POLICY but I'm new to network code so I could be
> wrong or missing something.
>
>> my local environment,  the result looks normal to me compare to
>> prior version.
>
> Can you share your number?
> We run netperf like this:
> netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1
> The full log of the run is attached for your reference.

Now I also changed to linux-net.git

commit 96b585267f552d4b6a28ea8bd75e5ed03deb6e71
[root@hp-dl388g8-08 ~]# uname -r
4.7.0.new
[root@hp-dl388g8-08 ~]# netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 --
-m 10K -H 127.0.0.1
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
127.0.0.1 () port 0 AF_INET
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     11814.56   4.65     4.65     0.775   0.774


commit f959fb442c35f4b61fea341401b8463dd0a1b959 (just before the buggie patch)
[root@localhost ~]# netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m
10K -H 127.0.0.1
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
127.0.0.1 () port 0 AF_INET
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     9454.90   5.22     5.22     1.086   1.085


I did tests on physical machine.
did you do it on guest ?

>
>>
>> Recently the sctp performance is not stable,  as during these patches,
>> netperf cannot get the result, but return ENOTCONN. which may
>> also affect the testing. anyway we've fixed the -ENOTCONN issue
>> already in the latest version.
>
> I tested commit 96b585267f55, which is Linus' git tree HEAD on 08/03, I
> guess the fix you mentioned should already be in there? But
> unfortunately, the throughput of netperf is still at low number(we did
> the test 5 times):
> $ cat */netperf.json
> {
>   "netperf.Throughput_Mbps": [
>     2470.6974999999998
>   ]
> }{
>   "netperf.Throughput_Mbps": [
>     2486.7675
>   ]
> }{
>   "netperf.Throughput_Mbps": [
>     2478.945
>   ]
> }{
>   "netperf.Throughput_Mbps": [
>     2429.465
>   ]
> }{
>   "netperf.Throughput_Mbps": [
>     2476.9150000000004
>   ]
>
> Considering what you have said that the patch shouldn't make a
> difference, the performance drop is really confusing. Any idea what
> could be the cause? Thanks.
Now I saw your tests result against the new kernel

Could you do the same test on the kernel before the problematic commit ?

[toc] | [prev] | [next] | [standalone]


#1457515 — Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression

FromAaron Lu <aaron.lu@intel.com>
Date2016-08-08 04:20 +0200
SubjectRe: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Message-ID<s3Ikx-3DH-9@gated-at.bofh.it>
In reply to#1457050
On Fri, Aug 05, 2016 at 07:53:38PM +0800, Xin Long wrote:
> >> It doesn't make much sense to me. the codes I added cannot be
> >> triggered without enable any pr policies. and I also did the tests in
> >
> > It seems these pr policies has to be turned on by user space, i.e.
> > netperf in this case?
> >
> > I checked netperf's source code, it doesn't seem set any option
> > related to SCTP PR POLICY but I'm new to network code so I could be
> > wrong or missing something.
> >
> >> my local environment,  the result looks normal to me compare to
> >> prior version.
> >
> > Can you share your number?
> > We run netperf like this:
> > netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1
> > The full log of the run is attached for your reference.
> 
> Now I also changed to linux-net.git
> 
> commit 96b585267f552d4b6a28ea8bd75e5ed03deb6e71
> [root@hp-dl388g8-08 ~]# uname -r
> 4.7.0.new
> [root@hp-dl388g8-08 ~]# netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 --
> -m 10K -H 127.0.0.1
> SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 127.0.0.1 () port 0 AF_INET
> Recv   Send    Send                          Utilization       Service Demand
> Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
> Size   Size    Size     Time     Throughput  local    remote   local   remote
> bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB
> 
> 212992 212992  10240    300.00     11814.56   4.65     4.65     0.775   0.774
> 
> 
> commit f959fb442c35f4b61fea341401b8463dd0a1b959 (just before the buggie patch)

I'm testing on Linus' master, can we all use that please?

> [root@localhost ~]# netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m
> 10K -H 127.0.0.1
> SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 127.0.0.1 () port 0 AF_INET
> Recv   Send    Send                          Utilization       Service Demand
> Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
> Size   Size    Size     Time     Throughput  local    remote   local   remote
> bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB
> 
> 212992 212992  10240    300.00     9454.90   5.22     5.22     1.086   1.085
> 
> 
> I did tests on physical machine.
> did you do it on guest ?

The test is done on a ivy-bridge desktop with 8G memory:
# cpudesc : Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz
# total memory : 8058152 kB

> 
> >
> >>
> >> Recently the sctp performance is not stable,  as during these patches,
> >> netperf cannot get the result, but return ENOTCONN. which may
> >> also affect the testing. anyway we've fixed the -ENOTCONN issue
> >> already in the latest version.
> >
> > I tested commit 96b585267f55, which is Linus' git tree HEAD on 08/03, I
> > guess the fix you mentioned should already be in there? But
> > unfortunately, the throughput of netperf is still at low number(we did
> > the test 5 times):
> > $ cat */netperf.json
> > {
> >   "netperf.Throughput_Mbps": [
> >     2470.6974999999998
> >   ]
> > }{
> >   "netperf.Throughput_Mbps": [
> >     2486.7675
> >   ]
> > }{
> >   "netperf.Throughput_Mbps": [
> >     2478.945
> >   ]
> > }{
> >   "netperf.Throughput_Mbps": [
> >     2429.465
> >   ]
> > }{
> >   "netperf.Throughput_Mbps": [
> >     2476.9150000000004
> >   ]
> >
> > Considering what you have said that the patch shouldn't make a
> > difference, the performance drop is really confusing. Any idea what
> > could be the cause? Thanks.
> Now I saw your tests result against the new kernel
> 
> Could you do the same test on the kernel before the problematic commit ?

Yes, the throughput of its parent commit is higer enough to trigger the
automatic bisect and then we send out the report.

Throughput of its parent commit 826d253d57b1("sctp: add SCTP_PR_ASSOC_STATUS
on sctp sockopt"):
Average:
"netperf.Throughput_Mbps": 3923.84375,

$ cat */netperf.json
{
  "netperf.Throughput_Mbps": [
    3869.25375
  ]
}{
  "netperf.Throughput_Mbps": [
    3952.58875
  ]
}{
  "netperf.Throughput_Mbps": [
    3936.89625
  ]
}{
  "netperf.Throughput_Mbps": [
    3936.63625
  ]
}

Feel free to let me know if you need any more information or you want me
to do more tests on other commits/machines, thanks.

Regards,
Aaron

[toc] | [prev] | [standalone]


Back to top | Article view | linux.kernel


csiph-web