Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1474652

Re: [PATCH v2 00/11] staging: octeon: multi rx group (queue) support

Show all headers | View raw

Hi,

On Wed, Aug 31, 2016 at 07:09:13PM -0700, Ed Swierk wrote:
> On 8/31/16 13:57, Aaro Koskinen wrote:
> > This series implements multiple RX group support that should improve
> > the networking performance on multi-core OCTEONs. Basically we register
> > IRQ and NAPI for each group, and ask the HW to select the group for
> > the incoming packets based on hash.
> > 
> > Tested on EdgeRouter Lite with a simple forwarding test using two flows
> > and 16 RX groups distributed between two cores - the routing throughput
> > is roughly doubled.
> > 
> > Also tested with EBH5600 (8 cores) and EBB6800 (16 cores) by sending
> > and receiving traffic in both directions using SGMII interfaces.
> 
> With this series on 4.4.19, rx works with receive_group_order > 0.

Good.

> Setting receive_group_order=4, I do see 16 Ethernet interrupts. I tried
> fiddling with various smp_affinity values (e.g. setting them all to
> ffffffff, or assigning a different one to each interrupt, or giving a
> few to some and a few to others), as well as different values for
> rps_cpus. 10-thread parallel iperf performance varies between 0.5 and 1.5
> Gbit/sec total depending on the particular settings.
> 
> With the SDK kernel I get over 8 Gbit/sec. It seems to be achieving that
> using just one interrupt (not even a separate one for tx, as far as I can
> tell) pegged to CPU 0 (the default smp_affinity). I must be missing some
> other major configuration tweak, perhaps specific to 10G.
> 
> Can you run a test on the EBB6800 with the interfaces in 10G mode?

Yes, I attached two EBB6800s with XAUI and ran iperf -P 10.

With single group it gives 2.9 Gbit/s, and with 16 groups (on 16 cores)
4.3 Gbit/s. In 16 group case none of the CPUs are even close to 100%,
so the bottleneck is somewhere else. I guess implementing the proper
SSO init should increase the throughput.

A.

Back to linux.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

[PATCH v2 00/11] staging: octeon: multi rx group (queue) support Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 06/11] staging: octeon: move irq into rx group specific data Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 04/11] staging: octeon: move common poll code into a separate function Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 02/11] staging: octeon: use passed interrupt number in the handler Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 03/11] staging: octeon: pass the NAPI instance reference to irq handler Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 01/11] staging: octeon: disable rx interrupts in oct_rx_shutdown Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 11/11] staging: octeon: prevent poll during rx init Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 09/11] staging: octeon: enable taking multiple rx groups into use Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 07/11] staging: octeon: move group number into rx group data Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 10/11] staging: octeon: set up pknd for all interfaces Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  [PATCH v2 08/11] staging: octeon: support enabling multiple rx groups Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-08-31 23:10 +0200
  Re: [PATCH v2 00/11] staging: octeon: multi rx group (queue) support Ed Swierk <eswierk@skyportsystems.com> - 2016-09-01 04:10 +0200
    Re: [PATCH v2 00/11] staging: octeon: multi rx group (queue) support Aaro Koskinen <aaro.koskinen@iki.fi> - 2016-09-01 23:30 +0200

csiph-web