RE: [RFC PATCH 00/30] Kernel NET policy

From	"Liang, Kan" <kan.liang@intel.com>
Newsgroups	linux.kernel
Subject	RE: [RFC PATCH 00/30] Kernel NET policy
Date	2016-07-18 20:00 +0200
Message-ID	<rWkZI-Tt-15@gated-at.bofh.it> (permalink)
References	<rWhIu-7l8-7@gated-at.bofh.it> <rWiuR-7Ug-13@gated-at.bofh.it> <rWj7A-89r-5@gated-at.bofh.it> <rWjKh-bB-15@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

> On Mon, Jul 18, 2016 at 5:51 PM, Liang, Kan <kan.liang@intel.com> wrote:
> >
> >
> >> >
> >> > It is a big challenge to get good network performance. First, the
> >> > network performance is not good with default system settings.
> >> > Second, it is too difficult to do automatic tuning for all possible
> >> > workloads, since workloads have different requirements. Some
> >> > workloads may want
> >> high throughput.
> >>
> >> Seems you did lots of tests to find optimal settings for a given base policy.
> >>
> > Yes. Current test only base on Intel i40e driver. The optimal settings
> > should vary for other devices. But adding settings for new device is not
> hard.
> >
> The optimal settings are very dependent on system architecture (NUMA
> config, #cpus, memory, etc.) and sometimes kernel version as well. A
> database that provides best configurations across different devices,
> architectures, and kernel version might be interesting; but beware that that is
> a whole bunch of work to maintain, Either way policy like this really should
> be handled in userspace.

The expression "optimal" I used here is not accurate. Sorry for that.
The NET policy tries to get good (near optimal) performance by very
simple configuration.
I agree that there are lots of dependencies for the optimal settings.
But most of the settings should be very similar. The near optimal performance
by applying those common settings are good enough for most users. 
We don't need to maintain a database for configurations across
devices/architectures/kernel version...

Thanks,
Kan

Thread

[RFC PATCH 00/30] Kernel NET policy kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 10/30] net/netpolicy: introduce netpolicy object kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 20/30] net/netpolicy: introduce per socket netpolicy kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 28/30] net/netpolicy: optimize for queue pair kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 29/30] net/netpolicy: limit the total record number kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 30/30] Documentation/networking: Document net policy kan.liang@intel.com - 2016-07-18 16:30 +0200
    Re: [RFC PATCH 30/30] Documentation/networking: Document net policy Randy Dunlap <rdunlap@infradead.org> - 2016-07-18 19:10 +0200
  [RFC PATCH 24/30] net/netpolicy: set rx queues according to policy kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 23/30] i40e/ethtool: support RX_CLS_LOC_ANY kan.liang@intel.com - 2016-07-18 16:30 +0200
    Re: [RFC PATCH 23/30] i40e/ethtool: support RX_CLS_LOC_ANY Alexander Duyck <alexander.duyck@gmail.com> - 2016-07-18 18:30 +0200
  [RFC PATCH 14/30] net/netpolicy: add MIX policy kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 19/30] net/netpolicy: implement netpolicy register kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 27/30] net/netpolicy: fast path for finding the queues kan.liang@intel.com - 2016-07-18 16:30 +0200
  [RFC PATCH 09/30] net/netpolicy: enable and disable net policy kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 18/30] net/netpolicy: handle channel changes kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 08/30] net/netpolicy: set and remove irq affinity kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 16/30] net/netpolicy: net device hotplug kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 04/30] net/netpolicy: get driver information kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 15/30] i40e/netpolicy: add MIX policy support kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 05/30] i40e/netpolicy: implement ndo_get_irq_info kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 11/30] net/netpolicy: set net policy by policy name kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 01/30] net: introduce NET policy kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 03/30] i40e/netpolicy: Implement ndo_netpolicy_init kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 17/30] net/netpolicy: support CPU hotplug kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 02/30] net/netpolicy: init NET policy kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 06/30] net/netpolicy: get CPU information kan.liang@intel.com - 2016-07-18 16:40 +0200
  [RFC PATCH 07/30] net/netpolicy: create CPU and queue mapping kan.liang@intel.com - 2016-07-18 16:40 +0200
  Re: [RFC PATCH 00/30] Kernel NET policy Florian Westphal <fw@strlen.de> - 2016-07-18 17:20 +0200
    Re: [RFC PATCH 00/30] Kernel NET policy Andi Kleen <andi@firstfloor.org> - 2016-07-18 17:50 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Cong Wang <xiyou.wangcong@gmail.com> - 2016-07-18 20:00 +0200
        Re: [RFC PATCH 00/30] Kernel NET policy Cong Wang <xiyou.wangcong@gmail.com> - 2016-07-18 22:20 +0200
          RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 22:30 +0200
        RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 22:20 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Hannes Frederic Sowa <hannes@stressinduktion.org> - 2016-07-18 21:10 +0200
        Re: [RFC PATCH 00/30] Kernel NET policy Andi Kleen <andi@firstfloor.org> - 2016-07-18 21:50 +0200
          Re: [RFC PATCH 00/30] Kernel NET policy Hannes Frederic Sowa <hannes@stressinduktion.org> - 2016-07-19 00:00 +0200
            RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-19 03:50 +0200
              Re: [RFC PATCH 00/30] Kernel NET policy David Miller <davem@davemloft.net> - 2016-07-19 07:10 +0200
                RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-19 15:50 +0200
    RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 18:00 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Florian Westphal <fw@strlen.de> - 2016-07-18 18:20 +0200
        RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 19:50 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Tom Herbert <tom@herbertland.com> - 2016-07-18 18:40 +0200
        RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 20:00 +0200
  Re: [RFC PATCH 00/30] Kernel NET policy Daniel Borkmann <daniel@iogearbox.net> - 2016-07-18 18:30 +0200
    RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 20:40 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Daniel Borkmann <daniel@iogearbox.net> - 2016-07-18 23:00 +0200
  Re: [RFC PATCH 00/30] Kernel NET policy Alexander Duyck <alexander.duyck@gmail.com> - 2016-07-18 19:10 +0200
    RE: [RFC PATCH 00/30] Kernel NET policy "Liang, Kan" <kan.liang@intel.com> - 2016-07-18 21:50 +0200
      Re: [RFC PATCH 00/30] Kernel NET policy Andi Kleen <andi@firstfloor.org> - 2016-07-18 22:00 +0200

csiph-web