Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1533319

Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3

Show all headers | View raw

On Wed, Nov 30, 2016 at 01:40:34PM +0100, Jesper Dangaard Brouer wrote:
> 
> On Sun, 27 Nov 2016 13:19:54 +0000 Mel Gorman <mgorman@techsingularity.net> wrote:
> 
> [...]
> > SLUB has been the default small kernel object allocator for quite some time
> > but it is not universally used due to performance concerns and a reliance
> > on high-order pages. The high-order concerns has two major components --
> > high-order pages are not always available and high-order page allocations
> > potentially contend on the zone->lock. This patch addresses some concerns
> > about the zone lock contention by extending the per-cpu page allocator to
> > cache high-order pages. The patch makes the following modifications
> > 
> > o New per-cpu lists are added to cache the high-order pages. This increases
> >   the cache footprint of the per-cpu allocator and overall usage but for
> >   some workloads, this will be offset by reduced contention on zone->lock.
> 
> This will also help performance of NIC driver that allocator
> higher-order pages for their RX-ring queue (and chop it up for MTU).
> I do like this patch, even-though I'm working on moving drivers away
> from allocation these high-order pages.
> 
> Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
> 

Thanks.

> [...]
> > This is the result from netperf running UDP_STREAM on localhost. It was
> > selected on the basis that it is slab-intensive and has been the subject
> > of previous SLAB vs SLUB comparisons with the caveat that this is not
> > testing between two physical hosts.
> 
> I do like you are using a networking test to benchmark this. Looking at
> the results, my initial response is that the improvements are basically
> too good to be true.
> 

FWIW, LKP independently measured the boost to be 23% so it's expected
there will be different results depending on exact configuration and CPU.

> Can you share how you tested this with netperf and the specific netperf
> parameters? 

The mmtests config file used is
configs/config-global-dhp__network-netperf-unbound so all details can be
extrapolated or reproduced from that.

> e.g.
>  How do you configure the send/recv sizes?

Static range of sizes specified in the config file.

>  Have you pinned netperf and netserver on different CPUs?
> 

No. While it's possible to do a pinned test which helps stability, it
also tends to be less reflective of what happens in a variety of
workloads so I took the "harder" option.

> For localhost testing, when netperf and netserver run on the same CPU,
> you observer half the performance, very intuitively.  When pinning
> netperf and netserver (via e.g. option -T 1,2) you observe the most
> stable results.  When allowing netperf and netserver to migrate between
> CPUs (default setting), the real fun starts and unstable results,
> because now the CPU scheduler is also being tested, and my experience
> is also more "fun" memory situations occurs, as I guess we are hopping
> between more per CPU alloc caches (also affecting the SLUB per CPU usage
> pattern).
> 

Yes which is another reason why I used an unbound configuration. I didn't
want to get an artificial boost from pinned server/client using the same
per-cpu caches. As a side-effect, it may mean that machines with fewer
CPUs get a greater boost as there are fewer per-cpu caches being used.

> > 2-socket modern machine
> >                                 4.9.0-rc5             4.9.0-rc5
> >                                   vanilla             hopcpu-v3
> 
> The kernel from 4.9.0-rc5-vanilla to 4.9.0-rc5-hopcpu-v3 only contains
> this single change right?

Yes.

-- 
Mel Gorman
SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-27 14:30 +0100
  Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Vlastimil Babka <vbabka@suse.cz> - 2016-11-28 12:10 +0100
    Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-28 12:50 +0100
    Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-30 10:00 +0100
  Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Christoph Lameter <cl@linux.com> - 2016-11-28 16:50 +0100
    Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-28 17:30 +0100
      Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Christoph Lameter <cl@linux.com> - 2016-11-28 17:40 +0100
        Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-28 19:50 +0100
          Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Christoph Lameter <cl@linux.com> - 2016-11-28 20:00 +0100
            Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Vlastimil Babka <vbabka@suse.cz> - 2016-11-28 22:10 +0100
  Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Johannes Weiner <hannes@cmpxchg.org> - 2016-11-28 21:00 +0100
  Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Jesper Dangaard Brouer <brouer@redhat.com> - 2016-11-30 13:50 +0100
    Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-30 15:10 +0100
      Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Jesper Dangaard Brouer <brouer@redhat.com> - 2016-11-30 16:10 +0100
        Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-30 17:40 +0100
          Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Jesper Dangaard Brouer <brouer@redhat.com> - 2016-12-01 18:40 +0100
            Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Paolo Abeni <pabeni@redhat.com> - 2016-12-01 23:20 +0100
              Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Jesper Dangaard Brouer <brouer@redhat.com> - 2016-12-02 16:40 +0100
                Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Paolo Abeni <pabeni@redhat.com> - 2016-12-02 16:50 +0100
  Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Michal Hocko <mhocko@kernel.org> - 2016-11-30 14:10 +0100
    Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Mel Gorman <mgorman@techsingularity.net> - 2016-11-30 15:20 +0100
      Re: [PATCH] mm: page_alloc: High-order per-cpu page allocator v3 Michal Hocko <mhocko@kernel.org> - 2016-11-30 16:10 +0100

csiph-web