Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1694762

Re: pcpu allocator on large NUMA machines

Show all headers | View raw

On Mon 24-07-17 09:57:14, Tejun Heo wrote:
> Hello,

Hi,
and thanks for ths swift answer

> On Mon, Jul 24, 2017 at 03:42:40PM +0200, Michal Hocko wrote:
[...]
> > My understanding of the pcpu allocator is basically close to zero but it
> > seems weird to me that we would need many TB of vmalloc address space
> > just to allocate vmalloc areas that are in range of hundreds of MB. So I
> > am wondering whether this is an expected behavior of the allocator or
> > there is a problem somwehere else.
> 
> It's not actually using the entire region but the area allocations try
> to follow the same topology as kernel linear address layouts.  ie. if
> kernel address for different NUMA nodes are apart by certain amount,
> the percpu allocator tries to replicate that for dynamic allocations
> which allows leaving the static and first dynamic area in the kernel
> linear address which helps reducing TLB pressure.
> 
> This optimization can be turned off when vmalloc area isn't spacious
> enough by using pcpu_page_first_chunk() instead of
> pcpu_embed_first_chunk() while initializing percpu allocator.

Thanks for the clarification, this is really helpful!

> Can you
> see whether replacing that in arch/powerpc/kernel/setup_64.c fixes the
> issue?  If so, all it needs to do is figuring out what conditions we
> need to check to opt out of embedding the first chunk.  Note that x86
> 32bit does about the same thing.

Hmm, I will need some help from PPC guys here. I cannot find something
ready to implement pcpup_populate_pte and I am not familiar with ppc
memory model to implement one myself.
-- 
Michal Hocko
SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

pcpu allocator on large NUMA machines Michal Hocko <mhocko@kernel.org> - 2017-07-24 15:50 +0200
  Re: pcpu allocator on large NUMA machines Tejun Heo <tj@kernel.org> - 2017-07-24 16:00 +0200
    Re: pcpu allocator on large NUMA machines Michal Hocko <mhocko@kernel.org> - 2017-07-24 16:30 +0200
      Re: pcpu allocator on large NUMA machines Michael Ellerman <mpe@ellerman.id.au> - 2017-07-25 03:30 +0200
        Re: pcpu allocator on large NUMA machines Tejun Heo <tj@kernel.org> - 2017-07-25 18:50 +0200

csiph-web