Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1470038 > unrolled thread

Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab: Improve performance of gathering slabinfo) stats

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3]  mm/slab: Improve performance of gathering slabinfo) stats Mel Gorman <mgorman@suse.de> - 2016-08-25 12:10 +0200
    Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab:  Improve performance of gathering slabinfo) stats Christoph Lameter <cl@linux.com> - 2016-08-25 22:00 +0200
      Re: what is the purpose of SLAB and SLUB Andi Kleen <andi@firstfloor.org> - 2016-08-26 22:50 +0200
        Re: what is the purpose of SLAB and SLUB Michal Hocko <mhocko@kernel.org> - 2016-08-29 15:50 +0200
          Re: what is the purpose of SLAB and SLUB Christoph Lameter <cl@linux.com> - 2016-08-29 16:50 +0200
      Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3]  mm/slab: Improve performance of gathering slabinfo) stats Mel Gorman <mgorman@suse.de> - 2016-08-30 11:50 +0200
        Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab:  Improve performance of gathering slabinfo) stats Christoph Lameter <cl@linux.com> - 2016-08-30 21:40 +0200

#1470038 — Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab: Improve performance of gathering slabinfo) stats

On Wed, Aug 24, 2016 at 11:01:43PM -0500, Christoph Lameter wrote:
> On Wed, 24 Aug 2016, Mel Gorman wrote:
> > If/when I get back to the page allocator, the priority would be a bulk
> > API for faster allocs of batches of order-0 pages instead of allocating
> > a large page and splitting.
> >
> 
> OMG. Do we really want to continue this? There are billions of Linux
> devices out there that require a reboot at least once a week. This is now
> standard with certain Android phones. In our company we reboot all
> machines every week because fragmentation degrades performance
> significantly. We need to finally face up to it and deal with the issue
> instead of continuing to produce more half ass-ed solutions.
> 

Flipping the lid aside, there will always be a need for fast management
of 4K pages. The primary use case is networking that sometimes uses
high-order pages to avoid allocator overhead and amortise DMA setup.
Userspace-mapped pages will always be 4K although fault-around may benefit
from bulk allocating the pages. That is relatively low hanging fruit that
would take a few weeks given a free schedule.

Dirty tracking of pages on a 4K boundary will always be required to avoid IO
multiplier effects that cannot be side-stepped by increasing the fundamental
unit of allocation.

Batching of tree_lock during reclaim for large files and swapping is also
relatively low hanging fruit that also is doable in a week or two.

A high-order per-cpu cache for SLUB to reduce zone->lock contention is
also relatively low hanging fruit with the caveat it makes per_cpu_pages
larger than a cache line.

If you want to rework the VM to use a larger fundamental unit, track
sub-units where required and deal with the internal fragmentation issues
then by all means go ahead and deal with it.
 
-- 
Mel Gorman
SUSE Labs

[toc] | [next] | [standalone]

#1470361 — Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab: Improve performance of gathering slabinfo) stats

On Thu, 25 Aug 2016, Mel Gorman wrote:

> Flipping the lid aside, there will always be a need for fast management
> of 4K pages. The primary use case is networking that sometimes uses
> high-order pages to avoid allocator overhead and amortise DMA setup.
> Userspace-mapped pages will always be 4K although fault-around may benefit
> from bulk allocating the pages. That is relatively low hanging fruit that
> would take a few weeks given a free schedule.

Userspace mapped pages can be hugepages as well as giant pages and that
has been there for a long time. Intermediate sizes would be useful too in
order to avoid having to keep lists of 4k pages around and continually
scan them.

> Dirty tracking of pages on a 4K boundary will always be required to avoid IO
> multiplier effects that cannot be side-stepped by increasing the fundamental
> unit of allocation.

Huge pages cannot be dirtied? This is an issue of hardware support. On
x867 you only have one size. I am pretty such that even intel would
support other sizes if needed. The case has been repeatedly made that 64k
pages f.e. would be useful to have on x86.


> Batching of tree_lock during reclaim for large files and swapping is also
> relatively low hanging fruit that also is doable in a week or two.

Ok these are good incremental improvement but they do not address the main
issue going forward.

> A high-order per-cpu cache for SLUB to reduce zone->lock contention is
> also relatively low hanging fruit with the caveat it makes per_cpu_pages
> larger than a cache line.

Would be great to have.

> If you want to rework the VM to use a larger fundamental unit, track
> sub-units where required and deal with the internal fragmentation issues
> then by all means go ahead and deal with it.

Hmmm... The time problem is always there. Tried various approaches over
the last decade. Could be a massive project. We really would need a
larger group of developers to effectively do this.

[toc] | [prev] | [next] | [standalone]

#1470988 — Re: what is the purpose of SLAB and SLUB

Christoph Lameter <cl@linux.com> writes:
>
>> If you want to rework the VM to use a larger fundamental unit, track
>> sub-units where required and deal with the internal fragmentation issues
>> then by all means go ahead and deal with it.
>
> Hmmm... The time problem is always there. Tried various approaches over
> the last decade. Could be a massive project. We really would need a
> larger group of developers to effectively do this.

I'm surprised that compactions is not able to fix the fragmentation.
Is the problem that there are too many non movable objects around?

-Andi

[toc] | [prev] | [next] | [standalone]

#1471855 — Re: what is the purpose of SLAB and SLUB

On Fri 26-08-16 13:47:47, Andi Kleen wrote:
> Christoph Lameter <cl@linux.com> writes:
> >
> >> If you want to rework the VM to use a larger fundamental unit, track
> >> sub-units where required and deal with the internal fragmentation issues
> >> then by all means go ahead and deal with it.
> >
> > Hmmm... The time problem is always there. Tried various approaches over
> > the last decade. Could be a massive project. We really would need a
> > larger group of developers to effectively do this.
> 
> I'm surprised that compactions is not able to fix the fragmentation.
> Is the problem that there are too many non movable objects around?

Compaction can certainly help and the more we are proactive in that
direction the better. Vlastimil has already done a first step in that
direction and we a have a dedicated kcompactd kernel thread for that
purpose. But I guess what Mel had in mind is the latency of higher
order pages which is inherently higher with the current page allocator
no matter how well the compaction works. There are other changes, mostly
for the fast path, needed to make higher order pages less of a second
citizen.
-- 
Michal Hocko
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1471889 — Re: what is the purpose of SLAB and SLUB

On Mon, 29 Aug 2016, Michal Hocko wrote:

> Compaction can certainly help and the more we are proactive in that
> direction the better. Vlastimil has already done a first step in that
> direction and we a have a dedicated kcompactd kernel thread for that
> purpose. But I guess what Mel had in mind is the latency of higher
> order pages which is inherently higher with the current page allocator
> no matter how well the compaction works. There are other changes, mostly
> for the fast path, needed to make higher order pages less of a second
> citizen.

Compaction needs to be able to move many more types of kernel objects out
of the way. I think if the callbacks that were merged for the migration of
CMA pages are made usable for slab allocations then we may make some
progress there. This would require the creator of a slab cache to specify
functions that allow the migration of an object. Would require additional
subsystem specific code. But doing that for inodes and dentries could be
very benficial for compaction.

[toc] | [prev] | [next] | [standalone]

#1472370

On Thu, Aug 25, 2016 at 02:55:43PM -0500, Christoph Lameter wrote:
> On Thu, 25 Aug 2016, Mel Gorman wrote:
> 
> > Flipping the lid aside, there will always be a need for fast management
> > of 4K pages. The primary use case is networking that sometimes uses
> > high-order pages to avoid allocator overhead and amortise DMA setup.
> > Userspace-mapped pages will always be 4K although fault-around may benefit
> > from bulk allocating the pages. That is relatively low hanging fruit that
> > would take a few weeks given a free schedule.
> 
> Userspace mapped pages can be hugepages as well as giant pages and that
> has been there for a long time. Intermediate sizes would be useful too in
> order to avoid having to keep lists of 4k pages around and continually
> scan them.
> 

Userspace pages cannot always be mapped as huge or giant. mprotect on a
4K boundary is an obvious example.

> > Dirty tracking of pages on a 4K boundary will always be required to avoid IO
> > multiplier effects that cannot be side-stepped by increasing the fundamental
> > unit of allocation.
> 
> Huge pages cannot be dirtied?

I didn't say that, I said they are required to avoid IO multiplier
effects. If a file is mapped as 2M or 1G then even a 1 byte write requires
2M or 1G of IO to writeback.

> This is an issue of hardware support. On
> x867 you only have one size. I am pretty such that even intel would
> support other sizes if needed. The case has been repeatedly made that 64k
> pages f.e. would be useful to have on x86.
> 

64K pages are not a universal win even on the arches that do support them.

-- 
Mel Gorman
SUSE Labs

[toc] | [prev] | [next] | [standalone]

#1472815 — Re: what is the purpose of SLAB and SLUB (was: Re: [PATCH v3] mm/slab: Improve performance of gathering slabinfo) stats

On Tue, 30 Aug 2016, Mel Gorman wrote:

> > Userspace mapped pages can be hugepages as well as giant pages and that
> > has been there for a long time. Intermediate sizes would be useful too in
> > order to avoid having to keep lists of 4k pages around and continually
> > scan them.
> >
>
> Userspace pages cannot always be mapped as huge or giant. mprotect on a
> 4K boundary is an obvious example.

Well if the pages are bigger then the boundaries will also be different.
The problem is that we are trying to keep the 4k illustion alive. This
causes churn in various subsystems. Implementation of a file cache
with arbitrary page order is rather straightforward. See
https://lkml.org/lkml/2007/4/19/261

There we run again against the problem of defragmentation. Avoiding decent
garbage collection in the kernel causes no end of additional trouble. I
think we need to face the issue and solve it. Then a lot of other
workaround and complex things are no longer necesary.

> > > Dirty tracking of pages on a 4K boundary will always be required to avoid IO
> > > multiplier effects that cannot be side-stepped by increasing the fundamental
> > > unit of allocation.
> >
> > Huge pages cannot be dirtied?
>
> I didn't say that, I said they are required to avoid IO multiplier
> effects. If a file is mapped as 2M or 1G then even a 1 byte write requires
> 2M or 1G of IO to writeback.

There are numerous use cases that I know of where this would be
acceptable. Some tuning would be required of course like a mininum period
until writeback occurs.

> > This is an issue of hardware support. On
> > x867 you only have one size. I am pretty such that even intel would
> > support other sizes if needed. The case has been repeatedly made that 64k
> > pages f.e. would be useful to have on x86.
> >
>
> 64K pages are not a universal win even on the arches that do support them.

There are always corner cases that regress with any kernel "enhancement".
64k page size was a signicant improvement for many of the loads when I
worked at SGI on Altix.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web