Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1608176

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

Show all headers | View raw

John Hubbard <jhubbard@nvidia.com> writes:

> On 03/23/2017 09:52 PM, Huang, Ying wrote:
>> John Hubbard <jhubbard@nvidia.com> writes:
>>
>>> On 03/23/2017 07:41 PM, Huang, Ying wrote:
>>>> David Rientjes <rientjes@google.com> writes:
>>>>
>>>>> On Mon, 20 Mar 2017, Huang, Ying wrote:
>>>>>
>>>>>> From: Huang Ying <ying.huang@intel.com>
>>>>>>
>>>>>> Now vzalloc() is used in swap code to allocate various data
>>>>>> structures, such as swap cache, swap slots cache, cluster info, etc.
>>>>>> Because the size may be too large on some system, so that normal
>>>>>> kzalloc() may fail.  But using kzalloc() has some advantages, for
>>>>>> example, less memory fragmentation, less TLB pressure, etc.  So change
>>>>>> the data structure allocation in swap code to use kvzalloc() which
>>>>>> will try kzalloc() firstly, and fallback to vzalloc() if kzalloc()
>>>>>> failed.
>>>>>>
>>>>>
>>>>> As questioned in -v1 of this patch, what is the benefit of directly
>>>>> compacting and reclaiming memory for high-order pages by first preferring
>>>>> kmalloc() if this does not require contiguous memory?
>>>>
>>>> The memory allocation here is only for swap on time, not for swap out/in
>>>> time.  The performance of swap on is not considered critical.  But if
>>>> the kmalloc() is used instead of the vmalloc(), the swap out/in
>>>> performance could be improved (marginally).  More importantly, the
>>>> interference for the other activity on the system could be reduced, For
>>>> example, less memory fragmentation, less TLB usage of swap subsystem,
>>>> etc.
>>>
>>> Hi Ying,
>>>
>>> I'm a little surprised to see vmalloc calls replaced with
>>> kmalloc-then-vmalloc calls, because that actually makes fragmentation
>>> worse (contrary to the above claim). That's because you will consume
>>> contiguous memory (even though you don't need it to be contiguous),
>>> whereas before, you would have been able to get by with page-at-a-time
>>> for vmalloc.
>>>
>>> So, things like THP will find fewer contiguous chunks, as a result of patches such as this.
>>
>> Hi, John,
>>
>> I don't think so.  The pages allocated by vmalloc() cannot be moved
>> during de-fragment.  For example, if 512 dis-continuous physical pages
>> are allocated via vmalloc(), at worst, one page will be allocate from
>> one distinct 2MB continous physical pages.  This makes 512 * 2MB = 1GB
>> memory cannot be used for THP allocation.  Because these pages cannot be
>> defragmented until vfree().
>
> kmalloc requires a resource that vmalloc does not: contiguous
> pages. Therefore, given the same mix of pages (some groups of
> contiguous pages, and a scattering of isolated single-page, or
> too-small-to-satisfy-entire-alloc groups of pages, and the same
> underlying page allocator, kmalloc *must* consume the more valuable
> contiguous pages. However, vmalloc *may* consume those same pages.
>
> So, if you run kmalloc a bunch of times, with higher-order requests,
> you *will* run out of contiguous pages (until more are freed up). If
> you run vmalloc with the same initial conditions and the same
> requests, you may not necessary use up those contiguous pages.
>
> It's true that there are benefits to doing a kmalloc-then-vmalloc, of
> course: if the pages are available, it's faster and uses less
> resources. Yes. I just don't think "less fragmentation" should be
> listed as a benefit, because you can definitely cause *more*
> fragmentation if you use up contiguous blocks unnecessarily.

Yes, I agree that for some cases, kmalloc() will use more contiguous
blocks, for example, non-movable pages are scattered all over the
memory.  But I still think in common cases, if defragement is enabled,
and non-movable pages allocation is restricted to some memory area if
possible, kmalloc() is better than vmalloc() as for fragmentation.

Best Regards,
Huang, Ying

> --
> thanks,
> john h

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure "Huang, Ying" <ying.huang@intel.com> - 2017-03-20 10:00 +0100
  [PATCH -v2 2/2] mm, swap: Sort swap entries before free "Huang, Ying" <ying.huang@intel.com> - 2017-03-20 10:00 +0100
  Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap  data structure David Rientjes <rientjes@google.com> - 2017-03-20 22:40 +0100
    Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure "Huang\, Ying" <ying.huang@intel.com> - 2017-03-24 03:50 +0100
      Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data  structure John Hubbard <jhubbard@nvidia.com> - 2017-03-24 05:30 +0100
        Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure "Huang\, Ying" <ying.huang@intel.com> - 2017-03-24 06:00 +0100
          Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data  structure John Hubbard <jhubbard@nvidia.com> - 2017-03-24 07:50 +0100
            Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure "Huang\, Ying" <ying.huang@intel.com> - 2017-03-24 08:20 +0100
              Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data  structure John Hubbard <jhubbard@nvidia.com> - 2017-03-24 08:40 +0100
                Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data  structure Dave Hansen <dave.hansen@linux.intel.com> - 2017-03-24 15:00 +0100
                Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap  data structure Tim Chen <tim.c.chen@linux.intel.com> - 2017-03-24 18:00 +0100
                Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data  structure John Hubbard <jhubbard@nvidia.com> - 2017-03-24 19:20 +0100

csiph-web