Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1167523 > unrolled thread

Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy allocations

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy  allocations Xishi Qiu <qiuxishi@huawei.com> - 2015-06-18 03:30 +0200
    Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy  allocations Vlastimil Babka <vbabka@suse.cz> - 2015-06-18 08:00 +0200
      Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy  allocations "Luck, Tony" <tony.luck@intel.com> - 2015-06-18 22:40 +0200

#1167523 — Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy allocations

On 2015/6/16 17:46, Vlastimil Babka wrote:

> On 06/16/2015 10:17 AM, Xishi Qiu wrote:
>> On 2015/6/16 15:53, Vlastimil Babka wrote:
>>
>>> On 06/04/2015 02:54 PM, Xishi Qiu wrote:
>>>>
>>>> I think add a new migratetype is btter and easier than a new zone, so I use
>>>
>>> If the mirrored memory is in a single reasonably compact (no large holes) range
>>> (per NUMA node) and won't dynamically change its size, then zone might be a
>>> better option. For one thing, it will still allow distinguishing movable and
>>> unmovable allocations within the mirrored memory.
>>>
>>> We had enough fun with MIGRATE_CMA and all kinds of checks it added to allocator
>>> hot paths, and even CMA is now considering moving to a separate zone.
>>>
>>
>> Hi, how about the problem of this case:
>> e.g. node 0: 0-4G(dma and dma32)
>>      node 1: 4G-8G(normal), 8-12G(mirror), 12-16G(normal),
>> so more than one normal zone in a node? or normal zone just span the mirror zone?
> 
> Normal zone can span the mirror zone just fine. However, it will result in zone
> scanners such as compaction to skip over the mirror zone inefficiently. Hmm...
> 

Hi Vlastimil,

If there are many mirror regions in one node, then it will be many holes in the
normal zone, is this fine?

Thanks,
Xishi Qiu

> 
> .
> 



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1167598

On 18.6.2015 3:23, Xishi Qiu wrote:
> On 2015/6/16 17:46, Vlastimil Babka wrote:
> 
>> On 06/16/2015 10:17 AM, Xishi Qiu wrote:
>>> On 2015/6/16 15:53, Vlastimil Babka wrote:
>>>
>>>> On 06/04/2015 02:54 PM, Xishi Qiu wrote:
>>>>>
>>>>> I think add a new migratetype is btter and easier than a new zone, so I use
>>>>
>>>> If the mirrored memory is in a single reasonably compact (no large holes) range
>>>> (per NUMA node) and won't dynamically change its size, then zone might be a
>>>> better option. For one thing, it will still allow distinguishing movable and
>>>> unmovable allocations within the mirrored memory.
>>>>
>>>> We had enough fun with MIGRATE_CMA and all kinds of checks it added to allocator
>>>> hot paths, and even CMA is now considering moving to a separate zone.
>>>>
>>>
>>> Hi, how about the problem of this case:
>>> e.g. node 0: 0-4G(dma and dma32)
>>>      node 1: 4G-8G(normal), 8-12G(mirror), 12-16G(normal),
>>> so more than one normal zone in a node? or normal zone just span the mirror zone?
>>
>> Normal zone can span the mirror zone just fine. However, it will result in zone
>> scanners such as compaction to skip over the mirror zone inefficiently. Hmm...

On the other hand, it would skip just as inefficiently over MIGRATE_MIRROR
pageblocks within a Normal zone. Since migrating pages between MIGRATE_MIRROR
and other types pageblocks would violate what the allocations requested.

Having separate zone instead would allow compaction to run specifically on the
zone and defragment movable allocations there (i.e. userspace pages if/when
userspace requesting mirrored memory is supported).

>>
> 
> Hi Vlastimil,
> 
> If there are many mirror regions in one node, then it will be many holes in the
> normal zone, is this fine?

Yeah, it doesn't matter how many holes there are.

> Thanks,
> Xishi Qiu
> 
>>
>> .
>>
> 
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1168276

On Thu, Jun 18, 2015 at 11:55:42AM +0200, Vlastimil Babka wrote:
> >>>If there are many mirror regions in one node, then it will be many holes in the
> >>>normal zone, is this fine?
> >>
> >>Yeah, it doesn't matter how many holes there are.
> >
> >So mirror zone and normal zone will span each other, right?
> >
> >e.g. node 1: 4G-8G(normal), 8-12G(mirror), 12-16G(normal), 16-24G(mirror), 24-28G(normal) ...
> >normal: start=4G, size=28-4=24G,
> >mirror: start=8G, size=24-8=16G,
> 
> Yes, that works. It's somewhat unfortunate wrt performance that the hardware
> does it like this though.

With current Xeon h/w you can have one mirrored range per memory
controller ... and there are two memory controllers on a cpu socket,
so two mirrored ranges per node.  So a map might look like:

SKT0: MC0: 0-2G Mirrored (but we may want to ignore mirror here to keep it for ZONE_DMA)
SKT0: MC0: 2G-4G No memory ... I/O mapping area
SKT0: MC0: 4G-34G Not mirrored
SKT0: MC1: 34G-40G Mirrored
SKT0: MC1: 40G-66G Not mirrored

SKT1: MC0: 66G-70G Mirror
SKT1: MC0: 70G-98G Not Mirrored
SKT1: MC1: 98G-102G Mirror
SKT1: MC1: 102G-130G Not Mirrored

... and so on.

> >I think zone is defined according to the special address range, like 16M(DMA), 4G(DMA32),
> 
> Traditionally yes. But then there is ZONE_MOVABLE, this year's LSF/MM we
> discussed (and didn't outright deny) ZONE_CMA...
> I'm not saying others will favour the new zone approach though, it's just my
> opinion that it might be a better option than a new migratetype.

If we are going to have lots of zones ... then perhaps we will
need a fast way to look at a "struct page" and decide which zone
it belongs to.  Complicated math on the address deosn't sound ideal.
If the complex zone model is just for 64-bit, are there enough bits
available in page->flags (3 bits for 8 options ... which we are close
to filling now ... 4 bits for future breathing room).

> >and is it appropriate to add a new mirror zone with a volatile physical address?
> 
> By "volatile" you mean what, that the example above would change
> dynamically? That would be rather challenging...

If we hot-add another cpu together with on die memory controllers connected
to more memory ... then some of the new memory might be mirrored.  Current
h/w doesn't allow mirrored areas to grow/shrink (though if there are a lot
of errors we may break a mirror so a whole range could lose the mirror attribute).

-Tony
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web