Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1253105 > unrolled thread

RE: [PATCH] mm: Introduce kernelcore=reliable option

Started by"Luck, Tony" <tony.luck@intel.com>
First post2015-10-21 20:20 +0200
Last post2015-10-30 20:50 +0100
Articles 7 — 3 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: [PATCH] mm: Introduce kernelcore=reliable option "Luck, Tony" <tony.luck@intel.com> - 2015-10-21 20:20 +0200
    Re: [PATCH] mm: Introduce kernelcore=reliable option Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-10-22 12:10 +0200
      RE: [PATCH] mm: Introduce kernelcore=reliable option "Luck, Tony" <tony.luck@intel.com> - 2015-10-23 01:30 +0200
        RE: [PATCH] mm: Introduce kernelcore=reliable option "Izumi, Taku" <izumi.taku@jp.fujitsu.com> - 2015-10-23 03:10 +0200
          Re: [PATCH] mm: Introduce kernelcore=reliable option "Luck, Tony" <tony.luck@intel.com> - 2015-10-23 03:50 +0200
            Re: [PATCH] mm: Introduce kernelcore=reliable option Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> - 2015-10-30 07:30 +0100
              RE: [PATCH] mm: Introduce kernelcore=reliable option "Luck, Tony" <tony.luck@intel.com> - 2015-10-30 20:50 +0100

#1253105 — RE: [PATCH] mm: Introduce kernelcore=reliable option

From"Luck, Tony" <tony.luck@intel.com>
Date2015-10-21 20:20 +0200
SubjectRE: [PATCH] mm: Introduce kernelcore=reliable option
Message-ID<qm69s-2wo-27@gated-at.bofh.it>
+	if (reliable_kernelcore) {
+		for_each_memblock(memory, r) {
+			if (memblock_is_mirror(r))
+				continue;

Should we have a safety check here that there is some mirrored memory?  If you give
the kernelcore=reliable option on a machine which doesn't have any mirror configured,
then we'll mark all memory as removable.  What happens then?  Do kernel allocations
fail?  Or do they fall back to using removable memory?

Is there a /proc or /sys file that shows the current counts for the removable zone?  I just
tried this patch with a high percentage of memory marked as mirror ... but I'd like to see
how much is actually being used to tune things a bit.

-Tony
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1253686

FromKamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Date2015-10-22 12:10 +0200
Message-ID<qmkYO-7xa-17@gated-at.bofh.it>
In reply to#1253105
On 2015/10/22 3:17, Luck, Tony wrote:
> +	if (reliable_kernelcore) {
> +		for_each_memblock(memory, r) {
> +			if (memblock_is_mirror(r))
> +				continue;
>
> Should we have a safety check here that there is some mirrored memory?  If you give
> the kernelcore=reliable option on a machine which doesn't have any mirror configured,
> then we'll mark all memory as removable.

You're right.

> What happens then?  Do kernel allocations fail?  Or do they fall back to using removable memory?

Maybe the kernel cannot boot because NORMAL zone is empty.

> Is there a /proc or /sys file that shows the current counts for the removable zone?  I just
> tried this patch with a high percentage of memory marked as mirror ... but I'd like to see
> how much is actually being used to tune things a bit.
>

I think /proc/zoneinfo can show detailed numbers per zone. Do we need some for meminfo ?

Thanks,
-Kame


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1254224

From"Luck, Tony" <tony.luck@intel.com>
Date2015-10-23 01:30 +0200
Message-ID<qmxsZ-x2-1@gated-at.bofh.it>
In reply to#1253686

[Multipart message — attachments visible in raw view] — view raw

> I think /proc/zoneinfo can show detailed numbers per zone. Do we need some for meminfo ?

I wrote a little script (attached) to summarize /proc/zoneinfo ... on my system it says

$ zoneinfo
Node          Normal         Movable             DMA           DMA32 
   0            0.00       103020.07            8.94         1554.46 
   1         9284.54        89870.43                                 
   2         9626.33        94050.09                                 
   3         9602.82        93650.04    

Not sure why I have zero Normal memory free on node0.  The sum of all those
free counts is 410667.72 MB ... which is close enough to the boot time message
showing the amount of mirror/total memory:

[    0.000000] efi: Memory: 80979/420096M mirrored memory

but a fair amount of the 80G of mirrored memory seems to have been miscounted
as Movable instead of Normal. Perhaps this is because I have two blocks of mirrored
memory on each node and the movable zone code doesn't expect that?

-Tony

[toc] | [prev] | [next] | [standalone]

#1254266

From"Izumi, Taku" <izumi.taku@jp.fujitsu.com>
Date2015-10-23 03:10 +0200
Message-ID<qmz1M-2PW-7@gated-at.bofh.it>
In reply to#1254224
 Dear Tony,

> -----Original Message-----
> From: Luck, Tony [mailto:tony.luck@intel.com]
> Sent: Friday, October 23, 2015 8:27 AM
> To: Kamezawa, Hiroyuki/亀澤 寛之; Izumi, Taku/泉 拓; linux-kernel@vger.kernel.org; linux-mm@kvack.org
> Cc: qiuxishi@huawei.com; mel@csn.ul.ie; akpm@linux-foundation.org; Hansen, Dave; matt@codeblueprint.co.uk
> Subject: RE: [PATCH] mm: Introduce kernelcore=reliable option
> 
> > I think /proc/zoneinfo can show detailed numbers per zone. Do we need some for meminfo ?
> 
> I wrote a little script (attached) to summarize /proc/zoneinfo ... on my system it says
> 
> $ zoneinfo
> Node          Normal         Movable             DMA           DMA32
>    0            0.00       103020.07            8.94         1554.46
>    1         9284.54        89870.43
>    2         9626.33        94050.09
>    3         9602.82        93650.04
> 
> Not sure why I have zero Normal memory free on node0.  The sum of all those
> free counts is 410667.72 MB ... which is close enough to the boot time message
> showing the amount of mirror/total memory:
> 
> [    0.000000] efi: Memory: 80979/420096M mirrored memory
> 
> but a fair amount of the 80G of mirrored memory seems to have been miscounted
> as Movable instead of Normal. Perhaps this is because I have two blocks of mirrored
> memory on each node and the movable zone code doesn't expect that?

 You were saying that OS view of memory of node is something like the following ?
  
    Node X:  |MMMMMM------MMMMMM--------|  
       (legend) M: mirrored  -: not mirrrored

 If so, is this a real Box's configuration?
 Sorry, I haven't got a real Address Range Mirror capable boxes yet ...
 I thought mirroring range is concatenated at the first part of each node.

 Sincerely,
 Taku Izumi

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1254279

From"Luck, Tony" <tony.luck@intel.com>
Date2015-10-23 03:50 +0200
Message-ID<qmzEu-3yN-9@gated-at.bofh.it>
In reply to#1254266
First part of each memory controller. I have two memory controllers on each node

Sent from my iPhone

> On Oct 22, 2015, at 18:01, Izumi, Taku <izumi.taku@jp.fujitsu.com> wrote:
> 
> Dear Tony,
> 
>> -----Original Message-----
>> From: Luck, Tony [mailto:tony.luck@intel.com]
>> Sent: Friday, October 23, 2015 8:27 AM
>> To: Kamezawa, Hiroyuki/亀澤 寛之; Izumi, Taku/泉 拓; linux-kernel@vger.kernel.org; linux-mm@kvack.org
>> Cc: qiuxishi@huawei.com; mel@csn.ul.ie; akpm@linux-foundation.org; Hansen, Dave; matt@codeblueprint.co.uk
>> Subject: RE: [PATCH] mm: Introduce kernelcore=reliable option
>> 
>>> I think /proc/zoneinfo can show detailed numbers per zone. Do we need some for meminfo ?
>> 
>> I wrote a little script (attached) to summarize /proc/zoneinfo ... on my system it says
>> 
>> $ zoneinfo
>> Node          Normal         Movable             DMA           DMA32
>>   0            0.00       103020.07            8.94         1554.46
>>   1         9284.54        89870.43
>>   2         9626.33        94050.09
>>   3         9602.82        93650.04
>> 
>> Not sure why I have zero Normal memory free on node0.  The sum of all those
>> free counts is 410667.72 MB ... which is close enough to the boot time message
>> showing the amount of mirror/total memory:
>> 
>> [    0.000000] efi: Memory: 80979/420096M mirrored memory
>> 
>> but a fair amount of the 80G of mirrored memory seems to have been miscounted
>> as Movable instead of Normal. Perhaps this is because I have two blocks of mirrored
>> memory on each node and the movable zone code doesn't expect that?
> 
> You were saying that OS view of memory of node is something like the following ?
> 
>    Node X:  |MMMMMM------MMMMMM--------|  
>       (legend) M: mirrored  -: not mirrrored
> 
> If so, is this a real Box's configuration?
> Sorry, I haven't got a real Address Range Mirror capable boxes yet ...
> I thought mirroring range is concatenated at the first part of each node.
> 
> Sincerely,
> Taku Izumi
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1259236

FromKamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Date2015-10-30 07:30 +0100
Message-ID<qpbmh-2ij-13@gated-at.bofh.it>
In reply to#1254279
On 2015/10/23 10:44, Luck, Tony wrote:
> First part of each memory controller. I have two memory controllers on each node
> 

If each memory controller has the same distance/latency, you (your firmware) don't need
to allocate reliable memory per each memory controller.
If distance is problem, another node should be allocated.

...is the behavior(splitting zone) really required ?

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1259752

From"Luck, Tony" <tony.luck@intel.com>
Date2015-10-30 20:50 +0100
Message-ID<qpnQu-1sY-15@gated-at.bofh.it>
In reply to#1259236
> If each memory controller has the same distance/latency, you (your firmware) don't need
> to allocate reliable memory per each memory controller.
> If distance is problem, another node should be allocated.
>
> ...is the behavior(splitting zone) really required ?

It's useful from a memory bandwidth perspective to have allocations
spread across both memory controllers. Keeping a whole bunch of
Xeon cores fed needs all the bandwidth you can get.

Socket0 is also a problem.  We want to mirror <4GB addresses because
there is a bunch of critical stuff there (entire kernel text+data). But we
can currently only mirror one block per memory controller, so we end up
with just 2GB mirrored (the 2GB-4GB range is MMIO).  This isn't enough
for even a small machine (I have 128GB on node0 ... but that is really the
bare minimum configuration ... 2GB is only enough to cover the "struct
page" allocations for node0).  I really have to allocate some more mirror
from the other memory controller.

-Tony

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web