Re: WTH is going on with memory hotplug sysf interface

From	Michal Hocko <mhocko@kernel.org>
Newsgroups	linux.kernel
Subject	Re: WTH is going on with memory hotplug sysf interface
Date	2017-03-13 10:30 +0100
Message-ID	<tkusG-6Sd-23@gated-at.bofh.it> (permalink)
References	(5 earlier) <ti2qS-1v4-17@gated-at.bofh.it> <timIX-7Bl-37@gated-at.bofh.it> <tj5PJ-5jo-23@gated-at.bofh.it> <tjtfj-4NB-5@gated-at.bofh.it> <tjwGd-7eN-11@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Fri 10-03-17 12:39:27, Yasuaki Ishimatsu wrote:
> On 03/10/2017 08:58 AM, Michal Hocko wrote:
[...]
> >OK so I did with -m 2G,slots=4,maxmem=4G -numa node,mem=1G -numa node,mem=1G which generated
> >[...]
> >[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x0009ffff]
> >[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x00100000-0x3fffffff]
> >[    0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0x40000000-0x7fffffff]
> >[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x27fffffff] hotplug
> >[    0.000000] NUMA: Node 0 [mem 0x00000000-0x0009ffff] + [mem 0x00100000-0x3fffffff] -> [mem 0x00000000-0x3fffffff]
> >[    0.000000] NODE_DATA(0) allocated [mem 0x3fffc000-0x3fffffff]
> >[    0.000000] NODE_DATA(1) allocated [mem 0x7ffdc000-0x7ffdffff]
> >[    0.000000] Zone ranges:
> >[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
> >[    0.000000]   DMA32    [mem 0x0000000001000000-0x000000007ffdffff]
> >[    0.000000]   Normal   empty
> >[    0.000000] Movable zone start for each node
> >[    0.000000] Early memory node ranges
> >[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
> >[    0.000000]   node   0: [mem 0x0000000000100000-0x000000003fffffff]
> >[    0.000000]   node   1: [mem 0x0000000040000000-0x000000007ffdffff]
> >
> >so there is neither any normal zone nor movable one at the boot time.
> >Then I hotplugged 1G slot
> >(qemu) object_add memory-backend-ram,id=mem1,size=1G
> >(qemu) device_add pc-dimm,id=dimm1,memdev=mem1
> >
> >unfortunatelly the memory didn't show up automatically and I got
> >[  116.375781] acpi PNP0C80:00: Enumeration failure
> >
> >so I had to probe it manually (prbably the BIOS my qemu uses doesn't
> >support auto probing - I haven't really dug further). Anyway the SRAT
> >table printed during the boot told that we should start at 0x100000000
> >
> ># echo 0x100000000 > /sys/devices/system/memory/probe
> ># grep . /sys/devices/system/memory/memory32/valid_zones
> >Normal Movable
> >
> >which looks reasonably right? Both Normal and Movable zones are allowed
> >
> ># echo $((0x100000000+(128<<20))) > /sys/devices/system/memory/probe
> ># grep . /sys/devices/system/memory/memory3?/valid_zones
> >/sys/devices/system/memory/memory32/valid_zones:Normal
> >/sys/devices/system/memory/memory33/valid_zones:Normal Movable
> >
> >Huh, so our valid_zones have changed under our feet...
> >
> ># echo $((0x100000000+2*(128<<20))) > /sys/devices/system/memory/probe
> ># grep . /sys/devices/system/memory/memory3?/valid_zones
> >/sys/devices/system/memory/memory32/valid_zones:Normal
> >/sys/devices/system/memory/memory33/valid_zones:Normal
> >/sys/devices/system/memory/memory34/valid_zones:Normal Movable
> >
> >and again. So only the last memblock is considered movable. Let's try to
> >online them now.
> >
> ># echo online_movable > /sys/devices/system/memory/memory34/state
> ># grep . /sys/devices/system/memory/memory3?/valid_zones
> >/sys/devices/system/memory/memory32/valid_zones:Normal
> >/sys/devices/system/memory/memory33/valid_zones:Normal Movable
> >/sys/devices/system/memory/memory34/valid_zones:Movable Normal
> >
> 
> I think there is no strong reason which kernel has the restriction.
> By setting the restrictions, it seems to have made management of
> these zone structs simple.

Could you be more specific please? How could this make management any
easier when udev is basically racing with the physical hotplug and the
result is basically undefined?
-- 
Michal Hocko
SUSE Labs

Thread

Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Igor Mammedov <imammedo@redhat.com> - 2017-03-02 15:00 +0100
  Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-02 16:10 +0100
    Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Igor Mammedov <imammedo@redhat.com> - 2017-03-02 18:20 +0100
      Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-03 10:10 +0100
        Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Igor Mammedov <imammedo@redhat.com> - 2017-03-03 18:40 +0100
          Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-06 16:10 +0100
            Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Igor Mammedov <imammedo@redhat.com> - 2017-03-07 13:50 +0100
              Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-09 14:00 +0100
                WTH is going on with memory hotplug sysf interface (was: Re: [RFC  PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-10 15:00 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-10 17:00 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Reza Arbab <arbab@linux.vnet.ibm.com> - 2017-03-10 20:10 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-13 10:30 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Reza Arbab <arbab@linux.vnet.ibm.com> - 2017-03-13 16:00 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-13 16:20 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Andi Kleen <ak@linux.intel.com> - 2017-03-14 00:20 +0100
                Re: WTH is going on with memory hotplug sysf interface Yasuaki Ishimatsu <yasu.isimatu@gmail.com> - 2017-03-10 18:40 +0100
                Re: WTH is going on with memory hotplug sysf interface Michal Hocko <mhocko@kernel.org> - 2017-03-13 10:30 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Igor Mammedov <imammedo@redhat.com> - 2017-03-13 11:40 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-13 11:50 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Igor Mammedov <imammedo@redhat.com> - 2017-03-13 15:00 +0100
                Re: WTH is going on with memory hotplug sysf interface (was: Re:  [RFC PATCH] mm, hotplug: get rid of auto_online_blocks) Michal Hocko <mhocko@kernel.org> - 2017-03-13 15:40 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Igor Mammedov <imammedo@redhat.com> - 2017-03-13 12:00 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-13 13:30 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-03-13 14:00 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-13 14:20 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-03-13 14:50 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Michal Hocko <mhocko@kernel.org> - 2017-03-13 15:40 +0100
                Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-03-13 16:20 +0100
            Re: [RFC PATCH] mm, hotplug: get rid of auto_online_blocks Daniel Kiper <daniel.kiper@oracle.com> - 2017-03-10 23:10 +0100

csiph-web