Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1382073 > unrolled thread

Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 0/2] memory_hotplug: introduce config and command line  options to set the default onlining policy David Rientjes <rientjes@google.com> - 2016-04-18 23:40 +0200
    Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy Vitaly Kuznetsov <vkuznets@redhat.com> - 2016-04-19 09:30 +0200
      Re: [PATCH 0/2] memory_hotplug: introduce config and command line  options to set the default onlining policy David Rientjes <rientjes@google.com> - 2016-04-20 23:40 +0200
        Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy Vitaly Kuznetsov <vkuznets@redhat.com> - 2016-04-21 09:30 +0200

#1382073 — Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy

On Thu, 7 Apr 2016, Vitaly Kuznetsov wrote:

> >> > This patchset continues the work I started with:
> >> > 
> >> > commit 31bc3858ea3ebcc3157b3f5f0e624c5962f5a7a6
> >> > Author: Vitaly Kuznetsov <vkuznets@redhat.com>
> >> > Date:   Tue Mar 15 14:56:48 2016 -0700
> >> > 
> >> >     memory-hotplug: add automatic onlining policy for the newly added memory
> >> > 
> >> > Initially I was going to stop there and bring the policy setting logic to
> >> > userspace. I met two issues on this way:
> >> > 
> >> > 1) It is possible to have memory hotplugged at boot (e.g. with QEMU). These
> >> >    blocks stay offlined if we turn the onlining policy on by userspace.
> >> > 
> >> > 2) My attempt to bring this policy setting to systemd failed, systemd
> >> >    maintainers suggest to change the default in kernel or ... to use tmpfiles.d
> >> >    to alter the policy (which looks like a hack to me): 
> >> >    https://github.com/systemd/systemd/pull/2938
> >> 
> >> That discussion really didn't come to a conclusion and I don't
> >> understand why you consider Lennert's "recommended way" to be a hack?
> >> 
> >> > Here I suggest to add a config option to set the default value for the policy
> >> > and a kernel command line parameter to make the override.
> >> 
> >> But the patchset looks pretty reasonable regardless of the above.
> >> 
> >
> > I don't understand why initscripts simply cannot crawl sysfs memory blocks 
> > and online them for the same behavior.
> 
> Yes, they can. With this patchset I don't bring any new features, it's
> rather a convenience so linux distros can make memory hotplug work
> 'out of the box' without such distro-specific initscripts. Memory
> hotplug is a standard feature of all major virt technologies so I think
> it's pretty reasonable to have an option to make it work 'by default'
> available.
> 

I'd personally disagree that we need more and more config options to take 
care of something that an initscript can easily do and most distros 
already have their own initscripts that this can be added to.  I don't see 
anything that the config option adds.

[toc] | [next] | [standalone]

#1382225 — Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy

David Rientjes <rientjes@google.com> writes:

> On Thu, 7 Apr 2016, Vitaly Kuznetsov wrote:
>
>> >> > This patchset continues the work I started with:
>> >> > 
>> >> > commit 31bc3858ea3ebcc3157b3f5f0e624c5962f5a7a6
>> >> > Author: Vitaly Kuznetsov <vkuznets@redhat.com>
>> >> > Date:   Tue Mar 15 14:56:48 2016 -0700
>> >> > 
>> >> >     memory-hotplug: add automatic onlining policy for the newly added memory
>> >> > 
>> >> > Initially I was going to stop there and bring the policy setting logic to
>> >> > userspace. I met two issues on this way:
>> >> > 
>> >> > 1) It is possible to have memory hotplugged at boot (e.g. with QEMU). These
>> >> >    blocks stay offlined if we turn the onlining policy on by userspace.
>> >> > 
>> >> > 2) My attempt to bring this policy setting to systemd failed, systemd
>> >> >    maintainers suggest to change the default in kernel or ... to use tmpfiles.d
>> >> >    to alter the policy (which looks like a hack to me): 
>> >> >    https://github.com/systemd/systemd/pull/2938
>> >> 
>> >> That discussion really didn't come to a conclusion and I don't
>> >> understand why you consider Lennert's "recommended way" to be a hack?
>> >> 
>> >> > Here I suggest to add a config option to set the default value for the policy
>> >> > and a kernel command line parameter to make the override.
>> >> 
>> >> But the patchset looks pretty reasonable regardless of the above.
>> >> 
>> >
>> > I don't understand why initscripts simply cannot crawl sysfs memory blocks 
>> > and online them for the same behavior.
>> 
>> Yes, they can. With this patchset I don't bring any new features, it's
>> rather a convenience so linux distros can make memory hotplug work
>> 'out of the box' without such distro-specific initscripts. Memory
>> hotplug is a standard feature of all major virt technologies so I think
>> it's pretty reasonable to have an option to make it work 'by default'
>> available.
>> 
>
> I'd personally disagree that we need more and more config options to take 
> care of something that an initscript can easily do and most distros 
> already have their own initscripts that this can be added to.  I don't see 
> anything that the config option adds.

Yes, but why does every distro need to solve the exact same issue by 
a distro-specific init script when we can allow setting reasonable
default in kernel?

If the config option itself is a problem (though I don't understand why)
we can get rid of it making the default 'online' and keeping the command
line parameter to disable it for cases when something goes wrong but why
not leave an option for those who want it the other way around?

Other than the above, let's imagine a 'unikernel' scenario when there
are no initscripts and we're in a virtualized environment. We may want to
have memory hotplug there too, but where would we put the 'onlining'
logic? In every userspace we want to run? This doesn't sound right.

-- 
  Vitaly

[toc] | [prev] | [next] | [standalone]

#1383759

On Tue, 19 Apr 2016, Vitaly Kuznetsov wrote:

> > I'd personally disagree that we need more and more config options to take 
> > care of something that an initscript can easily do and most distros 
> > already have their own initscripts that this can be added to.  I don't see 
> > anything that the config option adds.
> 
> Yes, but why does every distro need to solve the exact same issue by 
> a distro-specific init script when we can allow setting reasonable
> default in kernel?
> 

No, only distros that want to change the long-standing default which is 
"offline" since they apparently aren't worried about breaking existing 
userspace.

Changing defaults is always risky business in the kernel, especially when 
it's long standing.  If the default behavior is changeable, userspace 
needs to start testing for that and acting accordingly if it actually 
wants to default to offline (and there are existing tools that suppose the 
long-standing default).  The end result is that the kernel default doesn't 
matter anymore, we've just pushed it to userspace to either online or 
offline at the time of hotplug.

> If the config option itself is a problem (though I don't understand why)
> we can get rid of it making the default 'online' and keeping the command
> line parameter to disable it for cases when something goes wrong but why
> not leave an option for those who want it the other way around?
> 

That could break existing userspace that assumes the default is offline; 
if users are currently hotadding memory and then onlining it when needed 
rather than immediately, they break.  So that's not a possibility.

> Other than the above, let's imagine a 'unikernel' scenario when there
> are no initscripts and we're in a virtualized environment. We may want to
> have memory hotplug there too, but where would we put the 'onlining'
> logic? In every userspace we want to run? This doesn't sound right.
> 

Nobody is resisting hotplug notifiers.

[toc] | [prev] | [next] | [standalone]

#1383906 — Re: [PATCH 0/2] memory_hotplug: introduce config and command line options to set the default onlining policy

David Rientjes <rientjes@google.com> writes:

> On Tue, 19 Apr 2016, Vitaly Kuznetsov wrote:
>
>> > I'd personally disagree that we need more and more config options to take 
>> > care of something that an initscript can easily do and most distros 
>> > already have their own initscripts that this can be added to.  I don't see 
>> > anything that the config option adds.
>> 
>> Yes, but why does every distro need to solve the exact same issue by 
>> a distro-specific init script when we can allow setting reasonable
>> default in kernel?
>> 
>
> No, only distros that want to change the long-standing default which is 
> "offline" since they apparently aren't worried about breaking existing 
> userspace.
>
> Changing defaults is always risky business in the kernel, especially when 
> it's long standing.  If the default behavior is changeable, userspace 
> needs to start testing for that and acting accordingly if it actually 
> wants to default to offline (and there are existing tools that suppose the 
> long-standing default).  The end result is that the kernel default doesn't 
> matter anymore, we've just pushed it to userspace to either online or 
> offline at the time of hotplug.
>

"We don't break userspace". Yes, I know, but is there an example of such
userspace which is going to break?

E.g. RHEL7 ships the following udev rule by default:
# Memory hotadd request
SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"

which is not very smart but it does the job (with issues I'm trying to
solve). I'm not aware of any breakages reported after it was introduced.

My understanding is that the legacy default 'offline' was introduced
before memory hotplug became a frequently used feature in virtual
machines. When you hotplug physical memory you go to your server room,
open your server, insert memory dimm, ... - in this scenario 'offline'
is a reasonable default. But in VMs mempory hotplug is usually an
automatic from host side -- we address high memory pressure/tenant
requests.

>> If the config option itself is a problem (though I don't understand why)
>> we can get rid of it making the default 'online' and keeping the command
>> line parameter to disable it for cases when something goes wrong but why
>> not leave an option for those who want it the other way around?
>> 
>
> That could break existing userspace that assumes the default is offline; 
> if users are currently hotadding memory and then onlining it when needed 
> rather than immediately, they break.  So that's not a possibility.
>

Yes, so I introduce a config option. Next thing we do we enable it in
'bleeding edge' distros, e.g. Fedora and see who complains. My guess is
that nobody is going to complain.

>> Other than the above, let's imagine a 'unikernel' scenario when there
>> are no initscripts and we're in a virtualized environment. We may want to
>> have memory hotplug there too, but where would we put the 'onlining'
>> logic? In every userspace we want to run? This doesn't sound right.
>> 
>
> Nobody is resisting hotplug notifiers.

Yes, but we need to teach memory hotplug to every userspace instead.

-- 
  Vitaly

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web