Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #73532 > unrolled thread

Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE

Back to article view | Back to linux.debian.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Georg Müller <georgmueller@gmx.net> - 2021-10-27 10:30 +0200
    Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Salvatore Bonaccorso <carnil@debian.org> - 2021-10-30 16:10 +0200
      Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Henning Schild <henning.schild@siemens.com> - 2021-11-04 22:10 +0100
        Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Salvatore Bonaccorso <carnil@debian.org> - 2021-11-07 11:30 +0100
        Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Frederic Weisbecker <frederic@kernel.org> - 2021-11-13 00:40 +0100
          Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Salvatore Bonaccorso <carnil@debian.org> - 2021-11-13 08:50 +0100
          Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE Henning Schild <henning.schild@siemens.com> - 2021-11-15 11:50 +0100

#73532 — Bug#804857: linux: New feature: enable CONFIG_NO_HZ_FULL and CONFIG_RCU_NOCB_CPU/CONFIG_RCU_NOCB_CPU_NONE

> But for other configurations it is worse:
>
> config NO_HZ_FULL
>         bool "Full dynticks system (tickless)"
> ...
>          This is implemented at the expense of some overhead in user <-> kernel
>          transitions: syscalls, exceptions and interrupts. Even when it's
>          dynamically off.
>
>          Say N.
>


Upstream commit 176b8906 changed the description regarding NO_HZ_FULL:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61


 > By default, without passing the nohz_full parameter, this behaves just
 > like NO_HZ_IDLE.
 >
 > If you're a distro say Y.

[toc] | [next] | [standalone]

#73555

Control: tags -1 + moreinfo

On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:
> > But for other configurations it is worse:
> > 
> > config NO_HZ_FULL
> >         bool "Full dynticks system (tickless)"
> > ...
> >          This is implemented at the expense of some overhead in user <-> kernel
> >          transitions: syscalls, exceptions and interrupts. Even when it's
> >          dynamically off.
> > 
> >          Say N.
> > 
> 
> 
> Upstream commit 176b8906 changed the description regarding NO_HZ_FULL:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> 
> 
> > By default, without passing the nohz_full parameter, this behaves just
> > like NO_HZ_IDLE.
> >
> > If you're a distro say Y.

While this is changed, and distros encouraged to select it, selecting
this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN and
CONFIG_RCU_NOCB_CPU.

For CONFIG_VIRT_CPU_ACCOUNTING_GEN

          Select this option to enable task and CPU time accounting on full
          dynticks systems. This accounting is implemented by watching every
          kernel-user boundaries using the context tracking subsystem.
          The accounting is thus performed at the expense of some significant
          overhead.

          For now this is only useful if you are working on the full
          dynticks subsystem development.

          If unsure, say N.

which indicates some significant overhead.

And for CONFIG_RCU_NOCB_CPU

          Use this option to reduce OS jitter for aggressive HPC or
          real-time workloads.  It can also be used to offload RCU
          callback invocation to energy-efficient CPUs in battery-powered
          asymmetric multiprocessors.  The price of this reduced jitter
          is that the overhead of call_rcu() increases and that some
          workloads will incur significant increases in context-switch
          rates.

          This option offloads callback invocation from the set of CPUs
          specified at boot time by the rcu_nocbs parameter.  For each
          such CPU, a kthread ("rcuox/N") will be created to invoke
          callbacks, where the "N" is the CPU being offloaded, and where
          the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for
          RCU-sched (!PREEMPTION kernels).  Nothing prevents this kthread
          from running on the specified CPUs, but (1) the kthreads may be
          preempted between each callback, and (2) affinity or cgroups can
          be used to force the kthreads to run on whatever set of CPUs is
          desired.

          Say Y here if you need reduced OS jitter, despite added overhead.
          Say N here if you are unsure.

Adding as well overhead.

Is this still to be considered true?

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]

#73608

Am Sat, 30 Oct 2021 16:04:35 +0200
schrieb Salvatore Bonaccorso <carnil@debian.org>:

> Control: tags -1 + moreinfo
> 
> On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:
> > > But for other configurations it is worse:
> > > 
> > > config NO_HZ_FULL
> > >         bool "Full dynticks system (tickless)"
> > > ...
> > >          This is implemented at the expense of some overhead in
> > > user <-> kernel transitions: syscalls, exceptions and interrupts.
> > > Even when it's dynamically off.
> > > 
> > >          Say N.
> > >   
> > 
> > 
> > Upstream commit 176b8906 changed the description regarding
> > NO_HZ_FULL:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> > 
> >   
> > > By default, without passing the nohz_full parameter, this behaves
> > > just like NO_HZ_IDLE.
> > >
> > > If you're a distro say Y.  
> 
> While this is changed, and distros encouraged to select it, selecting
> this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN and
> CONFIG_RCU_NOCB_CPU.
> 
> For CONFIG_VIRT_CPU_ACCOUNTING_GEN
> 
>           Select this option to enable task and CPU time accounting
> on full dynticks systems. This accounting is implemented by watching
> every kernel-user boundaries using the context tracking subsystem.
>           The accounting is thus performed at the expense of some
> significant overhead.
> 
>           For now this is only useful if you are working on the full
>           dynticks subsystem development.
> 
>           If unsure, say N.
> 
> which indicates some significant overhead.

I can not answer that from the back of my head. Would have to dig as
well. Might get back in about two weeks if nobody else finds an answer.

But i took the liberty to include Frederic into Cc, the author of the
"distro reassure" patch.

Not sure such a change would be allowed for bullseye (5.10) and if the
answer for 5.10 would be another than for i.e. 5.15

Reading what it is, maybe it can in fact be decoupled from NO_HZ_FULL.
Which would mean an upstream patch and backporting (if preempt would do
that, but could be considered a "performance bug" i guess)

> And for CONFIG_RCU_NOCB_CPU
> 
>           Use this option to reduce OS jitter for aggressive HPC or
>           real-time workloads.  It can also be used to offload RCU
>           callback invocation to energy-efficient CPUs in
> battery-powered asymmetric multiprocessors.  The price of this
> reduced jitter is that the overhead of call_rcu() increases and that
> some workloads will incur significant increases in context-switch
>           rates.
> 
>           This option offloads callback invocation from the set of
> CPUs specified at boot time by the rcu_nocbs parameter.  For each
>           such CPU, a kthread ("rcuox/N") will be created to invoke
>           callbacks, where the "N" is the CPU being offloaded, and
> where the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for
>           RCU-sched (!PREEMPTION kernels).  Nothing prevents this
> kthread from running on the specified CPUs, but (1) the kthreads may
> be preempted between each callback, and (2) affinity or cgroups can
>           be used to force the kthreads to run on whatever set of
> CPUs is desired.
> 
>           Say Y here if you need reduced OS jitter, despite added
> overhead. Say N here if you are unsure.
> 
> Adding as well overhead.
> 
> Is this still to be considered true?

probably but only for people that actively choose to use it and only
for the CPUs they choose. "rcu_nocbs" cmdline param, if not set
everything will be as it was.
I already indicated that in the commit message of my MR:
https://salsa.debian.org/kernel-team/linux/-/merge_requests/385

regards,
Henning


> 
> Regards,
> Salvatore
> 

[toc] | [prev] | [next] | [standalone]

#73627

Hi Henning,

On Thu, Nov 04, 2021 at 10:05:02PM +0100, Henning Schild wrote:
> Am Sat, 30 Oct 2021 16:04:35 +0200
> schrieb Salvatore Bonaccorso <carnil@debian.org>:
> 
> > Control: tags -1 + moreinfo
> > 
> > On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:
> > > > But for other configurations it is worse:
> > > > 
> > > > config NO_HZ_FULL
> > > >         bool "Full dynticks system (tickless)"
> > > > ...
> > > >          This is implemented at the expense of some overhead in
> > > > user <-> kernel transitions: syscalls, exceptions and interrupts.
> > > > Even when it's dynamically off.
> > > > 
> > > >          Say N.
> > > >   
> > > 
> > > 
> > > Upstream commit 176b8906 changed the description regarding
> > > NO_HZ_FULL:
> > > 
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> > > 
> > >   
> > > > By default, without passing the nohz_full parameter, this behaves
> > > > just like NO_HZ_IDLE.
> > > >
> > > > If you're a distro say Y.  
> > 
> > While this is changed, and distros encouraged to select it, selecting
> > this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN and
> > CONFIG_RCU_NOCB_CPU.
> > 
> > For CONFIG_VIRT_CPU_ACCOUNTING_GEN
> > 
> >           Select this option to enable task and CPU time accounting
> > on full dynticks systems. This accounting is implemented by watching
> > every kernel-user boundaries using the context tracking subsystem.
> >           The accounting is thus performed at the expense of some
> > significant overhead.
> > 
> >           For now this is only useful if you are working on the full
> >           dynticks subsystem development.
> > 
> >           If unsure, say N.
> > 
> > which indicates some significant overhead.
> 
> I can not answer that from the back of my head. Would have to dig as
> well. Might get back in about two weeks if nobody else finds an answer.
> 
> But i took the liberty to include Frederic into Cc, the author of the
> "distro reassure" patch.

Thanks, lets see what Frederic can comment on it.

> Not sure such a change would be allowed for bullseye (5.10) and if the
> answer for 5.10 would be another than for i.e. 5.15
> 
> Reading what it is, maybe it can in fact be decoupled from NO_HZ_FULL.
> Which would mean an upstream patch and backporting (if preempt would do
> that, but could be considered a "performance bug" i guess)

It would for now defintively be anyway for the uploads following 5.15.
Enabling it now for the bullseye kernel is mostly out of scope I
think, unless not further discussed again. But the first aim here
would be to make it for unstable.

> > And for CONFIG_RCU_NOCB_CPU
> > 
> >           Use this option to reduce OS jitter for aggressive HPC or
> >           real-time workloads.  It can also be used to offload RCU
> >           callback invocation to energy-efficient CPUs in
> > battery-powered asymmetric multiprocessors.  The price of this
> > reduced jitter is that the overhead of call_rcu() increases and that
> > some workloads will incur significant increases in context-switch
> >           rates.
> > 
> >           This option offloads callback invocation from the set of
> > CPUs specified at boot time by the rcu_nocbs parameter.  For each
> >           such CPU, a kthread ("rcuox/N") will be created to invoke
> >           callbacks, where the "N" is the CPU being offloaded, and
> > where the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for
> >           RCU-sched (!PREEMPTION kernels).  Nothing prevents this
> > kthread from running on the specified CPUs, but (1) the kthreads may
> > be preempted between each callback, and (2) affinity or cgroups can
> >           be used to force the kthreads to run on whatever set of
> > CPUs is desired.
> > 
> >           Say Y here if you need reduced OS jitter, despite added
> > overhead. Say N here if you are unsure.
> > 
> > Adding as well overhead.
> > 
> > Is this still to be considered true?
> 
> probably but only for people that actively choose to use it and only
> for the CPUs they choose. "rcu_nocbs" cmdline param, if not set
> everything will be as it was.
> I already indicated that in the commit message of my MR:
> https://salsa.debian.org/kernel-team/linux/-/merge_requests/385

Right, but that is not fully clear to me from the above if it's only
an overhead when as well only the parameter or if adds overhead
already by enabling the option as well.

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]

#73671

On Thu, Nov 04, 2021 at 10:05:02PM +0100, Henning Schild wrote:
> Am Sat, 30 Oct 2021 16:04:35 +0200
> schrieb Salvatore Bonaccorso <carnil@debian.org>:
> 
> > Control: tags -1 + moreinfo
> > 
> > On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:
> > > > But for other configurations it is worse:
> > > > 
> > > > config NO_HZ_FULL
> > > >         bool "Full dynticks system (tickless)"
> > > > ...
> > > >          This is implemented at the expense of some overhead in
> > > > user <-> kernel transitions: syscalls, exceptions and interrupts.
> > > > Even when it's dynamically off.
> > > > 
> > > >          Say N.
> > > >   
> > > 
> > > 
> > > Upstream commit 176b8906 changed the description regarding
> > > NO_HZ_FULL:
> > > 
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> > > 
> > >   
> > > > By default, without passing the nohz_full parameter, this behaves
> > > > just like NO_HZ_IDLE.
> > > >
> > > > If you're a distro say Y.  
> > 
> > While this is changed, and distros encouraged to select it, selecting
> > this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN and
> > CONFIG_RCU_NOCB_CPU.
> > 
> > For CONFIG_VIRT_CPU_ACCOUNTING_GEN
> > 
> >           Select this option to enable task and CPU time accounting
> > on full dynticks systems. This accounting is implemented by watching
> > every kernel-user boundaries using the context tracking subsystem.
> >           The accounting is thus performed at the expense of some
> > significant overhead.
> > 
> >           For now this is only useful if you are working on the full
> >           dynticks subsystem development.
> > 
> >           If unsure, say N.
> > 
> > which indicates some significant overhead.
> 
> I can not answer that from the back of my head. Would have to dig as
> well. Might get back in about two weeks if nobody else finds an answer.
> 
> But i took the liberty to include Frederic into Cc, the author of the
> "distro reassure" patch.
> 
> Not sure such a change would be allowed for bullseye (5.10) and if the
> answer for 5.10 would be another than for i.e. 5.15
> 
> Reading what it is, maybe it can in fact be decoupled from NO_HZ_FULL.
> Which would mean an upstream patch and backporting (if preempt would do
> that, but could be considered a "performance bug" i guess)
> 
> > And for CONFIG_RCU_NOCB_CPU
> > 
> >           Use this option to reduce OS jitter for aggressive HPC or
> >           real-time workloads.  It can also be used to offload RCU
> >           callback invocation to energy-efficient CPUs in
> > battery-powered asymmetric multiprocessors.  The price of this
> > reduced jitter is that the overhead of call_rcu() increases and that
> > some workloads will incur significant increases in context-switch
> >           rates.
> > 
> >           This option offloads callback invocation from the set of
> > CPUs specified at boot time by the rcu_nocbs parameter.  For each
> >           such CPU, a kthread ("rcuox/N") will be created to invoke
> >           callbacks, where the "N" is the CPU being offloaded, and
> > where the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for
> >           RCU-sched (!PREEMPTION kernels).  Nothing prevents this
> > kthread from running on the specified CPUs, but (1) the kthreads may
> > be preempted between each callback, and (2) affinity or cgroups can
> >           be used to force the kthreads to run on whatever set of
> > CPUs is desired.
> > 
> >           Say Y here if you need reduced OS jitter, despite added
> > overhead. Say N here if you are unsure.
> > 
> > Adding as well overhead.
> > 
> > Is this still to be considered true?
> 
> probably but only for people that actively choose to use it and only
> for the CPUs they choose. "rcu_nocbs" cmdline param, if not set
> everything will be as it was.
> I already indicated that in the commit message of my MR:
> https://salsa.debian.org/kernel-team/linux/-/merge_requests/385

Ok so the past traditional combo for a distro is:

   CONFIG_NO_HZ_IDLE=y
   # CONFIG_RCU_NOCB_CPU is not set
   CONFIG_TICK_CPU_ACCOUNTING=y

Then nohz_full support has been introduced which allows userspace
tasks to run without being annoyed by tick interrupts. But ths feature
is for extreme workloads. So we arranged for this support not to add
additional overhead when it is not used.

This means that

   CONFIG_NO_HZ_FULL=y
   
which also automatically selects:

   CONFIG_RCU_NOCB_CPU=y
   CONFIG_VIRT_CPU_ACCOUNTING_GEN=y

are not expected to bring more overhead than their traditional
counterparts, unless kernel boot parameters such as "nohz_full="
or "rcu_nocbs=" are passed.

So you can safely enable CONFIG_NO_HZ_FULL=y. I guess the only corner
case is when you optimize your kernel for size and you are sure you
won't have any user of nohz_full for your kernel, but I suspect some
debian users, like me for example, might be interested in that feature.

I should clarify the help text for CONFIG_VIRT_CPU_ACCOUNTING_GEN that
is definitely out of date.

Thanks.

[toc] | [prev] | [next] | [standalone]

#73672

Hi Frederic,

On Sat, Nov 13, 2021 at 12:27:12AM +0100, Frederic Weisbecker wrote:
> On Thu, Nov 04, 2021 at 10:05:02PM +0100, Henning Schild wrote:
> > Am Sat, 30 Oct 2021 16:04:35 +0200
> > schrieb Salvatore Bonaccorso <carnil@debian.org>:
> > 
> > > Control: tags -1 + moreinfo
> > > 
> > > On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:
> > > > > But for other configurations it is worse:
> > > > > 
> > > > > config NO_HZ_FULL
> > > > >         bool "Full dynticks system (tickless)"
> > > > > ...
> > > > >          This is implemented at the expense of some overhead in
> > > > > user <-> kernel transitions: syscalls, exceptions and interrupts.
> > > > > Even when it's dynamically off.
> > > > > 
> > > > >          Say N.
> > > > >   
> > > > 
> > > > 
> > > > Upstream commit 176b8906 changed the description regarding
> > > > NO_HZ_FULL:
> > > > 
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> > > > 
> > > >   
> > > > > By default, without passing the nohz_full parameter, this behaves
> > > > > just like NO_HZ_IDLE.
> > > > >
> > > > > If you're a distro say Y.  
> > > 
> > > While this is changed, and distros encouraged to select it, selecting
> > > this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN and
> > > CONFIG_RCU_NOCB_CPU.
> > > 
> > > For CONFIG_VIRT_CPU_ACCOUNTING_GEN
> > > 
> > >           Select this option to enable task and CPU time accounting
> > > on full dynticks systems. This accounting is implemented by watching
> > > every kernel-user boundaries using the context tracking subsystem.
> > >           The accounting is thus performed at the expense of some
> > > significant overhead.
> > > 
> > >           For now this is only useful if you are working on the full
> > >           dynticks subsystem development.
> > > 
> > >           If unsure, say N.
> > > 
> > > which indicates some significant overhead.
> > 
> > I can not answer that from the back of my head. Would have to dig as
> > well. Might get back in about two weeks if nobody else finds an answer.
> > 
> > But i took the liberty to include Frederic into Cc, the author of the
> > "distro reassure" patch.
> > 
> > Not sure such a change would be allowed for bullseye (5.10) and if the
> > answer for 5.10 would be another than for i.e. 5.15
> > 
> > Reading what it is, maybe it can in fact be decoupled from NO_HZ_FULL.
> > Which would mean an upstream patch and backporting (if preempt would do
> > that, but could be considered a "performance bug" i guess)
> > 
> > > And for CONFIG_RCU_NOCB_CPU
> > > 
> > >           Use this option to reduce OS jitter for aggressive HPC or
> > >           real-time workloads.  It can also be used to offload RCU
> > >           callback invocation to energy-efficient CPUs in
> > > battery-powered asymmetric multiprocessors.  The price of this
> > > reduced jitter is that the overhead of call_rcu() increases and that
> > > some workloads will incur significant increases in context-switch
> > >           rates.
> > > 
> > >           This option offloads callback invocation from the set of
> > > CPUs specified at boot time by the rcu_nocbs parameter.  For each
> > >           such CPU, a kthread ("rcuox/N") will be created to invoke
> > >           callbacks, where the "N" is the CPU being offloaded, and
> > > where the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for
> > >           RCU-sched (!PREEMPTION kernels).  Nothing prevents this
> > > kthread from running on the specified CPUs, but (1) the kthreads may
> > > be preempted between each callback, and (2) affinity or cgroups can
> > >           be used to force the kthreads to run on whatever set of
> > > CPUs is desired.
> > > 
> > >           Say Y here if you need reduced OS jitter, despite added
> > > overhead. Say N here if you are unsure.
> > > 
> > > Adding as well overhead.
> > > 
> > > Is this still to be considered true?
> > 
> > probably but only for people that actively choose to use it and only
> > for the CPUs they choose. "rcu_nocbs" cmdline param, if not set
> > everything will be as it was.
> > I already indicated that in the commit message of my MR:
> > https://salsa.debian.org/kernel-team/linux/-/merge_requests/385
> 
> Ok so the past traditional combo for a distro is:
> 
>    CONFIG_NO_HZ_IDLE=y
>    # CONFIG_RCU_NOCB_CPU is not set
>    CONFIG_TICK_CPU_ACCOUNTING=y
> 
> Then nohz_full support has been introduced which allows userspace
> tasks to run without being annoyed by tick interrupts. But ths feature
> is for extreme workloads. So we arranged for this support not to add
> additional overhead when it is not used.
> 
> This means that
> 
>    CONFIG_NO_HZ_FULL=y
>    
> which also automatically selects:
> 
>    CONFIG_RCU_NOCB_CPU=y
>    CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
> 
> are not expected to bring more overhead than their traditional
> counterparts, unless kernel boot parameters such as "nohz_full="
> or "rcu_nocbs=" are passed.
> 
> So you can safely enable CONFIG_NO_HZ_FULL=y. I guess the only corner
> case is when you optimize your kernel for size and you are sure you
> won't have any user of nohz_full for your kernel, but I suspect some
> debian users, like me for example, might be interested in that feature.

Thank you. I pushed the change for the next upload:

https://salsa.debian.org/kernel-team/linux/-/commit/f6aad27f05c007d6f30b34ff77bc7ea47844f117

> I should clarify the help text for CONFIG_VIRT_CPU_ACCOUNTING_GEN that
> is definitely out of date.

Ack, thank you.

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]

#73696

Am Sat, 13 Nov 2021 00:27:12 +0100
schrieb Frederic Weisbecker <frederic@kernel.org>:

> On Thu, Nov 04, 2021 at 10:05:02PM +0100, Henning Schild wrote:
> > Am Sat, 30 Oct 2021 16:04:35 +0200
> > schrieb Salvatore Bonaccorso <carnil@debian.org>:
> >   
> > > Control: tags -1 + moreinfo
> > > 
> > > On Wed, Oct 27, 2021 at 10:16:56AM +0200, Georg Müller wrote:  
> > > > > But for other configurations it is worse:
> > > > > 
> > > > > config NO_HZ_FULL
> > > > >         bool "Full dynticks system (tickless)"
> > > > > ...
> > > > >          This is implemented at the expense of some overhead
> > > > > in user <-> kernel transitions: syscalls, exceptions and
> > > > > interrupts. Even when it's dynamically off.
> > > > > 
> > > > >          Say N.
> > > > >     
> > > > 
> > > > 
> > > > Upstream commit 176b8906 changed the description regarding
> > > > NO_HZ_FULL:
> > > > 
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=176b8906c399a170886ea4bad5b24763c6713d61
> > > > 
> > > >     
> > > > > By default, without passing the nohz_full parameter, this
> > > > > behaves just like NO_HZ_IDLE.
> > > > >
> > > > > If you're a distro say Y.    
> > > 
> > > While this is changed, and distros encouraged to select it,
> > > selecting this would enable both CONFIG_VIRT_CPU_ACCOUNTING_GEN
> > > and CONFIG_RCU_NOCB_CPU.
> > > 
> > > For CONFIG_VIRT_CPU_ACCOUNTING_GEN
> > > 
> > >           Select this option to enable task and CPU time
> > > accounting on full dynticks systems. This accounting is
> > > implemented by watching every kernel-user boundaries using the
> > > context tracking subsystem. The accounting is thus performed at
> > > the expense of some significant overhead.
> > > 
> > >           For now this is only useful if you are working on the
> > > full dynticks subsystem development.
> > > 
> > >           If unsure, say N.
> > > 
> > > which indicates some significant overhead.  
> > 
> > I can not answer that from the back of my head. Would have to dig as
> > well. Might get back in about two weeks if nobody else finds an
> > answer.
> > 
> > But i took the liberty to include Frederic into Cc, the author of
> > the "distro reassure" patch.
> > 
> > Not sure such a change would be allowed for bullseye (5.10) and if
> > the answer for 5.10 would be another than for i.e. 5.15
> > 
> > Reading what it is, maybe it can in fact be decoupled from
> > NO_HZ_FULL. Which would mean an upstream patch and backporting (if
> > preempt would do that, but could be considered a "performance bug"
> > i guess) 
> > > And for CONFIG_RCU_NOCB_CPU
> > > 
> > >           Use this option to reduce OS jitter for aggressive HPC
> > > or real-time workloads.  It can also be used to offload RCU
> > >           callback invocation to energy-efficient CPUs in
> > > battery-powered asymmetric multiprocessors.  The price of this
> > > reduced jitter is that the overhead of call_rcu() increases and
> > > that some workloads will incur significant increases in
> > > context-switch rates.
> > > 
> > >           This option offloads callback invocation from the set of
> > > CPUs specified at boot time by the rcu_nocbs parameter.  For each
> > >           such CPU, a kthread ("rcuox/N") will be created to
> > > invoke callbacks, where the "N" is the CPU being offloaded, and
> > > where the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s"
> > > for RCU-sched (!PREEMPTION kernels).  Nothing prevents this
> > > kthread from running on the specified CPUs, but (1) the kthreads
> > > may be preempted between each callback, and (2) affinity or
> > > cgroups can be used to force the kthreads to run on whatever set
> > > of CPUs is desired.
> > > 
> > >           Say Y here if you need reduced OS jitter, despite added
> > > overhead. Say N here if you are unsure.
> > > 
> > > Adding as well overhead.
> > > 
> > > Is this still to be considered true?  
> > 
> > probably but only for people that actively choose to use it and only
> > for the CPUs they choose. "rcu_nocbs" cmdline param, if not set
> > everything will be as it was.
> > I already indicated that in the commit message of my MR:
> > https://salsa.debian.org/kernel-team/linux/-/merge_requests/385  
> 
> Ok so the past traditional combo for a distro is:
> 
>    CONFIG_NO_HZ_IDLE=y
>    # CONFIG_RCU_NOCB_CPU is not set
>    CONFIG_TICK_CPU_ACCOUNTING=y
> 
> Then nohz_full support has been introduced which allows userspace
> tasks to run without being annoyed by tick interrupts. But ths feature
> is for extreme workloads. So we arranged for this support not to add
> additional overhead when it is not used.
> 
> This means that
> 
>    CONFIG_NO_HZ_FULL=y
>    
> which also automatically selects:
> 
>    CONFIG_RCU_NOCB_CPU=y
>    CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
> 
> are not expected to bring more overhead than their traditional
> counterparts, unless kernel boot parameters such as "nohz_full="
> or "rcu_nocbs=" are passed.
> 
> So you can safely enable CONFIG_NO_HZ_FULL=y. I guess the only corner
> case is when you optimize your kernel for size and you are sure you
> won't have any user of nohz_full for your kernel, but I suspect some
> debian users, like me for example, might be interested in that
> feature.
> 
> I should clarify the help text for CONFIG_VIRT_CPU_ACCOUNTING_GEN that
> is definitely out of date.

Thanks for the clarification. Maybe the patches dealing with the docs
should be backported also to stable kernels where applicable. So other
distros could find the "if you are a distro" i.e. in a recent 5.10 or
maybe even 4.x

regards,
Henning

> Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web