Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1178726 > unrolled thread

Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity setting

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity  setting Oleg Nesterov <oleg@redhat.com> - 2015-07-07 18:40 +0200
    Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity  setting Frederic Weisbecker <fweisbec@gmail.com> - 2015-07-07 19:20 +0200
      Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity  setting Oleg Nesterov <oleg@redhat.com> - 2015-07-07 19:40 +0200
        Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity  setting Oleg Nesterov <oleg@redhat.com> - 2015-07-07 19:50 +0200

#1178726 — Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity setting

On 07/06, Frederic Weisbecker wrote:
>
> The call_usermodehelper_exec_[a]sync() kernel threads are created by
> khelper precisely because

I think khelper should simply die. It doesn't make any sense today,
kmod.c should use some system wq instead. But see below.

> Not only useless it even breaks nohz full. The housekeeping work
> (general kernel internal code that user doesn't care much about) is
> handled by a reduced set of CPUs in nohz full, precisely those that are
> not included by nohz_full= kernel parameters. For example unbound
> workqueues are handled by housekeeping CPUs.

Confused... I do not see how workqueue_attrs->cpumask can depend on
tick_nohz_full_mask or housekeeping_mask. Could you explain?

> @@ -223,9 +223,6 @@ static int call_usermodehelper_exec_async(void *data)
>  	flush_signal_handlers(current, 1);
>  	spin_unlock_irq(&current->sighand->siglock);
>
> -	/* We can run anywhere, unlike our parent keventd(). */
> -	set_cpus_allowed_ptr(current, cpu_all_mask);
> -

I think this is fine, ->no_numa is true for khelper.

But this means that after this change kmod.c can't use a system wq,
->no_numa is false by default. And khelper is no_numa only because
it is __WQ_ORDERED, but kmod.c doesn't need really need__WQ_ORDERED,
except, again, this implies ->no_numa == T.

So perhaps init_workqueues() should create another global
WQ_UNBOUND/no_numa workqueue_struct so that we could kill khelper_wq?
Or kmod.c can use system_unbound_wq, but then we need to keep this
set_cpus_allowed_ptr().

To me khelper_wq looks just annoying...

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1178762

On Tue, Jul 07, 2015 at 06:30:30PM +0200, Oleg Nesterov wrote:
> On 07/06, Frederic Weisbecker wrote:
> >
> > The call_usermodehelper_exec_[a]sync() kernel threads are created by
> > khelper precisely because
> 
> I think khelper should simply die. It doesn't make any sense today,
> kmod.c should use some system wq instead. But see below.
> 
> > Not only useless it even breaks nohz full. The housekeeping work
> > (general kernel internal code that user doesn't care much about) is
> > handled by a reduced set of CPUs in nohz full, precisely those that are
> > not included by nohz_full= kernel parameters. For example unbound
> > workqueues are handled by housekeeping CPUs.
> 
> Confused... I do not see how workqueue_attrs->cpumask can depend on
> tick_nohz_full_mask or housekeeping_mask. Could you explain?

People who want CPU isolation will likely write
/sys/devices/virtual/workqueue/cpumask to a reduced set of CPUs, typically
CPU 0 that is used for housekeeping in nohz full.

In fact we should add the code which initialize wq_unbound_cpumask
to housekeeping_mask automatically.

So this cpumask is inherited to khelper because it is a singlethread
workqueue.

> 
> > @@ -223,9 +223,6 @@ static int call_usermodehelper_exec_async(void *data)
> >  	flush_signal_handlers(current, 1);
> >  	spin_unlock_irq(&current->sighand->siglock);
> >
> > -	/* We can run anywhere, unlike our parent keventd(). */
> > -	set_cpus_allowed_ptr(current, cpu_all_mask);
> > -
> 
> I think this is fine, ->no_numa is true for khelper.
> 
> But this means that after this change kmod.c can't use a system wq,
> ->no_numa is false by default. And khelper is no_numa only because
> it is __WQ_ORDERED, but kmod.c doesn't need really need__WQ_ORDERED,
> except, again, this implies ->no_numa == T.

I'm not sure what means no_numa in the context of workqueues, I guess
it's about having system workqueues bound to one CPU or several in the
same nodes. But indeed we can't use system workqueues because they are
per-cpu and we inherit that. And it's ridiculous to call set_cpus_allowed_ptr()
to fix that.

Hence why we use a singlethread, even though we don't care about ordering.
Ok I guess that's more or less what you just said :o)

> So perhaps init_workqueues() should create another global
> WQ_UNBOUND/no_numa workqueue_struct so that we could kill khelper_wq?
> Or kmod.c can use system_unbound_wq, but then we need to keep this
> set_cpus_allowed_ptr().
> 
> To me khelper_wq looks just annoying.

That's a good idea. I can do that!

Perhaps queuing there would be done through schedule_work_unbound() ?
Or schedule_work_no_numa()?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1178791

On 07/07, Frederic Weisbecker wrote:
>
> On Tue, Jul 07, 2015 at 06:30:30PM +0200, Oleg Nesterov wrote:
> >
> > > Not only useless it even breaks nohz full. The housekeeping work
> > > (general kernel internal code that user doesn't care much about) is
> > > handled by a reduced set of CPUs in nohz full, precisely those that are
> > > not included by nohz_full= kernel parameters. For example unbound
> > > workqueues are handled by housekeeping CPUs.
> >
> > Confused... I do not see how workqueue_attrs->cpumask can depend on
> > tick_nohz_full_mask or housekeeping_mask. Could you explain?
>
> People who want CPU isolation will likely write
> /sys/devices/virtual/workqueue/cpumask to a reduced set of CPUs, typically
> CPU 0 that is used for housekeeping in nohz full.

Well, khelper_wq is not WQ_SYSFS, so I am not sure this is possible.

But this doesn't really matter, people can change cpu affinity. But
"workqueues are handled by housekeeping CPUs" doesn't look right.

> In fact we should add the code which initialize wq_unbound_cpumask
> to housekeeping_mask automatically.

Perhaps, but until then the changelog above looks really confusing,
as if workqueue.c already does this automagically ;)

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1178802

On 07/07, Oleg Nesterov wrote:
>
> On 07/07, Frederic Weisbecker wrote:
> >
> >
> > People who want CPU isolation will likely write
> > /sys/devices/virtual/workqueue/cpumask to a reduced set of CPUs, typically
> > CPU 0 that is used for housekeeping in nohz full.
>
> Well, khelper_wq is not WQ_SYSFS, so I am not sure this is possible.

Please ignore, I see the new "workqueue: Create low-level unbound
workqueues cpumask" commit...


> But this doesn't really matter, people can change cpu affinity. But
> "workqueues are handled by housekeeping CPUs" doesn't look right.
>
> > In fact we should add the code which initialize wq_unbound_cpumask
> > to housekeeping_mask automatically.
>
> Perhaps, but until then the changelog above looks really confusing,
> as if workqueue.c already does this automagically ;)

Yes.

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web