Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1616323 > unrolled thread

[RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

Back to article view | Back to linux.kernel

Contents

  [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-04-04 20:50 +0200
    Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Ingo Molnar <mingo@kernel.org> - 2017-04-05 09:40 +0200
      Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-04-05 10:40 +0200
        Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Ingo Molnar <mingo@kernel.org> - 2017-04-06 08:20 +0200
          Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-04-06 09:40 +0200
            Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Ingo Molnar <mingo@kernel.org> - 2017-04-06 10:10 +0200
              Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-04-06 11:30 +0200
                Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 11:50 +0200
                  Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 13:00 +0200
                    Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 13:50 +0200
              Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 11:40 +0200
                Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 11:50 +0200
              Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 12:40 +0200
                Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Ingo Molnar <mingo@kernel.org> - 2017-04-06 13:10 +0200
                  Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 13:20 +0200
                    Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Ingo Molnar <mingo@kernel.org> - 2017-04-07 09:20 +0200
      Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 11:40 +0200
    Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 11:40 +0200
      Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Sebastian Andrzej Siewior <bigeasy@linutronix.de> - 2017-04-06 11:50 +0200
        Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 12:40 +0200
          Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 12:50 +0200
            Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 13:00 +0200
              Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 13:10 +0200
                Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 14:00 +0200
                  Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Thomas Gleixner <tglx@linutronix.de> - 2017-04-06 14:00 +0200
                    Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU  mask Peter Zijlstra <peterz@infradead.org> - 2017-04-06 14:40 +0200

Page 2 of 2 — ← Prev page 1 [2]

#1617866 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

[Multipart message — attachments visible in raw view] — view raw

On Thu, 6 Apr 2017, Peter Zijlstra wrote:
> On Thu, Apr 06, 2017 at 11:46:33AM +0200, Sebastian Andrzej Siewior wrote:
> > On 2017-04-06 11:32:24 [+0200], Peter Zijlstra wrote:
> > > On Tue, Apr 04, 2017 at 08:42:02PM +0200, Sebastian Andrzej Siewior wrote:
> > > > While converting the existing users I tried to stick with the rules
> > > > above however… well mostly CPUFREQ tries to temporary switch the CPU
> > > > mask to do something on a certain CPU and then switches the mask back it
> > > > its original value.
> > > 
> > > 
> > > There's a bunch of that through ancient and rotten parts of the kernel.
> > > All those sites are broken.
> > > 
> > > Nothing stops userspace from setting a different affinity right after
> > > the kernel does for those threads.
> > 
> > Good. So you are saying I should convert them to something like
> > queue_work_on()?
> 
> Not sure; iirc there were a few variants. Some can indeed simply do
> queue_work_on() and possibly wait for completion. some should maybe be a
> per-cpu kthread, others will be more 'interesting'.
> 
> IIRC MIPS has a case where only 1 in N cores has an FPU. And once a task
> uses FPU, it gets affined to the core that has one or something like
> that.
>
> Of course, nothing then stops someone else breaking that affinity. But I
> suspect it will simply fault on the next FPU instruction and 'reset' the
> mask or something. I've no clue and no real desire to know.

It does nasty games with it's own storage of p->thread.user_cpus_allowed
and a fully seperate implementation of sys_sched_set|getaffinity.

Plus a magic trap handler which forces the thread to a CPU with FPU when
the user_cpus_allowed mask intersects with the cpus_with_fpu_mask...

Magic crap, which could all be replaced by a simple function in the
scheduler which allows to push a task to a FPU CPU and then disable
migration.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1617871 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

On Thu, Apr 06, 2017 at 12:47:21PM +0200, Thomas Gleixner wrote:
> On Thu, 6 Apr 2017, Peter Zijlstra wrote:

> > IIRC MIPS has a case where only 1 in N cores has an FPU. And once a task
> > uses FPU, it gets affined to the core that has one or something like
> > that.
> >
> > Of course, nothing then stops someone else breaking that affinity. But I
> > suspect it will simply fault on the next FPU instruction and 'reset' the
> > mask or something. I've no clue and no real desire to know.
> 
> It does nasty games with it's own storage of p->thread.user_cpus_allowed
> and a fully seperate implementation of sys_sched_set|getaffinity.
> 
> Plus a magic trap handler which forces the thread to a CPU with FPU when
> the user_cpus_allowed mask intersects with the cpus_with_fpu_mask...
> 
> Magic crap, which could all be replaced by a simple function in the
> scheduler which allows to push a task to a FPU CPU and then disable
> migration.

If its even halfway coherent, I'd much rather let it stay where it it.

I really want to limit migrate_disable() to PREEMPT_RT=y where its used
to preserve spinlock semantics and not allow random other
migrate_disable() usage in the kernel.

Also note, that per the above, it can actually migrate to any core that
has an FPU on, so its not a good match for migrate_disable() in any
case.

[toc] | [prev] | [next] | [standalone]

#1617879 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

On Thu, 6 Apr 2017, Peter Zijlstra wrote:
> On Thu, Apr 06, 2017 at 12:47:21PM +0200, Thomas Gleixner wrote:
> > On Thu, 6 Apr 2017, Peter Zijlstra wrote:
> 
> > > IIRC MIPS has a case where only 1 in N cores has an FPU. And once a task
> > > uses FPU, it gets affined to the core that has one or something like
> > > that.
> > >
> > > Of course, nothing then stops someone else breaking that affinity. But I
> > > suspect it will simply fault on the next FPU instruction and 'reset' the
> > > mask or something. I've no clue and no real desire to know.
> > 
> > It does nasty games with it's own storage of p->thread.user_cpus_allowed
> > and a fully seperate implementation of sys_sched_set|getaffinity.
> > 
> > Plus a magic trap handler which forces the thread to a CPU with FPU when
> > the user_cpus_allowed mask intersects with the cpus_with_fpu_mask...
> > 
> > Magic crap, which could all be replaced by a simple function in the
> > scheduler which allows to push a task to a FPU CPU and then disable
> > migration.
> 
> If its even halfway coherent, I'd much rather let it stay where it it.
> 
> I really want to limit migrate_disable() to PREEMPT_RT=y where its used
> to preserve spinlock semantics and not allow random other
> migrate_disable() usage in the kernel.
> 
> Also note, that per the above, it can actually migrate to any core that
> has an FPU on, so its not a good match for migrate_disable() in any
> case.

Fair enough, but I prefer to have the ability for a temporary restriction
of the user space visible cpus allowed mask in the core code, which can be
used for both migrate_disable() and things like that MIPS FPU stuff rather
than all those home brewn hackeries which are prone to bitrot, security
issues and subtle wreckage. cpus_allowed should be solely under sched core
control and not accessible from anything outside.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1617902 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

On Thu, Apr 06, 2017 at 01:03:16PM +0200, Thomas Gleixner wrote:

> Fair enough, but I prefer to have the ability for a temporary restriction
> of the user space visible cpus allowed mask in the core code, which can be
> used for both migrate_disable() and things like that MIPS FPU stuff rather
> than all those home brewn hackeries which are prone to bitrot, security
> issues and subtle wreckage. cpus_allowed should be solely under sched core
> control and not accessible from anything outside.

So the big difference between what MIPS does and what migrate_disable()
needs here is the amount of storage.

For migrate_disable() we don't need a whole second cpumask bitmap. We
can simply use one of the static cpumask_of() things.

Given that cpumask bitmaps can be quite large, adding another one to
task_struct just for $very_rare_occasion, doesn't seem like a good
thing.

[toc] | [prev] | [next] | [standalone]

#1617904 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

On Thu, 6 Apr 2017, Peter Zijlstra wrote:
> On Thu, Apr 06, 2017 at 01:03:16PM +0200, Thomas Gleixner wrote:
> 
> > Fair enough, but I prefer to have the ability for a temporary restriction
> > of the user space visible cpus allowed mask in the core code, which can be
> > used for both migrate_disable() and things like that MIPS FPU stuff rather
> > than all those home brewn hackeries which are prone to bitrot, security
> > issues and subtle wreckage. cpus_allowed should be solely under sched core
> > control and not accessible from anything outside.
> 
> So the big difference between what MIPS does and what migrate_disable()
> needs here is the amount of storage.
> 
> For migrate_disable() we don't need a whole second cpumask bitmap. We
> can simply use one of the static cpumask_of() things.
> 
> Given that cpumask bitmaps can be quite large, adding another one to
> task_struct just for $very_rare_occasion, doesn't seem like a good
> thing.

A pointer to the effective mask is definitely enough. And that's what we
need for migrate_disable() as well. That still leaves the storage
requirement to MIPS.

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1617959 — Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask

On Thu, Apr 06, 2017 at 01:56:12PM +0200, Thomas Gleixner wrote:

> A pointer to the effective mask is definitely enough. And that's what we
> need for migrate_disable() as well. That still leaves the storage
> requirement to MIPS.

So then you're back to Ingo's proposal, right?

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web