Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1344561 > unrolled thread
| Started by | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| First post | 2016-02-26 19:50 +0100 |
| Last post | 2016-02-29 21:10 +0100 |
| Articles | 20 on this page of 55 — 8 participants |
Back to article view | Back to linux.kernel
[patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[patch 14/20] cpu/hotplug: Split out the state walk into functions Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Split out the state walk into functions tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 17/20] arch/hotplug: Call into idle with a proper state Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] arch/hotplug: Call into idle with a proper state tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 04/20] cpu/hotplug: Split out cpu down functions Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Split out cpu down functions tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Split out cpu down functions "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> - 2016-03-03 00:10 +0100
[patch 03/20] cpu/hotplug: Restructure cpu_up code Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> - 2016-03-03 00:00 +0100
[patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Move online calls to hotplugged cpu tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 02/20] cpu/hotplug: Restructure FROZEN state handling Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> - 2016-03-03 00:10 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> - 2016-03-03 00:50 +0100
[patch 11/20] cpu/hotplug: Implement setup/removal interface Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Implement setup/removal interface tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Let upcoming cpu bring itself fully up tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Richard Cochran <richardcochran@gmail.com> - 2016-03-02 18:30 +0100
[patch 15/20] cpu/hotplug: Create hotplug threads Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Create hotplug threads tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 09/20] cpu/hotplug: Add sysfs state interface Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Add sysfs state interface tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Add sysfs state interface Peter Zijlstra <peterz@infradead.org> - 2016-03-02 13:50 +0100
[patch 06/20] cpu/hotplug: Convert to a state machine for the control processor Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Convert to a state machine for the control processor tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Convert to a state machine for the control processor Peter Zijlstra <peterz@infradead.org> - 2016-03-02 12:30 +0100
[patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-02-27 03:20 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-02-27 03:30 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner <tglx@linutronix.de> - 2016-02-27 08:50 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-02-27 12:10 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner <tglx@linutronix.de> - 2016-02-27 12:40 +0100
Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-02-27 17:40 +0100
[tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
Re: [tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-03-02 21:20 +0100
Re: [tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner <tglx@linutronix.de> - 2016-03-03 08:40 +0100
[tip:smp/hotplug] cpu/hotplug: Plug death reporting race tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-03 11:10 +0100
Re: [tip:smp/hotplug] cpu/hotplug: Plug death reporting race "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-03-03 15:20 +0100
[patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Hand in target state to _cpu_up/down tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 19/20] cpu/hotplug: Make wait for dead cpu completion based Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:50 +0100
[tip:smp/hotplug] cpu/hotplug: Make wait for dead cpu completion based tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 20:00 +0100
[tip:smp/hotplug] cpu/hotplug: Convert the hotplugged cpu work to a state machine tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2016-03-01 21:00 +0100
[patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 20:00 +0100
Re: [patch 01/20] idle: Move x86ism out of generic code Brian Gerst <brgerst@gmail.com> - 2016-02-27 21:30 +0100
Re: [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner <tglx@linutronix.de> - 2016-02-29 20:40 +0100
Re: [patch 01/20] idle: Move x86ism out of generic code Will Deacon <will.deacon@arm.com> - 2016-02-29 20:50 +0100
Re: [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner <tglx@linutronix.de> - 2016-02-29 21:10 +0100
Page 1 of 3 [1] 2 3 Next page →
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework |
| Message-ID | <r6vCG-7sv-3@gated-at.bofh.it> |
Hi folks!
the following series contains the core infrastructure of our ongoing cpu
hotplug refactoring work. It's a follow up to the work I've done in 2013
https://lwn.net/Articles/535764/
What's wrong with the current cpu hotplug infrastructure?
- Asymmetry
The hotplug notifier mechanism is asymmetric versus the bringup and
teardown. This is mostly caused by the notifier mechanism.
- Largely undocumented dependencies
While some notifiers use explicitely defined notifier priorities, we have
quite some notifiers which use numerical priorities to express
dependencies without any documentation why.
- Control processor driven
Most of the bringup/teardown of a cpu is driven by a control
processor. While it is understandable, that preperatory steps, like idle
thread creation, memory allocation for and initialization of essential
facilities needs to be done before a cpu can boot, there is no reason why
everything else must run on a control processor. Todays bringup looks like
this:
Control CPU Booting CPU
do preparatory steps
kick cpu into life
do low level init
sync with booting cpu sync with control cpu
bring the rest up
- All or nothing approach
There is no way to do partial bringups. That's something which is really
desired because we waste e.g. at boot substantial amount of time just busy
waiting that the cpu comes to life. That's stupid as we could very well do
preparatory steps and the initial IPI for other cpus and then go back and
do the necessary low level synchronization with the freshly booted cpu.
- Minimal debuggability
Due to the notifier based design, it's impossible to switch between two
stages of the bringup/teardown back and forth in order to test the
correctness. So in many hotplug notifiers the cancel mechanisms are either
not existant or completely untested.
- Notifier [un]registering is tidious
To [un]register notifiers we need to protect against hotplug at every
callsite. There is no mechanism that bringup/teardown callbacks are issued
on the online cpus, so every caller needs to do it itself. That also
includes error rollback.
What's the new design?
The base of the new design is a symmetric state machine, where both the
control processor and the booting/dying cpu execute a well defined set of
states. Each state is symmetric in the end, except for some well defined
exceptions, and the bringup/teardown can be stopped and reversed at almost
all states.
So the bringup of a cpu will look like this in the future:
Control CPU Booting CPU
do preparatory steps
kick cpu into life
do low level init
sync with booting cpu sync with control cpu
bring itself up
The synchronization step does not require the control cpu to wait. That
mechanism can be done asynchronously via a worker or some other mechanism.
The teardown can be made very similar, so that the dying cpu cleans up and
brings itself down. Cleanups which need to be done after the cpu is gone,
can be scheduled asynchronously as well.
There is a long way to this, as we need to refactor the notion when a cpu is
available. Today we set the cpu online right after it comes out of the low
level bringup, which is not really correct.
The proper mechanism is to set it to available, i.e. cpu local threads, like
softirqd, hotplug thread etc. can be scheduled on that cpu, and once it
finished all booting steps, it's set to online, so general workloads can be
scheduled on it. The reverse happens on teardown. First thing to do is to
forbid scheduling of general workloads, then teardown all the per cpu
resources and finally shut it off completely.
The following patch series implements the basic infrastructure for this at the
core level. This includes the following:
- Basic state machine implementation with well defined states, so ordering
and prioritization can be expressed.
- Interfaces to [un]register state callbacks
This invokes the bringup/teardown callback on all online cpus with the
proper protection in place and [un]installs the callbacks in the state
machine array.
For callbacks which have no particular ordering requirement we have a
dynamic state space, so that drivers don't have to register an explicit
hotplug state.
If a callback fails, the code automatically does a rollback to the previous
state.
- Sysfs interface to drive the state machine to a particular step.
This is only partially functional today. Full functionality and therefor
testability will be achieved once we converted all existing hotplug
notifiers over to the new scheme.
- Run all CPU_ONLINE/DOWN_PREPARE notifiers on the booting/dying processor:
Control CPU Booting CPU
do preparatory steps
kick cpu into life
do low level init
sync with booting cpu sync with control cpu
wait for boot
bring itself up
Signal completion to control cpu
In a previous step of this work we've done a full tree mechanical conversion
of all hotplug notifiers to the new scheme. The balance is a net removal of
about 4000 lines of code.
This is not included in this post, as we decided to take a different
approach. Instead of mechanically converting everything over, we will do a
proper overhaul of the usage sites one by one so they nicely fit into the
symmetric callback scheme.
I decided to do that after I looked at the ugliness of some of the converted
sites and figured out that their hotplug mechanism is completely buggered
anyway. So there is no point to do a mechanical conversion first as we need to
go through the usage sites one by one again in order to achieve a full
symmetric and testable behaviour.
The lot is also available at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.hotplug
Thanks,
tglx
---
arch/ia64/kernel/smpboot.c | 2
arch/metag/kernel/smp.c | 2
arch/sparc/kernel/smp_32.c | 2
arch/sparc/kernel/smp_64.c | 2
arch/tile/kernel/smpboot.c | 2
arch/x86/kernel/process.c | 12
arch/x86/xen/smp.c | 2
b/arch/alpha/kernel/smp.c | 2
b/arch/arc/kernel/smp.c | 2
b/arch/arm/kernel/smp.c | 2
b/arch/arm64/kernel/smp.c | 2
b/arch/blackfin/mach-common/smp.c | 2
b/arch/hexagon/kernel/smp.c | 2
b/arch/m32r/kernel/smpboot.c | 2
b/arch/mips/kernel/smp.c | 2
b/arch/mn10300/kernel/smp.c | 2
b/arch/parisc/kernel/smp.c | 2
b/arch/powerpc/kernel/smp.c | 2
b/arch/s390/kernel/smp.c | 2
b/arch/sh/kernel/smp.c | 2
b/arch/x86/kernel/smpboot.c | 2
b/arch/xtensa/kernel/smp.c | 2
b/include/linux/cpuhotplug.h | 93 +++
b/include/trace/events/cpuhp.h | 66 ++
include/linux/cpu.h | 27
include/linux/notifier.h | 2
include/linux/rcupdate.h | 4
init/main.c | 16
kernel/cpu.c | 1099 +++++++++++++++++++++++++++++++++-----
kernel/rcu/tree.c | 26
kernel/sched/core.c | 10
kernel/sched/idle.c | 24
kernel/smp.c | 1
kernel/smpboot.c | 6
kernel/smpboot.h | 6
lib/Kconfig.debug | 13
36 files changed, 1217 insertions(+), 230 deletions(-)
[toc] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 14/20] cpu/hotplug: Split out the state walk into functions |
| Message-ID | <r6vCG-7sv-13@gated-at.bofh.it> |
| In reply to | #1344561 |
We need that for running callbacks on the AP and the BP.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 111 ++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 68 insertions(+), 43 deletions(-)
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
return 0;
}
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps)
+{
+ for (st->state++; st->state < st->target; st->state++) {
+ struct cpuhp_step *step = steps + st->state;
+
+ if (!step->skip_onerr)
+ cpuhp_invoke_callback(cpu, st->state, step->startup);
+ }
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps, enum cpuhp_state target)
+{
+ enum cpuhp_state prev_state = st->state;
+ int ret = 0;
+
+ for (; st->state > target; st->state--) {
+ struct cpuhp_step *step = steps + st->state;
+
+ ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+ if (ret) {
+ st->target = prev_state;
+ undo_cpu_down(cpu, st, steps);
+ break;
+ }
+ }
+ return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps)
+{
+ for (st->state--; st->state > st->target; st->state--) {
+ struct cpuhp_step *step = steps + st->state;
+
+ if (!step->skip_onerr)
+ cpuhp_invoke_callback(cpu, st->state, step->teardown);
+ }
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps, enum cpuhp_state target)
+{
+ enum cpuhp_state prev_state = st->state;
+ int ret = 0;
+
+ while (st->state < target) {
+ struct cpuhp_step *step;
+
+ st->state++;
+ step = steps + st->state;
+ ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+ if (ret) {
+ st->target = prev_state;
+ undo_cpu_up(cpu, st, steps);
+ break;
+ }
+ }
+ return ret;
+}
+
#ifdef CONFIG_HOTPLUG_CPU
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
-
void unregister_cpu_notifier(struct notifier_block *nb)
{
cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
#endif
#ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
- for (st->state++; st->state < st->target; st->state++) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
- if (!step->skip_onerr)
- cpuhp_invoke_callback(cpu, st->state, step->startup);
- }
-}
/* Requires cpu_add_remove_lock to be held */
static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int
prev_state = st->state;
st->target = target;
- for (; st->state > st->target; st->state--) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
+ ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
- ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
- if (ret) {
- st->target = prev_state;
- undo_cpu_down(cpu, st);
- break;
- }
- }
hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned
return 0;
}
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
- for (st->state--; st->state > st->target; st->state--) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
- if (!step->skip_onerr)
- cpuhp_invoke_callback(cpu, st->state, step->teardown);
- }
-}
-
/* Requires cpu_add_remove_lock to be held */
static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
struct task_struct *idle;
- int prev_state, ret = 0;
+ int ret = 0;
cpu_hotplug_begin();
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int
cpuhp_tasks_frozen = tasks_frozen;
- prev_state = st->state;
st->target = target;
- while (st->state < st->target) {
- struct cpuhp_step *step;
-
- st->state++;
- step = cpuhp_bp_states + st->state;
- ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
- if (ret) {
- st->target = prev_state;
- undo_cpu_up(cpu, st);
- break;
- }
- }
+ ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
out:
cpu_hotplug_done();
return ret;
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Split out the state walk into functions |
| Message-ID | <r7YCC-5hG-5@gated-at.bofh.it> |
| In reply to | #1344562 |
Commit-ID: 2e1a3483ce74d197876e58281925dd4e378a7f28
Gitweb: http://git.kernel.org/tip/2e1a3483ce74d197876e58281925dd4e378a7f28
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:37 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:56 +0100
cpu/hotplug: Split out the state walk into functions
We need that for running callbacks on the AP and the BP.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.374946234@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 111 ++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 68 insertions(+), 43 deletions(-)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3ec86bc..9572ca0 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
return 0;
}
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps)
+{
+ for (st->state++; st->state < st->target; st->state++) {
+ struct cpuhp_step *step = steps + st->state;
+
+ if (!step->skip_onerr)
+ cpuhp_invoke_callback(cpu, st->state, step->startup);
+ }
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps, enum cpuhp_state target)
+{
+ enum cpuhp_state prev_state = st->state;
+ int ret = 0;
+
+ for (; st->state > target; st->state--) {
+ struct cpuhp_step *step = steps + st->state;
+
+ ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+ if (ret) {
+ st->target = prev_state;
+ undo_cpu_down(cpu, st, steps);
+ break;
+ }
+ }
+ return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps)
+{
+ for (st->state--; st->state > st->target; st->state--) {
+ struct cpuhp_step *step = steps + st->state;
+
+ if (!step->skip_onerr)
+ cpuhp_invoke_callback(cpu, st->state, step->teardown);
+ }
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+ struct cpuhp_step *steps, enum cpuhp_state target)
+{
+ enum cpuhp_state prev_state = st->state;
+ int ret = 0;
+
+ while (st->state < target) {
+ struct cpuhp_step *step;
+
+ st->state++;
+ step = steps + st->state;
+ ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+ if (ret) {
+ st->target = prev_state;
+ undo_cpu_up(cpu, st, steps);
+ break;
+ }
+ }
+ return ret;
+}
+
#ifdef CONFIG_HOTPLUG_CPU
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
-
void unregister_cpu_notifier(struct notifier_block *nb)
{
cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
#endif
#ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
- for (st->state++; st->state < st->target; st->state++) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
- if (!step->skip_onerr)
- cpuhp_invoke_callback(cpu, st->state, step->startup);
- }
-}
/* Requires cpu_add_remove_lock to be held */
static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
prev_state = st->state;
st->target = target;
- for (; st->state > st->target; st->state--) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
+ ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
- ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
- if (ret) {
- st->target = prev_state;
- undo_cpu_down(cpu, st);
- break;
- }
- }
hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned int cpu)
return 0;
}
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
- for (st->state--; st->state > st->target; st->state--) {
- struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
- if (!step->skip_onerr)
- cpuhp_invoke_callback(cpu, st->state, step->teardown);
- }
-}
-
/* Requires cpu_add_remove_lock to be held */
static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
struct task_struct *idle;
- int prev_state, ret = 0;
+ int ret = 0;
cpu_hotplug_begin();
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
cpuhp_tasks_frozen = tasks_frozen;
- prev_state = st->state;
st->target = target;
- while (st->state < st->target) {
- struct cpuhp_step *step;
-
- st->state++;
- step = cpuhp_bp_states + st->state;
- ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
- if (ret) {
- st->target = prev_state;
- undo_cpu_up(cpu, st);
- break;
- }
- }
+ ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
out:
cpu_hotplug_done();
return ret;
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 17/20] arch/hotplug: Call into idle with a proper state |
| Message-ID | <r6vCG-7sv-15@gated-at.bofh.it> |
| In reply to | #1344561 |
Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/alpha/kernel/smp.c | 2 +-
arch/arc/kernel/smp.c | 2 +-
arch/arm/kernel/smp.c | 2 +-
arch/arm64/kernel/smp.c | 2 +-
arch/blackfin/mach-common/smp.c | 2 +-
arch/hexagon/kernel/smp.c | 2 +-
arch/ia64/kernel/smpboot.c | 2 +-
arch/m32r/kernel/smpboot.c | 2 +-
arch/metag/kernel/smp.c | 2 +-
arch/mips/kernel/smp.c | 2 +-
arch/mn10300/kernel/smp.c | 2 +-
arch/parisc/kernel/smp.c | 2 +-
arch/powerpc/kernel/smp.c | 2 +-
arch/s390/kernel/smp.c | 2 +-
arch/sh/kernel/smp.c | 2 +-
arch/sparc/kernel/smp_32.c | 2 +-
arch/sparc/kernel/smp_64.c | 2 +-
arch/tile/kernel/smpboot.c | 2 +-
arch/x86/kernel/smpboot.c | 2 +-
arch/x86/xen/smp.c | 2 +-
arch/xtensa/kernel/smp.c | 2 +-
include/linux/cpuhotplug.h | 1 +
22 files changed, 22 insertions(+), 21 deletions(-)
Index: b/arch/alpha/kernel/smp.c
===================================================================
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
cpuid, current, current->active_mm));
preempt_disable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/* Wait until hwrpb->txrdy is clear for cpu. Return -1 on timeout. */
Index: b/arch/arc/kernel/smp.c
===================================================================
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
local_irq_enable();
preempt_disable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/*
Index: b/arch/arm/kernel/smp.c
===================================================================
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(v
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/arm64/kernel/smp.c
===================================================================
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(v
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
#ifdef CONFIG_HOTPLUG_CPU
Index: b/arch/blackfin/mach-common/smp.c
===================================================================
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
/* We are done with local CPU inits, unblock the boot CPU. */
set_cpu_online(cpu, true);
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_prepare_boot_cpu(void)
Index: b/arch/hexagon/kernel/smp.c
===================================================================
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
Index: b/arch/ia64/kernel/smpboot.c
===================================================================
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
preempt_disable();
smp_callin();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
Index: b/arch/m32r/kernel/smpboot.c
===================================================================
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
*/
local_flush_tlb_all();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
Index: b/arch/metag/kernel/smp.c
===================================================================
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(v
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/mips/kernel/smp.c
===================================================================
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
WARN_ON_ONCE(!irqs_disabled());
mp_ops->smp_finish();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void stop_this_cpu(void *dummy)
Index: b/arch/mn10300/kernel/smp.c
===================================================================
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
#ifdef CONFIG_GENERIC_CLOCKEVENTS
init_clockevents();
#endif
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
Index: b/arch/parisc/kernel/smp.c
===================================================================
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
local_irq_enable(); /* Interrupts have been off until now */
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
/* NOTREACHED */
panic("smp_callin() AAAAaaaaahhhh....\n");
Index: b/arch/powerpc/kernel/smp.c
===================================================================
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
BUG();
}
Index: b/arch/s390/kernel/smp.c
===================================================================
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cp
set_cpu_online(smp_processor_id(), true);
inc_irq_stat(CPU_RST);
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/* Upping and downing of CPUs */
Index: b/arch/sh/kernel/smp.c
===================================================================
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
set_cpu_online(cpu, true);
per_cpu(cpu_state, cpu) = CPU_ONLINE;
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
extern struct {
Index: b/arch/sparc/kernel/smp_32.c
===================================================================
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *
local_irq_enable();
wmb();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
/* We should never reach here! */
BUG();
Index: b/arch/sparc/kernel/smp_64.c
===================================================================
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void cpu_panic(void)
Index: b/arch/tile/kernel/smpboot.c
===================================================================
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
/* Set up tile-timer clock-event device on this cpu */
setup_tile_timer();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
int __cpu_up(unsigned int cpu, struct task_struct *tidle)
Index: b/arch/x86/kernel/smpboot.c
===================================================================
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void
x86_cpuinit.setup_percpu_clockev();
wmb();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_store_boot_cpu_info(void)
Index: b/arch/x86/xen/smp.c
===================================================================
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_an
xen_pvh_secondary_vcpu_init(cpu);
#endif
cpu_bringup();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void xen_smp_intr_free(unsigned int cpu)
Index: b/arch/xtensa/kernel/smp.c
===================================================================
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
complete(&cpu_running);
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void mx_cpu_start(void *p)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
CPUHP_CPU_SET_ACTIVE,
CPUHP_KICK_AP_THREAD,
CPUHP_BP_ONLINE,
+ CPUHP_AP_ONLINE_IDLE,
CPUHP_AP_SMPBOOT_THREADS,
CPUHP_AP_NOTIFY_ONLINE,
CPUHP_AP_ONLINE_DYN,
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] arch/hotplug: Call into idle with a proper state |
| Message-ID | <r7YCC-5hG-3@gated-at.bofh.it> |
| In reply to | #1344563 |
Commit-ID: fc6d73d67436e7784758a831227bd019547a3f73
Gitweb: http://git.kernel.org/tip/fc6d73d67436e7784758a831227bd019547a3f73
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:40 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:57 +0100
arch/hotplug: Call into idle with a proper state
Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.614102639@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/alpha/kernel/smp.c | 2 +-
arch/arc/kernel/smp.c | 2 +-
arch/arm/kernel/smp.c | 2 +-
arch/arm64/kernel/smp.c | 2 +-
arch/blackfin/mach-common/smp.c | 2 +-
arch/hexagon/kernel/smp.c | 2 +-
arch/ia64/kernel/smpboot.c | 2 +-
arch/m32r/kernel/smpboot.c | 2 +-
arch/metag/kernel/smp.c | 2 +-
arch/mips/kernel/smp.c | 2 +-
arch/mn10300/kernel/smp.c | 2 +-
arch/parisc/kernel/smp.c | 2 +-
arch/powerpc/kernel/smp.c | 2 +-
arch/s390/kernel/smp.c | 2 +-
arch/sh/kernel/smp.c | 2 +-
arch/sparc/kernel/smp_32.c | 2 +-
arch/sparc/kernel/smp_64.c | 2 +-
arch/tile/kernel/smpboot.c | 2 +-
arch/x86/kernel/smpboot.c | 2 +-
arch/x86/xen/smp.c | 2 +-
arch/xtensa/kernel/smp.c | 2 +-
include/linux/cpuhotplug.h | 1 +
22 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index 2f24447f..46bf263 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
cpuid, current, current->active_mm));
preempt_disable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/* Wait until hwrpb->txrdy is clear for cpu. Return -1 on timeout. */
diff --git a/arch/arc/kernel/smp.c b/arch/arc/kernel/smp.c
index 424e937..4cb3add 100644
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
local_irq_enable();
preempt_disable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/*
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 37312f6..baee702 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(void)
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_cpus_done(unsigned int max_cpus)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index b1adc51..4607657 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(void)
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
#ifdef CONFIG_HOTPLUG_CPU
diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c
index 0030e21..23c4ef5 100644
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
/* We are done with local CPU inits, unblock the boot CPU. */
set_cpu_online(cpu, true);
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_prepare_boot_cpu(void)
diff --git a/arch/hexagon/kernel/smp.c b/arch/hexagon/kernel/smp.c
index ff759f2..983bae7 100644
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
diff --git a/arch/ia64/kernel/smpboot.c b/arch/ia64/kernel/smpboot.c
index 0e76fad..74fe317 100644
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
preempt_disable();
smp_callin();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
diff --git a/arch/m32r/kernel/smpboot.c b/arch/m32r/kernel/smpboot.c
index a468467..f98d2f6 100644
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
*/
local_flush_tlb_all();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
diff --git a/arch/metag/kernel/smp.c b/arch/metag/kernel/smp.c
index c3c6f08..bad1323 100644
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(void)
/*
* OK, it's off to the idle thread for us
*/
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_cpus_done(unsigned int max_cpus)
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index bd4385a..f2112a8 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
WARN_ON_ONCE(!irqs_disabled());
mp_ops->smp_finish();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void stop_this_cpu(void *dummy)
diff --git a/arch/mn10300/kernel/smp.c b/arch/mn10300/kernel/smp.c
index f984193..426173c 100644
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
#ifdef CONFIG_GENERIC_CLOCKEVENTS
init_clockevents();
#endif
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
return 0;
}
diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c
index 52e8597..c2a9cc5 100644
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
local_irq_enable(); /* Interrupts have been off until now */
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
/* NOTREACHED */
panic("smp_callin() AAAAaaaaahhhh....\n");
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index ec9ec20..cc13d4c 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
BUG();
}
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 3c65a8e..40a6b4f 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cpuvoid)
set_cpu_online(smp_processor_id(), true);
inc_irq_stat(CPU_RST);
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
/* Upping and downing of CPUs */
diff --git a/arch/sh/kernel/smp.c b/arch/sh/kernel/smp.c
index de6be00..13f633a 100644
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
set_cpu_online(cpu, true);
per_cpu(cpu_state, cpu) = CPU_ONLINE;
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
extern struct {
diff --git a/arch/sparc/kernel/smp_32.c b/arch/sparc/kernel/smp_32.c
index b3a5d81..fb30e7c 100644
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *arg)
local_irq_enable();
wmb();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
/* We should never reach here! */
BUG();
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 19cd08d..8a6151a 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
local_irq_enable();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void cpu_panic(void)
diff --git a/arch/tile/kernel/smpboot.c b/arch/tile/kernel/smpboot.c
index 20d52a9..6c0abaa 100644
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
/* Set up tile-timer clock-event device on this cpu */
setup_tile_timer();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
int __cpu_up(unsigned int cpu, struct task_struct *tidle)
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 24d57f7..293b22a 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void *unused)
x86_cpuinit.setup_percpu_clockev();
wmb();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
void __init smp_store_boot_cpu_info(void)
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 3f4ebf0..3c6d17f 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_and_idle(int cpu)
xen_pvh_secondary_vcpu_init(cpu);
#endif
cpu_bringup();
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void xen_smp_intr_free(unsigned int cpu)
diff --git a/arch/xtensa/kernel/smp.c b/arch/xtensa/kernel/smp.c
index 4d02e38..fc4ad21 100644
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
complete(&cpu_running);
- cpu_startup_entry(CPUHP_ONLINE);
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
static void mx_cpu_start(void *p)
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 8a715bb..4aa263a 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
CPUHP_CPU_SET_ACTIVE,
CPUHP_KICK_AP_THREAD,
CPUHP_BP_ONLINE,
+ CPUHP_AP_ONLINE_IDLE,
CPUHP_AP_SMPBOOT_THREADS,
CPUHP_AP_NOTIFY_ONLINE,
CPUHP_AP_ONLINE_DYN,
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 04/20] cpu/hotplug: Split out cpu down functions |
| Message-ID | <r6vCG-7sv-17@gated-at.bofh.it> |
| In reply to | #1344561 |
Split cpu_down in separate functions in preparation for state machine
conversion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 83 +++++++++++++++++++++++++++++++++++++----------------------
1 file changed, 53 insertions(+), 30 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
}
#ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
- BUG_ON(cpu_notify(val, cpu));
-}
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int d
read_unlock(&tasklist_lock);
}
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+ BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+ int err, nr_calls = 0;
+
+ err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+ if (err) {
+ nr_calls--;
+ __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+ pr_warn("%s: attempt to take down CPU %u failed\n",
+ __func__, cpu);
+ }
+ return err;
+}
+
/* Take this CPU down. */
static int take_cpu_down(void *_param)
{
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
return 0;
}
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
{
- int err, nr_calls = 0;
-
- if (num_online_cpus() == 1)
- return -EBUSY;
-
- if (!cpu_online(cpu))
- return -EINVAL;
-
- cpu_hotplug_begin();
-
- cpuhp_tasks_frozen = tasks_frozen;
-
- err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
- if (err) {
- nr_calls--;
- __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
- pr_warn("%s: attempt to take down CPU %u failed\n",
- __func__, cpu);
- goto out_release;
- }
+ int err;
/*
* By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, i
/* CPU didn't die: tell everyone. Can't complain. */
cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
irq_unlock_sparse();
- goto out_release;
+ return err;
}
BUG_ON(cpu_online(cpu));
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, i
/* This actually kills the CPU. */
__cpu_die(cpu);
- /* CPU is completely dead: tell everyone. Too late to complain. */
tick_cleanup_dead_cpu(cpu);
- cpu_notify_nofail(CPU_DEAD, cpu);
+ return 0;
+}
+static int notify_dead(unsigned int cpu)
+{
+ cpu_notify_nofail(CPU_DEAD, cpu);
check_for_tasks(cpu);
+ return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+ int err;
+
+ if (num_online_cpus() == 1)
+ return -EBUSY;
+
+ if (!cpu_online(cpu))
+ return -EINVAL;
+
+ cpu_hotplug_begin();
+
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ err = notify_down_prepare(cpu);
+ if (err)
+ goto out_release;
+ err = takedown_cpu(cpu);
+ if (err)
+ goto out_release;
+
+ notify_dead(cpu);
out_release:
cpu_hotplug_done();
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Split out cpu down functions |
| Message-ID | <r7YCC-5hG-17@gated-at.bofh.it> |
| In reply to | #1344564 |
Commit-ID: 984581728eb4b2e10baed3d606f85a119795b207
Gitweb: http://git.kernel.org/tip/984581728eb4b2e10baed3d606f85a119795b207
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:25 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
cpu/hotplug: Split out cpu down functions
Split cpu_down in separate functions in preparation for state machine
conversion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.511796562@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 83 ++++++++++++++++++++++++++++++++++++++----------------------
1 file changed, 53 insertions(+), 30 deletions(-)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 15a4136..0b5d259 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
}
#ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
- BUG_ON(cpu_notify(val, cpu));
-}
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int dead_cpu)
read_unlock(&tasklist_lock);
}
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+ BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+ int err, nr_calls = 0;
+
+ err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+ if (err) {
+ nr_calls--;
+ __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+ pr_warn("%s: attempt to take down CPU %u failed\n",
+ __func__, cpu);
+ }
+ return err;
+}
+
/* Take this CPU down. */
static int take_cpu_down(void *_param)
{
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
return 0;
}
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
{
- int err, nr_calls = 0;
-
- if (num_online_cpus() == 1)
- return -EBUSY;
-
- if (!cpu_online(cpu))
- return -EINVAL;
-
- cpu_hotplug_begin();
-
- cpuhp_tasks_frozen = tasks_frozen;
-
- err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
- if (err) {
- nr_calls--;
- __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
- pr_warn("%s: attempt to take down CPU %u failed\n",
- __func__, cpu);
- goto out_release;
- }
+ int err;
/*
* By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
/* CPU didn't die: tell everyone. Can't complain. */
cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
irq_unlock_sparse();
- goto out_release;
+ return err;
}
BUG_ON(cpu_online(cpu));
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
/* This actually kills the CPU. */
__cpu_die(cpu);
- /* CPU is completely dead: tell everyone. Too late to complain. */
tick_cleanup_dead_cpu(cpu);
- cpu_notify_nofail(CPU_DEAD, cpu);
+ return 0;
+}
+static int notify_dead(unsigned int cpu)
+{
+ cpu_notify_nofail(CPU_DEAD, cpu);
check_for_tasks(cpu);
+ return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+ int err;
+
+ if (num_online_cpus() == 1)
+ return -EBUSY;
+
+ if (!cpu_online(cpu))
+ return -EINVAL;
+
+ cpu_hotplug_begin();
+
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ err = notify_down_prepare(cpu);
+ if (err)
+ goto out_release;
+ err = takedown_cpu(cpu);
+ if (err)
+ goto out_release;
+
+ notify_dead(cpu);
out_release:
cpu_hotplug_done();
[toc] | [prev] | [next] | [standalone]
| From | "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> |
|---|---|
| Date | 2016-03-03 00:10 +0100 |
| Subject | Re: [tip:smp/hotplug] cpu/hotplug: Split out cpu down functions |
| Message-ID | <r8o41-6yA-1@gated-at.bofh.it> |
| In reply to | #1346975 |
On 3/1/16 2:52 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID: 984581728eb4b2e10baed3d606f85a119795b207
> Gitweb: http://git.kernel.org/tip/984581728eb4b2e10baed3d606f85a119795b207
> Author: Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:25 +0000
> Committer: Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
>
> cpu/hotplug: Split out cpu down functions
>
> Split cpu_down in separate functions in preparation for state machine
> conversion.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.511796562@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>
Regards,
Srivatsa S. Bhat
> kernel/cpu.c | 83 ++++++++++++++++++++++++++++++++++++++----------------------
> 1 file changed, 53 insertions(+), 30 deletions(-)
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 15a4136..0b5d259 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
> }
>
> #ifdef CONFIG_HOTPLUG_CPU
> -
> -static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> -{
> - BUG_ON(cpu_notify(val, cpu));
> -}
> EXPORT_SYMBOL(register_cpu_notifier);
> EXPORT_SYMBOL(__register_cpu_notifier);
>
> @@ -353,6 +348,25 @@ static inline void check_for_tasks(int dead_cpu)
> read_unlock(&tasklist_lock);
> }
>
> +static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> +{
> + BUG_ON(cpu_notify(val, cpu));
> +}
> +
> +static int notify_down_prepare(unsigned int cpu)
> +{
> + int err, nr_calls = 0;
> +
> + err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
> + if (err) {
> + nr_calls--;
> + __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
> + pr_warn("%s: attempt to take down CPU %u failed\n",
> + __func__, cpu);
> + }
> + return err;
> +}
> +
> /* Take this CPU down. */
> static int take_cpu_down(void *_param)
> {
> @@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
> return 0;
> }
>
> -/* Requires cpu_add_remove_lock to be held */
> -static int _cpu_down(unsigned int cpu, int tasks_frozen)
> +static int takedown_cpu(unsigned int cpu)
> {
> - int err, nr_calls = 0;
> -
> - if (num_online_cpus() == 1)
> - return -EBUSY;
> -
> - if (!cpu_online(cpu))
> - return -EINVAL;
> -
> - cpu_hotplug_begin();
> -
> - cpuhp_tasks_frozen = tasks_frozen;
> -
> - err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
> - if (err) {
> - nr_calls--;
> - __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
> - pr_warn("%s: attempt to take down CPU %u failed\n",
> - __func__, cpu);
> - goto out_release;
> - }
> + int err;
>
> /*
> * By now we've cleared cpu_active_mask, wait for all preempt-disabled
> @@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
> /* CPU didn't die: tell everyone. Can't complain. */
> cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
> irq_unlock_sparse();
> - goto out_release;
> + return err;
> }
> BUG_ON(cpu_online(cpu));
>
> @@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
> /* This actually kills the CPU. */
> __cpu_die(cpu);
>
> - /* CPU is completely dead: tell everyone. Too late to complain. */
> tick_cleanup_dead_cpu(cpu);
> - cpu_notify_nofail(CPU_DEAD, cpu);
> + return 0;
> +}
>
> +static int notify_dead(unsigned int cpu)
> +{
> + cpu_notify_nofail(CPU_DEAD, cpu);
> check_for_tasks(cpu);
> + return 0;
> +}
> +
> +/* Requires cpu_add_remove_lock to be held */
> +static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
> +{
> + int err;
> +
> + if (num_online_cpus() == 1)
> + return -EBUSY;
> +
> + if (!cpu_online(cpu))
> + return -EINVAL;
> +
> + cpu_hotplug_begin();
> +
> + cpuhp_tasks_frozen = tasks_frozen;
> +
> + err = notify_down_prepare(cpu);
> + if (err)
> + goto out_release;
> + err = takedown_cpu(cpu);
> + if (err)
> + goto out_release;
> +
> + notify_dead(cpu);
>
> out_release:
> cpu_hotplug_done();
>
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 03/20] cpu/hotplug: Restructure cpu_up code |
| Message-ID | <r6vCG-7sv-19@gated-at.bofh.it> |
| In reply to | #1344561 |
Split out into separate functions, so we can convert it to a state machine.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 69 ++++++++++++++++++++++++++++++++++++++++-------------------
1 file changed, 47 insertions(+), 22 deletions(-)
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val,
return __cpu_notify(val, cpu, -1, NULL);
}
+/* Notifier wrappers for transitioning to state machine */
+static int notify_prepare(unsigned int cpu)
+{
+ int nr_calls = 0;
+ int ret;
+
+ ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
+ if (ret) {
+ nr_calls--;
+ printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
+ __func__, cpu);
+ __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+ }
+ return ret;
+}
+
+static int notify_online(unsigned int cpu)
+{
+ cpu_notify(CPU_ONLINE, cpu);
+ return 0;
+}
+
+static int bringup_cpu(unsigned int cpu)
+{
+ struct task_struct *idle = idle_thread_get(cpu);
+ int ret;
+
+ /* Arch-specific enabling code. */
+ ret = __cpu_up(cpu, idle);
+ if (ret) {
+ cpu_notify(CPU_UP_CANCELED, cpu);
+ return ret;
+ }
+ BUG_ON(!cpu_online(cpu));
+ return 0;
+}
+
#ifdef CONFIG_HOTPLUG_CPU
static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
@@ -481,7 +518,7 @@ void smpboot_thread_init(void)
static int _cpu_up(unsigned int cpu, int tasks_frozen)
{
struct task_struct *idle;
- int ret, nr_calls = 0;
+ int ret;
cpu_hotplug_begin();
@@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int
goto out;
}
+ cpuhp_tasks_frozen = tasks_frozen;
+
ret = smpboot_create_threads(cpu);
if (ret)
goto out;
- cpuhp_tasks_frozen = tasks_frozen;
-
- ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
- if (ret) {
- nr_calls--;
- pr_warn("%s: attempt to bring up CPU %u failed\n",
- __func__, cpu);
- goto out_notify;
- }
-
- /* Arch-specific enabling code. */
- ret = __cpu_up(cpu, idle);
-
- if (ret != 0)
- goto out_notify;
- BUG_ON(!cpu_online(cpu));
+ ret = notify_prepare(cpu);
+ if (ret)
+ goto out;
- /* Now call notifier in preparation. */
- cpu_notify(CPU_ONLINE, cpu);
+ ret = bringup_cpu(cpu);
+ if (ret)
+ goto out;
-out_notify:
- if (ret != 0)
- __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+ notify_online(cpu);
out:
cpu_hotplug_done();
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code |
| Message-ID | <r7YCC-5hG-11@gated-at.bofh.it> |
| In reply to | #1344565 |
Commit-ID: ba997462435f48ad1501320e9da8770fd40c59b1
Gitweb: http://git.kernel.org/tip/ba997462435f48ad1501320e9da8770fd40c59b1
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:24 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
cpu/hotplug: Restructure cpu_up code
Split out into separate functions, so we can convert it to a state machine.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.429389195@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 69 +++++++++++++++++++++++++++++++++++++++++-------------------
1 file changed, 47 insertions(+), 22 deletions(-)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 41a6cb8..15a4136 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val, unsigned int cpu)
return __cpu_notify(val, cpu, -1, NULL);
}
+/* Notifier wrappers for transitioning to state machine */
+static int notify_prepare(unsigned int cpu)
+{
+ int nr_calls = 0;
+ int ret;
+
+ ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
+ if (ret) {
+ nr_calls--;
+ printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
+ __func__, cpu);
+ __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+ }
+ return ret;
+}
+
+static int notify_online(unsigned int cpu)
+{
+ cpu_notify(CPU_ONLINE, cpu);
+ return 0;
+}
+
+static int bringup_cpu(unsigned int cpu)
+{
+ struct task_struct *idle = idle_thread_get(cpu);
+ int ret;
+
+ /* Arch-specific enabling code. */
+ ret = __cpu_up(cpu, idle);
+ if (ret) {
+ cpu_notify(CPU_UP_CANCELED, cpu);
+ return ret;
+ }
+ BUG_ON(!cpu_online(cpu));
+ return 0;
+}
+
#ifdef CONFIG_HOTPLUG_CPU
static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
@@ -481,7 +518,7 @@ void smpboot_thread_init(void)
static int _cpu_up(unsigned int cpu, int tasks_frozen)
{
struct task_struct *idle;
- int ret, nr_calls = 0;
+ int ret;
cpu_hotplug_begin();
@@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
goto out;
}
+ cpuhp_tasks_frozen = tasks_frozen;
+
ret = smpboot_create_threads(cpu);
if (ret)
goto out;
- cpuhp_tasks_frozen = tasks_frozen;
-
- ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
- if (ret) {
- nr_calls--;
- pr_warn("%s: attempt to bring up CPU %u failed\n",
- __func__, cpu);
- goto out_notify;
- }
-
- /* Arch-specific enabling code. */
- ret = __cpu_up(cpu, idle);
-
- if (ret != 0)
- goto out_notify;
- BUG_ON(!cpu_online(cpu));
+ ret = notify_prepare(cpu);
+ if (ret)
+ goto out;
- /* Now call notifier in preparation. */
- cpu_notify(CPU_ONLINE, cpu);
+ ret = bringup_cpu(cpu);
+ if (ret)
+ goto out;
-out_notify:
- if (ret != 0)
- __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+ notify_online(cpu);
out:
cpu_hotplug_done();
[toc] | [prev] | [next] | [standalone]
| From | "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> |
|---|---|
| Date | 2016-03-03 00:00 +0100 |
| Subject | Re: [tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code |
| Message-ID | <r8nUm-6fn-1@gated-at.bofh.it> |
| In reply to | #1346969 |
On 3/1/16 2:52 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID: ba997462435f48ad1501320e9da8770fd40c59b1
> Gitweb: http://git.kernel.org/tip/ba997462435f48ad1501320e9da8770fd40c59b1
> Author: Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:24 +0000
> Committer: Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
>
> cpu/hotplug: Restructure cpu_up code
>
> Split out into separate functions, so we can convert it to a state machine.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.429389195@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>
Regards,
Srivatsa S. Bhat
> kernel/cpu.c | 69 +++++++++++++++++++++++++++++++++++++++++-------------------
> 1 file changed, 47 insertions(+), 22 deletions(-)
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 41a6cb8..15a4136 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val, unsigned int cpu)
> return __cpu_notify(val, cpu, -1, NULL);
> }
>
> +/* Notifier wrappers for transitioning to state machine */
> +static int notify_prepare(unsigned int cpu)
> +{
> + int nr_calls = 0;
> + int ret;
> +
> + ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
> + if (ret) {
> + nr_calls--;
> + printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
> + __func__, cpu);
> + __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
> + }
> + return ret;
> +}
> +
> +static int notify_online(unsigned int cpu)
> +{
> + cpu_notify(CPU_ONLINE, cpu);
> + return 0;
> +}
> +
> +static int bringup_cpu(unsigned int cpu)
> +{
> + struct task_struct *idle = idle_thread_get(cpu);
> + int ret;
> +
> + /* Arch-specific enabling code. */
> + ret = __cpu_up(cpu, idle);
> + if (ret) {
> + cpu_notify(CPU_UP_CANCELED, cpu);
> + return ret;
> + }
> + BUG_ON(!cpu_online(cpu));
> + return 0;
> +}
> +
> #ifdef CONFIG_HOTPLUG_CPU
>
> static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> @@ -481,7 +518,7 @@ void smpboot_thread_init(void)
> static int _cpu_up(unsigned int cpu, int tasks_frozen)
> {
> struct task_struct *idle;
> - int ret, nr_calls = 0;
> + int ret;
>
> cpu_hotplug_begin();
>
> @@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
> goto out;
> }
>
> + cpuhp_tasks_frozen = tasks_frozen;
> +
> ret = smpboot_create_threads(cpu);
> if (ret)
> goto out;
>
> - cpuhp_tasks_frozen = tasks_frozen;
> -
> - ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
> - if (ret) {
> - nr_calls--;
> - pr_warn("%s: attempt to bring up CPU %u failed\n",
> - __func__, cpu);
> - goto out_notify;
> - }
> -
> - /* Arch-specific enabling code. */
> - ret = __cpu_up(cpu, idle);
> -
> - if (ret != 0)
> - goto out_notify;
> - BUG_ON(!cpu_online(cpu));
> + ret = notify_prepare(cpu);
> + if (ret)
> + goto out;
>
> - /* Now call notifier in preparation. */
> - cpu_notify(CPU_ONLINE, cpu);
> + ret = bringup_cpu(cpu);
> + if (ret)
> + goto out;
>
> -out_notify:
> - if (ret != 0)
> - __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
> + notify_online(cpu);
> out:
> cpu_hotplug_done();
>
>
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu |
| Message-ID | <r6vCH-7sv-25@gated-at.bofh.it> |
| In reply to | #1344561 |
Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpuhotplug.h | 10 +--
kernel/cpu.c | 144 ++++++++++++++++++++++++++++++---------------
2 files changed, 102 insertions(+), 52 deletions(-)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
CPUHP_AP_ONLINE,
CPUHP_TEARDOWN_CPU,
CPUHP_CPU_SET_ACTIVE,
- CPUHP_SMPBOOT_THREADS,
- CPUHP_NOTIFY_ONLINE,
- CPUHP_ONLINE_DYN,
- CPUHP_ONLINE_DYN_END = CPUHP_ONLINE_DYN + 30,
+ CPUHP_KICK_AP_THREAD,
+ CPUHP_BP_ONLINE,
+ CPUHP_AP_SMPBOOT_THREADS,
+ CPUHP_AP_NOTIFY_ONLINE,
+ CPUHP_AP_ONLINE_DYN,
+ CPUHP_AP_ONLINE_DYN_END = CPUHP_AP_ONLINE_DYN + 30,
CPUHP_ONLINE,
};
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int
/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
{
- enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+ enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
}
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned in
ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
}
} else {
+ /* Cannot happen .... */
+ BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
/* Regular hotplug work */
if (st->state < st->target)
ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int
}
/* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
{
- struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
- enum cpuhp_state state = st->state;
-
- trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
st->result = 0;
st->cb = NULL;
/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned i
smp_mb();
st->should_run = true;
wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ enum cpuhp_state state = st->state;
+
+ trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+ __cpuhp_kick_ap_work(st);
wait_for_completion(&st->done);
trace_cpuhp_exit(cpu, st->state, state, st->result);
return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu
else
synchronize_rcu();
+ /* Park the hotplug thread */
+ kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
/*
* Prevent irq alloc/free while the dying cpu reorganizes the
* interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int
prev_state = st->state;
st->target = target;
+ /*
+ * If the current CPU state is in the range of the AP hotplug thread,
+ * then we need to kick the thread.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD) {
+ ret = cpuhp_kick_ap_work(cpu);
+ /*
+ * The AP side has done the error rollback already. Just
+ * return the error code..
+ */
+ if (ret)
+ goto out;
+
+ /*
+ * We might have stopped still in the range of the AP hotplug
+ * thread. Nothing to do anymore.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD)
+ goto out;
+ }
+ /*
+ * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+ * to do the further cleanups.
+ */
ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
cpu_hotplug_done();
/* This post dead nonsense must die */
if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cp
*/
static int cpuhp_set_cpu_active(unsigned int cpu)
{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
/* The cpu is marked online, set it active now */
set_cpu_active(cpu, true);
- /* Unpark the stopper thread */
+ /* Unpark the stopper thread and the hotplug thread */
stop_machine_unpark(cpu);
+ kthread_unpark(st->thread);
return 0;
}
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int
cpuhp_tasks_frozen = tasks_frozen;
st->target = target;
+ /*
+ * If the current CPU state is in the range of the AP hotplug thread,
+ * then we need to kick the thread once more.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD) {
+ ret = cpuhp_kick_ap_work(cpu);
+ /*
+ * The AP side has done the error rollback already. Just
+ * return the error code..
+ */
+ if (ret)
+ goto out;
+ }
+
+ /*
+ * Try to reach the target state. We max out on the BP at
+ * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+ * responsible for bringing it up to the target state.
+ */
+ target = min((int)target, CPUHP_KICK_AP_THREAD);
ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
out:
cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states
.startup = cpuhp_set_cpu_active,
.teardown = NULL,
},
- [CPUHP_SMPBOOT_THREADS] = {
- .name = "smpboot:threads",
- .startup = smpboot_unpark_threads,
- .teardown = smpboot_park_threads,
- },
- [CPUHP_NOTIFY_ONLINE] = {
- .name = "notify:online",
- .startup = notify_online,
- .teardown = notify_down_prepare,
- .cant_stop = true,
+ [CPUHP_KICK_AP_THREAD] = {
+ .name = "cpuhp:kickthread",
+ .startup = cpuhp_kick_ap_work,
+ .teardown = cpuhp_kick_ap_work,
},
#endif
- [CPUHP_ONLINE] = {
+ [CPUHP_BP_ONLINE] = {
.name = "online",
.startup = NULL,
.teardown = NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states
.skip_onerr = true,
.cant_stop = true,
},
+ [CPUHP_AP_SMPBOOT_THREADS] = {
+ .name = "smpboot:threads",
+ .startup = smpboot_unpark_threads,
+ .teardown = smpboot_park_threads,
+ },
+ [CPUHP_AP_NOTIFY_ONLINE] = {
+ .name = "notify:online",
+ .startup = notify_online,
+ .teardown = notify_down_prepare,
+ },
#endif
[CPUHP_ONLINE] = {
.name = "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_sta
static bool cpuhp_is_ap_state(enum cpuhp_state state)
{
- return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+ if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+ return true;
+ return state > CPUHP_BP_ONLINE;
}
static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum
return cpuhp_get_step(state)->teardown;
}
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
- int (*cb)(unsigned int cpu) = __cb;
-
- BUG_ON(cb(smp_processor_id()));
-}
-
/*
* Call the startup/teardown function for a step either on the AP or
* on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enu
if (!cb)
return 0;
-
- /*
- * This invokes the callback directly for now. In a later step we
- * convert that to use cpuhp_invoke_callback().
- */
- if (cpuhp_is_ap_state(state)) {
- /*
- * Note, that a function called on the AP is not
- * allowed to fail.
- */
- if (cpu_online(cpu))
- smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
- return 0;
- }
-
/*
* The non AP bound callbacks can fail on bringup. On teardown
* e.g. module removal we crash for now.
*/
- ret = cb(cpu);
+#ifdef CONFIG_SMP
+ if (cpuhp_is_ap_state(state))
+ ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+ else
+ ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+ ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
BUG_ON(ret && !bringup);
return ret;
}
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuh
enum cpuhp_state i;
mutex_lock(&cpuhp_state_mutex);
- for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
- if (cpuhp_bp_states[i].name)
+ for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+ if (cpuhp_ap_states[i].name)
continue;
- cpuhp_bp_states[i].name = "Reserved";
+ cpuhp_ap_states[i].name = "Reserved";
mutex_unlock(&cpuhp_state_mutex);
return i;
}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state
get_online_cpus();
/* currently assignments for the ONLINE state are possible */
- if (state == CPUHP_ONLINE_DYN) {
+ if (state == CPUHP_AP_ONLINE_DYN) {
dyn_state = 1;
ret = cpuhp_reserve_state(state);
if (ret < 0)
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Move online calls to hotplugged cpu |
| Message-ID | <r7YCE-5hG-39@gated-at.bofh.it> |
| In reply to | #1344566 |
Commit-ID: 1cf4f629d9d246519a1e76c021806f2a51ddba4d
Gitweb: http://git.kernel.org/tip/1cf4f629d9d246519a1e76c021806f2a51ddba4d
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:39 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:57 +0100
cpu/hotplug: Move online calls to hotplugged cpu
Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.536364371@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpuhotplug.h | 10 ++--
kernel/cpu.c | 144 ++++++++++++++++++++++++++++++---------------
2 files changed, 102 insertions(+), 52 deletions(-)
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 3867910..8a715bb 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
CPUHP_AP_ONLINE,
CPUHP_TEARDOWN_CPU,
CPUHP_CPU_SET_ACTIVE,
- CPUHP_SMPBOOT_THREADS,
- CPUHP_NOTIFY_ONLINE,
- CPUHP_ONLINE_DYN,
- CPUHP_ONLINE_DYN_END = CPUHP_ONLINE_DYN + 30,
+ CPUHP_KICK_AP_THREAD,
+ CPUHP_BP_ONLINE,
+ CPUHP_AP_SMPBOOT_THREADS,
+ CPUHP_AP_NOTIFY_ONLINE,
+ CPUHP_AP_ONLINE_DYN,
+ CPUHP_AP_ONLINE_DYN_END = CPUHP_AP_ONLINE_DYN + 30,
CPUHP_ONLINE,
};
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 9048c33..e220e56 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int cpu)
/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
{
- enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+ enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
}
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned int cpu)
ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
}
} else {
+ /* Cannot happen .... */
+ BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
/* Regular hotplug work */
if (st->state < st->target)
ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
}
/* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
{
- struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
- enum cpuhp_state state = st->state;
-
- trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
st->result = 0;
st->cb = NULL;
/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned int cpu)
smp_mb();
st->should_run = true;
wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ enum cpuhp_state state = st->state;
+
+ trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+ __cpuhp_kick_ap_work(st);
wait_for_completion(&st->done);
trace_cpuhp_exit(cpu, st->state, state, st->result);
return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu)
else
synchronize_rcu();
+ /* Park the hotplug thread */
+ kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
/*
* Prevent irq alloc/free while the dying cpu reorganizes the
* interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
prev_state = st->state;
st->target = target;
+ /*
+ * If the current CPU state is in the range of the AP hotplug thread,
+ * then we need to kick the thread.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD) {
+ ret = cpuhp_kick_ap_work(cpu);
+ /*
+ * The AP side has done the error rollback already. Just
+ * return the error code..
+ */
+ if (ret)
+ goto out;
+
+ /*
+ * We might have stopped still in the range of the AP hotplug
+ * thread. Nothing to do anymore.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD)
+ goto out;
+ }
+ /*
+ * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+ * to do the further cleanups.
+ */
ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
cpu_hotplug_done();
/* This post dead nonsense must die */
if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cpu)
*/
static int cpuhp_set_cpu_active(unsigned int cpu)
{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
/* The cpu is marked online, set it active now */
set_cpu_active(cpu, true);
- /* Unpark the stopper thread */
+ /* Unpark the stopper thread and the hotplug thread */
stop_machine_unpark(cpu);
+ kthread_unpark(st->thread);
return 0;
}
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
cpuhp_tasks_frozen = tasks_frozen;
st->target = target;
+ /*
+ * If the current CPU state is in the range of the AP hotplug thread,
+ * then we need to kick the thread once more.
+ */
+ if (st->state >= CPUHP_KICK_AP_THREAD) {
+ ret = cpuhp_kick_ap_work(cpu);
+ /*
+ * The AP side has done the error rollback already. Just
+ * return the error code..
+ */
+ if (ret)
+ goto out;
+ }
+
+ /*
+ * Try to reach the target state. We max out on the BP at
+ * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+ * responsible for bringing it up to the target state.
+ */
+ target = min((int)target, CPUHP_KICK_AP_THREAD);
ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
out:
cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states[] = {
.startup = cpuhp_set_cpu_active,
.teardown = NULL,
},
- [CPUHP_SMPBOOT_THREADS] = {
- .name = "smpboot:threads",
- .startup = smpboot_unpark_threads,
- .teardown = smpboot_park_threads,
- },
- [CPUHP_NOTIFY_ONLINE] = {
- .name = "notify:online",
- .startup = notify_online,
- .teardown = notify_down_prepare,
- .cant_stop = true,
+ [CPUHP_KICK_AP_THREAD] = {
+ .name = "cpuhp:kickthread",
+ .startup = cpuhp_kick_ap_work,
+ .teardown = cpuhp_kick_ap_work,
},
#endif
- [CPUHP_ONLINE] = {
+ [CPUHP_BP_ONLINE] = {
.name = "online",
.startup = NULL,
.teardown = NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states[] = {
.skip_onerr = true,
.cant_stop = true,
},
+ [CPUHP_AP_SMPBOOT_THREADS] = {
+ .name = "smpboot:threads",
+ .startup = smpboot_unpark_threads,
+ .teardown = smpboot_park_threads,
+ },
+ [CPUHP_AP_NOTIFY_ONLINE] = {
+ .name = "notify:online",
+ .startup = notify_online,
+ .teardown = notify_down_prepare,
+ },
#endif
[CPUHP_ONLINE] = {
.name = "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_state state)
static bool cpuhp_is_ap_state(enum cpuhp_state state)
{
- return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+ if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+ return true;
+ return state > CPUHP_BP_ONLINE;
}
static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
return cpuhp_get_step(state)->teardown;
}
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
- int (*cb)(unsigned int cpu) = __cb;
-
- BUG_ON(cb(smp_processor_id()));
-}
-
/*
* Call the startup/teardown function for a step either on the AP or
* on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
if (!cb)
return 0;
-
- /*
- * This invokes the callback directly for now. In a later step we
- * convert that to use cpuhp_invoke_callback().
- */
- if (cpuhp_is_ap_state(state)) {
- /*
- * Note, that a function called on the AP is not
- * allowed to fail.
- */
- if (cpu_online(cpu))
- smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
- return 0;
- }
-
/*
* The non AP bound callbacks can fail on bringup. On teardown
* e.g. module removal we crash for now.
*/
- ret = cb(cpu);
+#ifdef CONFIG_SMP
+ if (cpuhp_is_ap_state(state))
+ ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+ else
+ ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+ ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
BUG_ON(ret && !bringup);
return ret;
}
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuhp_state state)
enum cpuhp_state i;
mutex_lock(&cpuhp_state_mutex);
- for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
- if (cpuhp_bp_states[i].name)
+ for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+ if (cpuhp_ap_states[i].name)
continue;
- cpuhp_bp_states[i].name = "Reserved";
+ cpuhp_ap_states[i].name = "Reserved";
mutex_unlock(&cpuhp_state_mutex);
return i;
}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state state,
get_online_cpus();
/* currently assignments for the ONLINE state are possible */
- if (state == CPUHP_ONLINE_DYN) {
+ if (state == CPUHP_AP_ONLINE_DYN) {
dyn_state = 1;
ret = cpuhp_reserve_state(state);
if (ret < 0)
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 02/20] cpu/hotplug: Restructure FROZEN state handling |
| Message-ID | <r6vCH-7sv-27@gated-at.bofh.it> |
| In reply to | #1344561 |
There are only a few callbacks which really care about FROZEN
vs. !FROZEN. No need to have extra states for this.
Publish the frozen state in an extra variable which is updated under
the hotplug lock and let the users interested deal with it w/o
imposing that extra state checks on everyone.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 +
kernel/cpu.c | 69 +++++++++++++++++++++-------------------------------
2 files changed, 31 insertions(+), 40 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -118,6 +118,7 @@ enum {
#ifdef CONFIG_SMP
+extern bool cpuhp_tasks_frozen;
/* Need to know about CPUs going up/down? */
#if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
#define cpu_notifier(fn, pri) { \
@@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
#define cpu_notifier_register_done cpu_maps_update_done
#else /* CONFIG_SMP */
+#define cpuhp_tasks_frozen 0
#define cpu_notifier(fn, pri) do { (void)(fn); } while (0)
#define __cpu_notifier(fn, pri) do { (void)(fn); } while (0)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -29,6 +29,8 @@
#ifdef CONFIG_SMP
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
+bool cpuhp_tasks_frozen;
+EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
/*
* The following two APIs (cpu_maps_update_begin/done) must be used when
@@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notif
return raw_notifier_chain_register(&cpu_chain, nb);
}
-static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
+static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
int *nr_calls)
{
+ unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
+ void *hcpu = (void *)(long)cpu;
+
int ret;
- ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
+ ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
nr_calls);
return notifier_to_errno(ret);
}
-static int cpu_notify(unsigned long val, void *v)
+static int cpu_notify(unsigned long val, unsigned int cpu)
{
- return __cpu_notify(val, v, -1, NULL);
+ return __cpu_notify(val, cpu, -1, NULL);
}
#ifdef CONFIG_HOTPLUG_CPU
-static void cpu_notify_nofail(unsigned long val, void *v)
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
{
- BUG_ON(cpu_notify(val, v));
+ BUG_ON(cpu_notify(val, cpu));
}
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
@@ -311,27 +316,21 @@ static inline void check_for_tasks(int d
read_unlock(&tasklist_lock);
}
-struct take_cpu_down_param {
- unsigned long mod;
- void *hcpu;
-};
-
/* Take this CPU down. */
static int take_cpu_down(void *_param)
{
- struct take_cpu_down_param *param = _param;
- int err;
+ int err, cpu = smp_processor_id();
/* Ensure this CPU doesn't handle any more interrupts. */
err = __cpu_disable();
if (err < 0)
return err;
- cpu_notify(CPU_DYING | param->mod, param->hcpu);
+ cpu_notify(CPU_DYING, cpu);
/* Give up timekeeping duties */
tick_handover_do_timer();
/* Park the stopper thread */
- stop_machine_park((long)param->hcpu);
+ stop_machine_park(cpu);
return 0;
}
@@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
static int _cpu_down(unsigned int cpu, int tasks_frozen)
{
int err, nr_calls = 0;
- void *hcpu = (void *)(long)cpu;
- unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
- struct take_cpu_down_param tcd_param = {
- .mod = mod,
- .hcpu = hcpu,
- };
if (num_online_cpus() == 1)
return -EBUSY;
@@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, i
cpu_hotplug_begin();
- err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
if (err) {
nr_calls--;
- __cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
+ __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
pr_warn("%s: attempt to take down CPU %u failed\n",
__func__, cpu);
goto out_release;
@@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, i
/*
* So now all preempt/rcu users must observe !cpu_active().
*/
- err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+ err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
if (err) {
/* CPU didn't die: tell everyone. Can't complain. */
- cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
+ cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
irq_unlock_sparse();
goto out_release;
}
@@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, i
/* CPU is completely dead: tell everyone. Too late to complain. */
tick_cleanup_dead_cpu(cpu);
- cpu_notify_nofail(CPU_DEAD | mod, hcpu);
+ cpu_notify_nofail(CPU_DEAD, cpu);
check_for_tasks(cpu);
out_release:
cpu_hotplug_done();
if (!err)
- cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+ cpu_notify_nofail(CPU_POST_DEAD, cpu);
return err;
}
@@ -485,10 +480,8 @@ void smpboot_thread_init(void)
/* Requires cpu_add_remove_lock to be held */
static int _cpu_up(unsigned int cpu, int tasks_frozen)
{
- int ret, nr_calls = 0;
- void *hcpu = (void *)(long)cpu;
- unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
struct task_struct *idle;
+ int ret, nr_calls = 0;
cpu_hotplug_begin();
@@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int
if (ret)
goto out;
- ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
if (ret) {
nr_calls--;
pr_warn("%s: attempt to bring up CPU %u failed\n",
@@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int
BUG_ON(!cpu_online(cpu));
/* Now call notifier in preparation. */
- cpu_notify(CPU_ONLINE | mod, hcpu);
+ cpu_notify(CPU_ONLINE, cpu);
out_notify:
if (ret != 0)
- __cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
+ __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
out:
cpu_hotplug_done();
@@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
*/
void notify_cpu_starting(unsigned int cpu)
{
- unsigned long val = CPU_STARTING;
-
-#ifdef CONFIG_PM_SLEEP_SMP
- if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
- val = CPU_STARTING_FROZEN;
-#endif /* CONFIG_PM_SLEEP_SMP */
- cpu_notify(val, (void *)(long)cpu);
+ cpu_notify(CPU_STARTING, cpu);
}
#endif /* CONFIG_SMP */
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling |
| Message-ID | <r7YCE-5hG-49@gated-at.bofh.it> |
| In reply to | #1344567 |
Commit-ID: 090e77c391dd983c8945b8e2e16d09f378d2e334
Gitweb: http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
cpu/hotplug: Restructure FROZEN state handling
There are only a few callbacks which really care about FROZEN
vs. !FROZEN. No need to have extra states for this.
Publish the frozen state in an extra variable which is updated under
the hotplug lock and let the users interested deal with it w/o
imposing that extra state checks on everyone.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 ++
kernel/cpu.c | 69 ++++++++++++++++++++++-------------------------------
2 files changed, 31 insertions(+), 40 deletions(-)
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index d2ca8c3..f2fb549 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -118,6 +118,7 @@ enum {
#ifdef CONFIG_SMP
+extern bool cpuhp_tasks_frozen;
/* Need to know about CPUs going up/down? */
#if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
#define cpu_notifier(fn, pri) { \
@@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
#define cpu_notifier_register_done cpu_maps_update_done
#else /* CONFIG_SMP */
+#define cpuhp_tasks_frozen 0
#define cpu_notifier(fn, pri) do { (void)(fn); } while (0)
#define __cpu_notifier(fn, pri) do { (void)(fn); } while (0)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 5b9d396..41a6cb8 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -29,6 +29,8 @@
#ifdef CONFIG_SMP
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
+bool cpuhp_tasks_frozen;
+EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
/*
* The following two APIs (cpu_maps_update_begin/done) must be used when
@@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notifier_block *nb)
return raw_notifier_chain_register(&cpu_chain, nb);
}
-static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
+static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
int *nr_calls)
{
+ unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
+ void *hcpu = (void *)(long)cpu;
+
int ret;
- ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
+ ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
nr_calls);
return notifier_to_errno(ret);
}
-static int cpu_notify(unsigned long val, void *v)
+static int cpu_notify(unsigned long val, unsigned int cpu)
{
- return __cpu_notify(val, v, -1, NULL);
+ return __cpu_notify(val, cpu, -1, NULL);
}
#ifdef CONFIG_HOTPLUG_CPU
-static void cpu_notify_nofail(unsigned long val, void *v)
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
{
- BUG_ON(cpu_notify(val, v));
+ BUG_ON(cpu_notify(val, cpu));
}
EXPORT_SYMBOL(register_cpu_notifier);
EXPORT_SYMBOL(__register_cpu_notifier);
@@ -311,27 +316,21 @@ static inline void check_for_tasks(int dead_cpu)
read_unlock(&tasklist_lock);
}
-struct take_cpu_down_param {
- unsigned long mod;
- void *hcpu;
-};
-
/* Take this CPU down. */
static int take_cpu_down(void *_param)
{
- struct take_cpu_down_param *param = _param;
- int err;
+ int err, cpu = smp_processor_id();
/* Ensure this CPU doesn't handle any more interrupts. */
err = __cpu_disable();
if (err < 0)
return err;
- cpu_notify(CPU_DYING | param->mod, param->hcpu);
+ cpu_notify(CPU_DYING, cpu);
/* Give up timekeeping duties */
tick_handover_do_timer();
/* Park the stopper thread */
- stop_machine_park((long)param->hcpu);
+ stop_machine_park(cpu);
return 0;
}
@@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
static int _cpu_down(unsigned int cpu, int tasks_frozen)
{
int err, nr_calls = 0;
- void *hcpu = (void *)(long)cpu;
- unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
- struct take_cpu_down_param tcd_param = {
- .mod = mod,
- .hcpu = hcpu,
- };
if (num_online_cpus() == 1)
return -EBUSY;
@@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
cpu_hotplug_begin();
- err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
if (err) {
nr_calls--;
- __cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
+ __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
pr_warn("%s: attempt to take down CPU %u failed\n",
__func__, cpu);
goto out_release;
@@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
/*
* So now all preempt/rcu users must observe !cpu_active().
*/
- err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+ err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
if (err) {
/* CPU didn't die: tell everyone. Can't complain. */
- cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
+ cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
irq_unlock_sparse();
goto out_release;
}
@@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
/* CPU is completely dead: tell everyone. Too late to complain. */
tick_cleanup_dead_cpu(cpu);
- cpu_notify_nofail(CPU_DEAD | mod, hcpu);
+ cpu_notify_nofail(CPU_DEAD, cpu);
check_for_tasks(cpu);
out_release:
cpu_hotplug_done();
if (!err)
- cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+ cpu_notify_nofail(CPU_POST_DEAD, cpu);
return err;
}
@@ -485,10 +480,8 @@ void smpboot_thread_init(void)
/* Requires cpu_add_remove_lock to be held */
static int _cpu_up(unsigned int cpu, int tasks_frozen)
{
- int ret, nr_calls = 0;
- void *hcpu = (void *)(long)cpu;
- unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
struct task_struct *idle;
+ int ret, nr_calls = 0;
cpu_hotplug_begin();
@@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
if (ret)
goto out;
- ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
+ cpuhp_tasks_frozen = tasks_frozen;
+
+ ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
if (ret) {
nr_calls--;
pr_warn("%s: attempt to bring up CPU %u failed\n",
@@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
BUG_ON(!cpu_online(cpu));
/* Now call notifier in preparation. */
- cpu_notify(CPU_ONLINE | mod, hcpu);
+ cpu_notify(CPU_ONLINE, cpu);
out_notify:
if (ret != 0)
- __cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
+ __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
out:
cpu_hotplug_done();
@@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
*/
void notify_cpu_starting(unsigned int cpu)
{
- unsigned long val = CPU_STARTING;
-
-#ifdef CONFIG_PM_SLEEP_SMP
- if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
- val = CPU_STARTING_FROZEN;
-#endif /* CONFIG_PM_SLEEP_SMP */
- cpu_notify(val, (void *)(long)cpu);
+ cpu_notify(CPU_STARTING, cpu);
}
#endif /* CONFIG_SMP */
[toc] | [prev] | [next] | [standalone]
| From | "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> |
|---|---|
| Date | 2016-03-03 00:10 +0100 |
| Subject | Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling |
| Message-ID | <r8o42-6yA-3@gated-at.bofh.it> |
| In reply to | #1346987 |
On 3/1/16 2:51 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID: 090e77c391dd983c8945b8e2e16d09f378d2e334
> Gitweb: http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
> Author: Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
> Committer: Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
>
> cpu/hotplug: Restructure FROZEN state handling
>
> There are only a few callbacks which really care about FROZEN
> vs. !FROZEN. No need to have extra states for this.
>
> Publish the frozen state in an extra variable which is updated under
> the hotplug lock and let the users interested deal with it w/o
> imposing that extra state checks on everyone.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>
Regards,
Srivatsa S. Bhat
> include/linux/cpu.h | 2 ++
> kernel/cpu.c | 69 ++++++++++++++++++++++-------------------------------
> 2 files changed, 31 insertions(+), 40 deletions(-)
>
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index d2ca8c3..f2fb549 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -118,6 +118,7 @@ enum {
>
>
> #ifdef CONFIG_SMP
> +extern bool cpuhp_tasks_frozen;
> /* Need to know about CPUs going up/down? */
> #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
> #define cpu_notifier(fn, pri) { \
> @@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
> #define cpu_notifier_register_done cpu_maps_update_done
>
> #else /* CONFIG_SMP */
> +#define cpuhp_tasks_frozen 0
>
> #define cpu_notifier(fn, pri) do { (void)(fn); } while (0)
> #define __cpu_notifier(fn, pri) do { (void)(fn); } while (0)
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 5b9d396..41a6cb8 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -29,6 +29,8 @@
> #ifdef CONFIG_SMP
> /* Serializes the updates to cpu_online_mask, cpu_present_mask */
> static DEFINE_MUTEX(cpu_add_remove_lock);
> +bool cpuhp_tasks_frozen;
> +EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
>
> /*
> * The following two APIs (cpu_maps_update_begin/done) must be used when
> @@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notifier_block *nb)
> return raw_notifier_chain_register(&cpu_chain, nb);
> }
>
> -static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
> +static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
> int *nr_calls)
> {
> + unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
> + void *hcpu = (void *)(long)cpu;
> +
> int ret;
>
> - ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
> + ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
> nr_calls);
>
> return notifier_to_errno(ret);
> }
>
> -static int cpu_notify(unsigned long val, void *v)
> +static int cpu_notify(unsigned long val, unsigned int cpu)
> {
> - return __cpu_notify(val, v, -1, NULL);
> + return __cpu_notify(val, cpu, -1, NULL);
> }
>
> #ifdef CONFIG_HOTPLUG_CPU
>
> -static void cpu_notify_nofail(unsigned long val, void *v)
> +static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> {
> - BUG_ON(cpu_notify(val, v));
> + BUG_ON(cpu_notify(val, cpu));
> }
> EXPORT_SYMBOL(register_cpu_notifier);
> EXPORT_SYMBOL(__register_cpu_notifier);
> @@ -311,27 +316,21 @@ static inline void check_for_tasks(int dead_cpu)
> read_unlock(&tasklist_lock);
> }
>
> -struct take_cpu_down_param {
> - unsigned long mod;
> - void *hcpu;
> -};
> -
> /* Take this CPU down. */
> static int take_cpu_down(void *_param)
> {
> - struct take_cpu_down_param *param = _param;
> - int err;
> + int err, cpu = smp_processor_id();
>
> /* Ensure this CPU doesn't handle any more interrupts. */
> err = __cpu_disable();
> if (err < 0)
> return err;
>
> - cpu_notify(CPU_DYING | param->mod, param->hcpu);
> + cpu_notify(CPU_DYING, cpu);
> /* Give up timekeeping duties */
> tick_handover_do_timer();
> /* Park the stopper thread */
> - stop_machine_park((long)param->hcpu);
> + stop_machine_park(cpu);
> return 0;
> }
>
> @@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
> static int _cpu_down(unsigned int cpu, int tasks_frozen)
> {
> int err, nr_calls = 0;
> - void *hcpu = (void *)(long)cpu;
> - unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
> - struct take_cpu_down_param tcd_param = {
> - .mod = mod,
> - .hcpu = hcpu,
> - };
>
> if (num_online_cpus() == 1)
> return -EBUSY;
> @@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>
> cpu_hotplug_begin();
>
> - err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
> + cpuhp_tasks_frozen = tasks_frozen;
> +
> + err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
> if (err) {
> nr_calls--;
> - __cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
> + __cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
> pr_warn("%s: attempt to take down CPU %u failed\n",
> __func__, cpu);
> goto out_release;
> @@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
> /*
> * So now all preempt/rcu users must observe !cpu_active().
> */
> - err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
> + err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
> if (err) {
> /* CPU didn't die: tell everyone. Can't complain. */
> - cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
> + cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
> irq_unlock_sparse();
> goto out_release;
> }
> @@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>
> /* CPU is completely dead: tell everyone. Too late to complain. */
> tick_cleanup_dead_cpu(cpu);
> - cpu_notify_nofail(CPU_DEAD | mod, hcpu);
> + cpu_notify_nofail(CPU_DEAD, cpu);
>
> check_for_tasks(cpu);
>
> out_release:
> cpu_hotplug_done();
> if (!err)
> - cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
> + cpu_notify_nofail(CPU_POST_DEAD, cpu);
> return err;
> }
>
> @@ -485,10 +480,8 @@ void smpboot_thread_init(void)
> /* Requires cpu_add_remove_lock to be held */
> static int _cpu_up(unsigned int cpu, int tasks_frozen)
> {
> - int ret, nr_calls = 0;
> - void *hcpu = (void *)(long)cpu;
> - unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
> struct task_struct *idle;
> + int ret, nr_calls = 0;
>
> cpu_hotplug_begin();
>
> @@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
> if (ret)
> goto out;
>
> - ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
> + cpuhp_tasks_frozen = tasks_frozen;
> +
> + ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
> if (ret) {
> nr_calls--;
> pr_warn("%s: attempt to bring up CPU %u failed\n",
> @@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
> BUG_ON(!cpu_online(cpu));
>
> /* Now call notifier in preparation. */
> - cpu_notify(CPU_ONLINE | mod, hcpu);
> + cpu_notify(CPU_ONLINE, cpu);
>
> out_notify:
> if (ret != 0)
> - __cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
> + __cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
> out:
> cpu_hotplug_done();
>
> @@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
> */
> void notify_cpu_starting(unsigned int cpu)
> {
> - unsigned long val = CPU_STARTING;
> -
> -#ifdef CONFIG_PM_SLEEP_SMP
> - if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
> - val = CPU_STARTING_FROZEN;
> -#endif /* CONFIG_PM_SLEEP_SMP */
> - cpu_notify(val, (void *)(long)cpu);
> + cpu_notify(CPU_STARTING, cpu);
> }
>
> #endif /* CONFIG_SMP */
>
[toc] | [prev] | [next] | [standalone]
| From | "Srivatsa S. Bhat" <srivatsa@csail.mit.edu> |
|---|---|
| Date | 2016-03-03 00:50 +0100 |
| Subject | Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling |
| Message-ID | <r8oGK-6PW-1@gated-at.bofh.it> |
| In reply to | #1346987 |
On 3/1/16 2:51 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID: 090e77c391dd983c8945b8e2e16d09f378d2e334
> Gitweb: http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
> Author: Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
> Committer: Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
>
> cpu/hotplug: Restructure FROZEN state handling
>
> There are only a few callbacks which really care about FROZEN
> vs. !FROZEN. No need to have extra states for this.
>
> Publish the frozen state in an extra variable which is updated under
> the hotplug lock and let the users interested deal with it w/o
> imposing that extra state checks on everyone.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> include/linux/cpu.h | 2 ++
> kernel/cpu.c | 69 ++++++++++++++++++++++-------------------------------
> 2 files changed, 31 insertions(+), 40 deletions(-)
>
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index d2ca8c3..f2fb549 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -118,6 +118,7 @@ enum {
>
>
> #ifdef CONFIG_SMP
> +extern bool cpuhp_tasks_frozen;
> /* Need to know about CPUs going up/down? */
> #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
> #define cpu_notifier(fn, pri) { \
> @@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
> #define cpu_notifier_register_done cpu_maps_update_done
>
> #else /* CONFIG_SMP */
> +#define cpuhp_tasks_frozen 0
>
> #define cpu_notifier(fn, pri) do { (void)(fn); } while (0)
> #define __cpu_notifier(fn, pri) do { (void)(fn); } while (0)
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 5b9d396..41a6cb8 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -29,6 +29,8 @@
> #ifdef CONFIG_SMP
> /* Serializes the updates to cpu_online_mask, cpu_present_mask */
> static DEFINE_MUTEX(cpu_add_remove_lock);
> +bool cpuhp_tasks_frozen;
> +EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
>
One small nitpick though: we don't need to export this symbol yet; it can
be deferred until the callbacks that need it are actually modified to use
this value (presumably in a later patchset).
Regards,
Srivatsa S. Bhat
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 11/20] cpu/hotplug: Implement setup/removal interface |
| Message-ID | <r6vCH-7sv-33@gated-at.bofh.it> |
| In reply to | #1344561 |
Implement function which allow to setup/remove hotplug state callbacks.
The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.
The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.
This includes rollback to the previous state in case of failure.
A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them
For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.
[ Dynamic registering implemented by Sebastian Siewior ]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpuhotplug.h | 67 +++++++++++++
kernel/cpu.c | 224 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 291 insertions(+)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
CPUHP_AP_ONLINE,
CPUHP_TEARDOWN_CPU,
CPUHP_NOTIFY_ONLINE,
+ CPUHP_ONLINE_DYN,
+ CPUHP_ONLINE_DYN_END = CPUHP_ONLINE_DYN + 30,
CPUHP_ONLINE,
};
+int __cpuhp_setup_state(enum cpuhp_state state, const char *name, bool invoke,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state: The state for which the calls are installed
+ * @name: Name of the callback (will be used in debug output)
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ * callbacks
+ * @state: The state for which the calls are installed
+ * @name: Name of the callback.
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state: The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+ __cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ * teardown
+ * @state: The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+ __cpuhp_remove_state(state, false);
+}
+
#endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states
},
};
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+ if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+ return -EINVAL;
+ return 0;
+}
+
static bool cpuhp_is_ap_state(enum cpuhp_state state)
{
return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step
return sp + state;
}
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ /* (Un)Install the callbacks for further cpu hotplug operations */
+ struct cpuhp_step *sp;
+
+ mutex_lock(&cpuhp_state_mutex);
+ sp = cpuhp_get_step(state);
+ sp->startup = startup;
+ sp->teardown = teardown;
+ sp->name = name;
+ mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+ return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+ int (*cb)(unsigned int cpu) = __cb;
+
+ BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+ int (*cb)(unsigned int), bool bringup)
+{
+ int ret;
+
+ if (!cb)
+ return 0;
+
+ /*
+ * This invokes the callback directly for now. In a later step we
+ * convert that to use cpuhp_invoke_callback().
+ */
+ if (cpuhp_is_ap_state(state)) {
+ /*
+ * Note, that a function called on the AP is not
+ * allowed to fail.
+ */
+ if (cpu_online(cpu))
+ smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+ return 0;
+ }
+
+ /*
+ * The non AP bound callbacks can fail on bringup. On teardown
+ * e.g. module removal we crash for now.
+ */
+ ret = cb(cpu);
+ BUG_ON(ret && !bringup);
+ return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+ int (*teardown)(unsigned int cpu))
+{
+ int cpu;
+
+ if (!teardown)
+ return;
+
+ /* Roll back the already executed steps on the other cpus */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpu >= failedcpu)
+ break;
+
+ /* Did we invoke the startup call on that cpu ? */
+ if (cpustate >= state)
+ cpuhp_issue_call(cpu, state, teardown, false);
+ }
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+ enum cpuhp_state i;
+
+ mutex_lock(&cpuhp_state_mutex);
+ for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+ if (cpuhp_bp_states[i].name)
+ continue;
+
+ cpuhp_bp_states[i].name = "Reserved";
+ mutex_unlock(&cpuhp_state_mutex);
+ return i;
+ }
+ mutex_unlock(&cpuhp_state_mutex);
+ WARN(1, "No more dynamic states available for CPU hotplug\n");
+ return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state: The state to setup
+ * @invoke: If true, the startup function is invoked for cpus where
+ * cpu state >= @state
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+ const char *name, bool invoke,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ int cpu, ret = 0;
+ int dyn_state = 0;
+
+ if (cpuhp_cb_check(state) || !name)
+ return -EINVAL;
+
+ get_online_cpus();
+
+ /* currently assignments for the ONLINE state are possible */
+ if (state == CPUHP_ONLINE_DYN) {
+ dyn_state = 1;
+ ret = cpuhp_reserve_state(state);
+ if (ret < 0)
+ goto out;
+ state = ret;
+ }
+
+ cpuhp_store_callbacks(state, name, startup, teardown);
+
+ if (!invoke || !startup)
+ goto out;
+
+ /*
+ * Try to call the startup callback for each present cpu
+ * depending on the hotplug state of the cpu.
+ */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpustate < state)
+ continue;
+
+ ret = cpuhp_issue_call(cpu, state, startup, true);
+ if (ret) {
+ cpuhp_rollback_install(cpu, state, teardown);
+ cpuhp_store_callbacks(state, NULL, NULL, NULL);
+ goto out;
+ }
+ }
+out:
+ put_online_cpus();
+ if (!ret && dyn_state)
+ return state;
+ return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state: The state to remove
+ * @invoke: If true, the teardown function is invoked for cpus where
+ * cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+ int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+ int cpu;
+
+ BUG_ON(cpuhp_cb_check(state));
+
+ get_online_cpus();
+
+ if (!invoke || !teardown)
+ goto remove;
+
+ /*
+ * Call the teardown callback for each present cpu depending
+ * on the hotplug state of the cpu. This function is not
+ * allowed to fail currently!
+ */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpustate >= state)
+ cpuhp_issue_call(cpu, state, teardown, false);
+ }
+remove:
+ cpuhp_store_callbacks(state, NULL, NULL, NULL);
+ put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
#if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
static ssize_t show_cpuhp_state(struct device *dev,
struct device_attribute *attr, char *buf)
[toc] | [prev] | [next] | [standalone]
| From | tip-bot for Thomas Gleixner <tipbot@zytor.com> |
|---|---|
| Date | 2016-03-01 21:00 +0100 |
| Subject | [tip:smp/hotplug] cpu/hotplug: Implement setup/removal interface |
| Message-ID | <r7YCE-5hG-55@gated-at.bofh.it> |
| In reply to | #1344568 |
Commit-ID: 5b7aa87e0482be768486e0c2277aa4122487eb9d
Gitweb: http://git.kernel.org/tip/5b7aa87e0482be768486e0c2277aa4122487eb9d
Author: Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:33 +0000
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:55 +0100
cpu/hotplug: Implement setup/removal interface
Implement function which allow to setup/remove hotplug state callbacks.
The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.
The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.
This includes rollback to the previous state in case of failure.
A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them
For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.
[ Dynamic registering implemented by Sebastian Siewior ]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.103464877@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpuhotplug.h | 67 ++++++++++++++
kernel/cpu.c | 224 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 291 insertions(+)
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index d9303cc..2993526 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
CPUHP_AP_ONLINE,
CPUHP_TEARDOWN_CPU,
CPUHP_NOTIFY_ONLINE,
+ CPUHP_ONLINE_DYN,
+ CPUHP_ONLINE_DYN_END = CPUHP_ONLINE_DYN + 30,
CPUHP_ONLINE,
};
+int __cpuhp_setup_state(enum cpuhp_state state, const char *name, bool invoke,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state: The state for which the calls are installed
+ * @name: Name of the callback (will be used in debug output)
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ * callbacks
+ * @state: The state for which the calls are installed
+ * @name: Name of the callback.
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state: The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+ __cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ * teardown
+ * @state: The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+ __cpuhp_remove_state(state, false);
+}
+
#endif
diff --git a/kernel/cpu.c b/kernel/cpu.c
index be9335d..b5eacb9 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states[] = {
},
};
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+ if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+ return -EINVAL;
+ return 0;
+}
+
static bool cpuhp_is_ap_state(enum cpuhp_state state)
{
return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
return sp + state;
}
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+ const char *name,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ /* (Un)Install the callbacks for further cpu hotplug operations */
+ struct cpuhp_step *sp;
+
+ mutex_lock(&cpuhp_state_mutex);
+ sp = cpuhp_get_step(state);
+ sp->startup = startup;
+ sp->teardown = teardown;
+ sp->name = name;
+ mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+ return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+ int (*cb)(unsigned int cpu) = __cb;
+
+ BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+ int (*cb)(unsigned int), bool bringup)
+{
+ int ret;
+
+ if (!cb)
+ return 0;
+
+ /*
+ * This invokes the callback directly for now. In a later step we
+ * convert that to use cpuhp_invoke_callback().
+ */
+ if (cpuhp_is_ap_state(state)) {
+ /*
+ * Note, that a function called on the AP is not
+ * allowed to fail.
+ */
+ if (cpu_online(cpu))
+ smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+ return 0;
+ }
+
+ /*
+ * The non AP bound callbacks can fail on bringup. On teardown
+ * e.g. module removal we crash for now.
+ */
+ ret = cb(cpu);
+ BUG_ON(ret && !bringup);
+ return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+ int (*teardown)(unsigned int cpu))
+{
+ int cpu;
+
+ if (!teardown)
+ return;
+
+ /* Roll back the already executed steps on the other cpus */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpu >= failedcpu)
+ break;
+
+ /* Did we invoke the startup call on that cpu ? */
+ if (cpustate >= state)
+ cpuhp_issue_call(cpu, state, teardown, false);
+ }
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+ enum cpuhp_state i;
+
+ mutex_lock(&cpuhp_state_mutex);
+ for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+ if (cpuhp_bp_states[i].name)
+ continue;
+
+ cpuhp_bp_states[i].name = "Reserved";
+ mutex_unlock(&cpuhp_state_mutex);
+ return i;
+ }
+ mutex_unlock(&cpuhp_state_mutex);
+ WARN(1, "No more dynamic states available for CPU hotplug\n");
+ return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state: The state to setup
+ * @invoke: If true, the startup function is invoked for cpus where
+ * cpu state >= @state
+ * @startup: startup callback function
+ * @teardown: teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+ const char *name, bool invoke,
+ int (*startup)(unsigned int cpu),
+ int (*teardown)(unsigned int cpu))
+{
+ int cpu, ret = 0;
+ int dyn_state = 0;
+
+ if (cpuhp_cb_check(state) || !name)
+ return -EINVAL;
+
+ get_online_cpus();
+
+ /* currently assignments for the ONLINE state are possible */
+ if (state == CPUHP_ONLINE_DYN) {
+ dyn_state = 1;
+ ret = cpuhp_reserve_state(state);
+ if (ret < 0)
+ goto out;
+ state = ret;
+ }
+
+ cpuhp_store_callbacks(state, name, startup, teardown);
+
+ if (!invoke || !startup)
+ goto out;
+
+ /*
+ * Try to call the startup callback for each present cpu
+ * depending on the hotplug state of the cpu.
+ */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpustate < state)
+ continue;
+
+ ret = cpuhp_issue_call(cpu, state, startup, true);
+ if (ret) {
+ cpuhp_rollback_install(cpu, state, teardown);
+ cpuhp_store_callbacks(state, NULL, NULL, NULL);
+ goto out;
+ }
+ }
+out:
+ put_online_cpus();
+ if (!ret && dyn_state)
+ return state;
+ return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state: The state to remove
+ * @invoke: If true, the teardown function is invoked for cpus where
+ * cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+ int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+ int cpu;
+
+ BUG_ON(cpuhp_cb_check(state));
+
+ get_online_cpus();
+
+ if (!invoke || !teardown)
+ goto remove;
+
+ /*
+ * Call the teardown callback for each present cpu depending
+ * on the hotplug state of the cpu. This function is not
+ * allowed to fail currently!
+ */
+ for_each_present_cpu(cpu) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int cpustate = st->state;
+
+ if (cpustate >= state)
+ cpuhp_issue_call(cpu, state, teardown, false);
+ }
+remove:
+ cpuhp_store_callbacks(state, NULL, NULL, NULL);
+ put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
#if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
static ssize_t show_cpuhp_state(struct device *dev,
struct device_attribute *attr, char *buf)
[toc] | [prev] | [next] | [standalone]
| From | Thomas Gleixner <tglx@linutronix.de> |
|---|---|
| Date | 2016-02-26 19:50 +0100 |
| Subject | [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core |
| Message-ID | <r6vCH-7sv-29@gated-at.bofh.it> |
| In reply to | #1344561 |
Move the scheduler cpu online notifier part to the hotplug core. This is
anyway the highest priority callback and we need that functionality right now
for the next changes.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpuhotplug.h | 1 +
kernel/cpu.c | 18 ++++++++++++++++++
kernel/sched/core.c | 10 ----------
3 files changed, 19 insertions(+), 10 deletions(-)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,6 +10,7 @@ enum cpuhp_state {
CPUHP_AP_NOTIFY_STARTING,
CPUHP_AP_ONLINE,
CPUHP_TEARDOWN_CPU,
+ CPUHP_CPU_SET_ACTIVE,
CPUHP_NOTIFY_ONLINE,
CPUHP_ONLINE_DYN,
CPUHP_ONLINE_DYN_END = CPUHP_ONLINE_DYN + 30,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -666,6 +666,19 @@ void notify_cpu_starting(unsigned int cp
}
}
+/*
+ * Called from the idle task. We need to set active here, so we can kick off
+ * the stopper thread.
+ */
+static int cpuhp_set_cpu_active(unsigned int cpu)
+{
+ /* The cpu is marked online, set it active now */
+ set_cpu_active(cpu, true);
+ /* Unpark the stopper thread */
+ stop_machine_unpark(cpu);
+ return 0;
+}
+
static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
{
for (st->state--; st->state > st->target; st->state--) {
@@ -941,6 +954,11 @@ static struct cpuhp_step cpuhp_bp_states
.teardown = takedown_cpu,
.cant_stop = true,
},
+ [CPUHP_CPU_SET_ACTIVE] = {
+ .name = "cpu:active",
+ .startup = cpuhp_set_cpu_active,
+ .teardown = NULL,
+ },
[CPUHP_NOTIFY_ONLINE] = {
.name = "notify:online",
.startup = notify_online,
Index: b/kernel/sched/core.c
===================================================================
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5692,16 +5692,6 @@ static int sched_cpu_active(struct notif
set_cpu_rq_start_time();
return NOTIFY_OK;
- case CPU_ONLINE:
- /*
- * At this point a starting CPU has marked itself as online via
- * set_cpu_online(). But it might not yet have marked itself
- * as active, which is essential from here on.
- */
- set_cpu_active(cpu, true);
- stop_machine_unpark(cpu);
- return NOTIFY_OK;
-
case CPU_DOWN_FAILED:
set_cpu_active(cpu, true);
return NOTIFY_OK;
[toc] | [prev] | [next] | [standalone]
Page 1 of 3 [1] 2 3 Next page →
Back to top | Article view | linux.kernel
csiph-web