Groups > linux.kernel > #1731174 > unrolled thread

[patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

Started by	Thomas Gleixner <tglx@linutronix.de>
First post	2017-09-12 21:50 +0200
Last post	2017-09-14 10:20 +0200
Articles	19 on this page of 59 — 6 participants

Back to article view | Back to linux.kernel

  [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace  duct tape Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 21:50 +0200
    [patch V2 14/29] lockup_detector: Split out cpumask write function Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 21:50 +0200
      [tip:core/urgent] watchdog/core: Split out cpumask write function tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 05/29] lockup_detector: Remove broken suspend/resume  interfaces Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 21:50 +0200
      [tip:core/urgent] watchdog/core: Remove broken suspend/resume  interfaces tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 16/29] lockup_detector: Create new thread handling  infrastructure Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 21:50 +0200
      [tip:core/urgent] watchdog/core: Create new thread handling  infrastructure tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 01/29] hardlockup_detector: Provide interface to  stop/restart perf events Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 21:50 +0200
      [tip:core/urgent] watchdog/hardlockup: Provide interface to  stop/restart perf events tip-bot for Peter Zijlstra <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 26/29] lockup_detector/perf: Implement CPU enable  replacement Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Implement CPU enable  replacement tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 06/29] lockup_detector: Rework cpu hotplug locking Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Rework CPU hotplug locking tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 28/29] lockup_detector/perf: Simplify deferred event destroy Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Simplify deferred event  destroy tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 04/29] parisc: Use lockup_detector_stop() Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      Re: [patch V2 04/29] parisc: Use lockup_detector_stop() Helge Deller <deller@gmx.de> - 2017-09-14 11:10 +0200
        Re: [patch V2 04/29] parisc: Use lockup_detector_stop() Don Zickus <dzickus@redhat.com> - 2017-09-14 15:50 +0200
      [tip:core/urgent] parisc, watchdog/core: Use lockup_detector_stop() tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 17/29] lockup_detector: Get rid of the thread  teardown/setup dance Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Get rid of the thread  teardown/setup dance tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 03/29] lockup_detector: Provide interface to stop from  poweroff() Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Provide interface to stop from  poweroff() tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 10/29] lockup_detector/perf: Prevent cpu hotplug deadlock Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Prevent CPU hotplug  deadlock tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 13/29] lockup_detector: Cleanup the ifdef maze Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Clean up the #ifdef maze tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 29/29] lockup_detector: Cleanup hotplug locking mess Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup: Clean up hotplug locking  mess tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 09/29] lockup_detector/perf: Remove broken self disable on  failure Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Remove broken self  disable on failure tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 08/29] lockup_detector: Mark hardlockup_detector_disable()  __init Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Mark hardlockup_detector_disable()  __init tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 12:50 +0200
    [patch V2 18/29] lockup_detector: Further simplify sysctl handling Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Further simplify sysctl handling tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 12/29] lockup_detector: Cleanup stub functions Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Clean up stub functions tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 15/29] smpboot/threads: Avoid runtime allocation Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] smpboot/threads, watchdog/core: Avoid runtime  allocation tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 20/29] lockup_detector/sysctl: Get rid of the ifdeffery Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/sysctl: Get rid of the #ifdeffery tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 27/29] lockup_detector: Use new perf CPU enable mechanism Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Use new perf CPU enable  mechanism tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 24/29] lockup_detector/perf: Implement init time perf  validation Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Implement init time  perf validation tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 19/29] lockup_detector: Cleanup header mess Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Clean up header mess tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 22/29] lockup_detector: Make watchdog_nmi_reconfigure() two  stage Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core, powerpc: Make  watchdog_nmi_reconfigure() two stage tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 25/29] lockup_detector: Implement init time detection of  perf Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      Re: [patch V2 25/29] lockup_detector: Implement init time detection  of perf Don Zickus <dzickus@redhat.com> - 2017-09-13 20:10 +0200
        Re: [patch V2 25/29] lockup_detector: Implement init time detection  of perf Thomas Gleixner <tglx@linutronix.de> - 2017-09-13 20:10 +0200
          Re: [patch V2 25/29] lockup_detector: Implement init time detection  of perf Ingo Molnar <mingo@kernel.org> - 2017-09-14 07:30 +0200
      [tip:core/urgent] watchdog/hardlockup/perf: Implement init time  detection of perf tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    [patch V2 23/29] lockup_detector: Get rid of the racy update loop Thomas Gleixner <tglx@linutronix.de> - 2017-09-12 22:00 +0200
      [tip:core/urgent] watchdog/core: Get rid of the racy update loop tip-bot for Thomas Gleixner <tipbot@zytor.com> - 2017-09-14 13:00 +0200
    Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Don Zickus <dzickus@redhat.com> - 2017-09-13 20:10 +0200
      Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Ingo Molnar <mingo@kernel.org> - 2017-09-14 07:30 +0200
      Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Thomas Gleixner <tglx@linutronix.de> - 2017-09-14 10:20 +0200

Page 3 of 3 — ← Prev page 1 2 [3]

#1732213 — [tip:core/urgent] watchdog/sysctl: Get rid of the #ifdeffery

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/sysctl: Get rid of the #ifdeffery
Message-ID	<upA2f-3WF-49@gated-at.bofh.it>
In reply to	#1731200

Commit-ID:  51d4052b01ca555e0d1d5fe297b309beb6c64aa0
Gitweb:     http://git.kernel.org/tip/51d4052b01ca555e0d1d5fe297b309beb6c64aa0
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:14 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:07 +0200

watchdog/sysctl: Get rid of the #ifdeffery

The sysctl of the nmi_watchdog file prevents writes by setting:

    min = max = 0

if none of the users is enabled. That involves ifdeffery and is competely
non obvious.

If none of the facilities is enabeld, then the file can simply be made read
only. Move the ifdeffery into the header and use a constant for file
permissions.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194147.706073616@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 include/linux/nmi.h | 6 ++++++
 kernel/sysctl.c     | 6 +-----
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/include/linux/nmi.h b/include/linux/nmi.h
index cfebb3b..5774b443 100644
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -81,6 +81,12 @@ extern unsigned int hardlockup_panic;
 static inline void hardlockup_detector_disable(void) {}
 #endif
 
+#if defined(CONFIG_HAVE_NMI_WATCHDOG) || defined(CONFIG_HARDLOCKUP_DETECTOR)
+# define NMI_WATCHDOG_SYSCTL_PERM	0644
+#else
+# define NMI_WATCHDOG_SYSCTL_PERM	0444
+#endif
+
 #if defined(CONFIG_HARDLOCKUP_DETECTOR_PERF)
 extern void arch_touch_nmi_watchdog(void);
 extern void hardlockup_detector_perf_stop(void);
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 6648fbb..539cb4e 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -891,14 +891,10 @@ static struct ctl_table kern_table[] = {
 		.procname       = "nmi_watchdog",
 		.data           = &nmi_watchdog_enabled,
 		.maxlen         = sizeof (int),
-		.mode           = 0644,
+		.mode		= NMI_WATCHDOG_SYSCTL_PERM,
 		.proc_handler   = proc_nmi_watchdog,
 		.extra1		= &zero,
-#if defined(CONFIG_HAVE_NMI_WATCHDOG) || defined(CONFIG_HARDLOCKUP_DETECTOR)
 		.extra2		= &one,
-#else
-		.extra2		= &zero,
-#endif
 	},
 	{
 		.procname	= "watchdog_cpumask",

[toc] | [prev] | [next] | [standalone]

#1731201 — [patch V2 27/29] lockup_detector: Use new perf CPU enable mechanism

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 27/29] lockup_detector: Use new perf CPU enable mechanism
Message-ID	<uoZvJ-5zK-51@gated-at.bofh.it>
In reply to	#1731174

Get rid of the hodgepodge which tries to be smart about perf being
unavailable and error printout rate limiting.

That's all not required simply because this is never invoked when the perf
NMI watchdog is not functional.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20170831073055.173111470@linutronix.de

---
 kernel/watchdog.c     |    4 +-
 kernel/watchdog_hld.c |   88 ++------------------------------------------------
 2 files changed, 8 insertions(+), 84 deletions(-)

--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -107,6 +107,7 @@ static int __init hardlockup_all_cpu_bac
  */
 int __weak watchdog_nmi_enable(unsigned int cpu)
 {
+	hardlockup_detector_perf_enable();
 	return 0;
 }
 
@@ -465,7 +466,8 @@ static void watchdog_enable(unsigned int
 	/* Initialize timestamp */
 	__touch_watchdog();
 	/* Enable the perf event */
-	watchdog_nmi_enable(cpu);
+	if (watchdog_enabled & NMI_WATCHDOG_ENABLED)
+		watchdog_nmi_enable(cpu);
 
 	watchdog_set_prio(SCHED_FIFO, MAX_RT_PRIO - 1);
 }
--- a/kernel/watchdog_hld.c
+++ b/kernel/watchdog_hld.c
@@ -25,7 +25,7 @@ static DEFINE_PER_CPU(struct perf_event
 static struct cpumask dead_events_mask;
 
 static unsigned long hardlockup_allcpu_dumped;
-static bool hardlockup_detector_disabled;
+static unsigned int watchdog_cpus;
 
 void arch_touch_nmi_watchdog(void)
 {
@@ -160,84 +160,6 @@ static void watchdog_overflow_callback(s
 	return;
 }
 
-/*
- * People like the simple clean cpu node info on boot.
- * Reduce the watchdog noise by only printing messages
- * that are different from what cpu0 displayed.
- */
-static unsigned long firstcpu_err;
-static atomic_t watchdog_cpus;
-
-int watchdog_nmi_enable(unsigned int cpu)
-{
-	struct perf_event_attr *wd_attr;
-	struct perf_event *event = per_cpu(watchdog_ev, cpu);
-	int firstcpu = 0;
-
-	/* nothing to do if the hard lockup detector is disabled */
-	if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
-		goto out;
-
-	/* A failure disabled the hardlockup detector permanently */
-	if (hardlockup_detector_disabled)
-		return -ENODEV;
-
-	/* is it already setup and enabled? */
-	if (event && event->state > PERF_EVENT_STATE_OFF)
-		goto out;
-
-	/* it is setup but not enabled */
-	if (event != NULL)
-		goto out_enable;
-
-	if (atomic_inc_return(&watchdog_cpus) == 1)
-		firstcpu = 1;
-
-	wd_attr = &wd_hw_attr;
-	wd_attr->sample_period = hw_nmi_get_sample_period(watchdog_thresh);
-
-	/* Try to register using hardware perf events */
-	event = perf_event_create_kernel_counter(wd_attr, cpu, NULL, watchdog_overflow_callback, NULL);
-
-	/* save the first cpu's error for future comparision */
-	if (firstcpu && IS_ERR(event))
-		firstcpu_err = PTR_ERR(event);
-
-	if (!IS_ERR(event)) {
-		/* only print for the first cpu initialized */
-		if (firstcpu || firstcpu_err)
-			pr_info("enabled on all CPUs, permanently consumes one hw-PMU counter.\n");
-		goto out_save;
-	}
-
-	/* skip displaying the same error again */
-	if (!firstcpu && (PTR_ERR(event) == firstcpu_err))
-		return PTR_ERR(event);
-
-	/* vary the KERN level based on the returned errno */
-	if (PTR_ERR(event) == -EOPNOTSUPP)
-		pr_info("disabled (cpu%i): not supported (no LAPIC?)\n", cpu);
-	else if (PTR_ERR(event) == -ENOENT)
-		pr_warn("disabled (cpu%i): hardware events not enabled\n",
-			 cpu);
-	else
-		pr_err("disabled (cpu%i): unable to create perf event: %ld\n",
-			cpu, PTR_ERR(event));
-
-	pr_info("Disabling hard lockup detector permanently\n");
-	hardlockup_detector_disabled = true;
-
-	return PTR_ERR(event);
-
-	/* success path */
-out_save:
-	per_cpu(watchdog_ev, cpu) = event;
-out_enable:
-	perf_event_enable(per_cpu(watchdog_ev, cpu));
-out:
-	return 0;
-}
-
 static int hardlockup_detector_event_create(void)
 {
 	unsigned int cpu = smp_processor_id();
@@ -267,6 +189,9 @@ void hardlockup_detector_perf_enable(voi
 	if (hardlockup_detector_event_create())
 		return;
 
+	if (!watchdog_cpus++)
+		pr_info("Enabled. Permanently consumes one hw-PMU counter.\n");
+
 	perf_event_enable(this_cpu_read(watchdog_ev));
 }
 
@@ -282,10 +207,7 @@ void hardlockup_detector_perf_disable(vo
 		this_cpu_write(watchdog_ev, NULL);
 		this_cpu_write(dead_event, event);
 		cpumask_set_cpu(smp_processor_id(), &dead_events_mask);
-
-		/* watchdog_nmi_enable() expects this to be zero initially. */
-		if (atomic_dec_and_test(&watchdog_cpus))
-			firstcpu_err = 0;
+		watchdog_cpus--;
 	}
 }

[toc] | [prev] | [next] | [standalone]

#1732199 — [tip:core/urgent] watchdog/hardlockup/perf: Use new perf CPU enable mechanism

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/hardlockup/perf: Use new perf CPU enable mechanism
Message-ID	<upA2e-3WF-17@gated-at.bofh.it>
In reply to	#1731201

Commit-ID:  146c9d0e9dfdb62ed6afd43cc263efafbbfd1dcf
Gitweb:     http://git.kernel.org/tip/146c9d0e9dfdb62ed6afd43cc263efafbbfd1dcf
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:21 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:08 +0200

watchdog/hardlockup/perf: Use new perf CPU enable mechanism

Get rid of the hodgepodge which tries to be smart about perf being
unavailable and error printout rate limiting.

That's all not required simply because this is never invoked when the perf
NMI watchdog is not functional.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194148.259651788@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/watchdog.c     |  4 ++-
 kernel/watchdog_hld.c | 88 +++------------------------------------------------
 2 files changed, 8 insertions(+), 84 deletions(-)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index fd8a998..5eb1196 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -107,6 +107,7 @@ __setup("hardlockup_all_cpu_backtrace=", hardlockup_all_cpu_backtrace_setup);
  */
 int __weak watchdog_nmi_enable(unsigned int cpu)
 {
+	hardlockup_detector_perf_enable();
 	return 0;
 }
 
@@ -465,7 +466,8 @@ static void watchdog_enable(unsigned int cpu)
 	/* Initialize timestamp */
 	__touch_watchdog();
 	/* Enable the perf event */
-	watchdog_nmi_enable(cpu);
+	if (watchdog_enabled & NMI_WATCHDOG_ENABLED)
+		watchdog_nmi_enable(cpu);
 
 	watchdog_set_prio(SCHED_FIFO, MAX_RT_PRIO - 1);
 }
diff --git a/kernel/watchdog_hld.c b/kernel/watchdog_hld.c
index 99a3f22..509bb6b 100644
--- a/kernel/watchdog_hld.c
+++ b/kernel/watchdog_hld.c
@@ -25,7 +25,7 @@ static DEFINE_PER_CPU(struct perf_event *, dead_event);
 static struct cpumask dead_events_mask;
 
 static unsigned long hardlockup_allcpu_dumped;
-static bool hardlockup_detector_disabled;
+static unsigned int watchdog_cpus;
 
 void arch_touch_nmi_watchdog(void)
 {
@@ -160,84 +160,6 @@ static void watchdog_overflow_callback(struct perf_event *event,
 	return;
 }
 
-/*
- * People like the simple clean cpu node info on boot.
- * Reduce the watchdog noise by only printing messages
- * that are different from what cpu0 displayed.
- */
-static unsigned long firstcpu_err;
-static atomic_t watchdog_cpus;
-
-int watchdog_nmi_enable(unsigned int cpu)
-{
-	struct perf_event_attr *wd_attr;
-	struct perf_event *event = per_cpu(watchdog_ev, cpu);
-	int firstcpu = 0;
-
-	/* nothing to do if the hard lockup detector is disabled */
-	if (!(watchdog_enabled & NMI_WATCHDOG_ENABLED))
-		goto out;
-
-	/* A failure disabled the hardlockup detector permanently */
-	if (hardlockup_detector_disabled)
-		return -ENODEV;
-
-	/* is it already setup and enabled? */
-	if (event && event->state > PERF_EVENT_STATE_OFF)
-		goto out;
-
-	/* it is setup but not enabled */
-	if (event != NULL)
-		goto out_enable;
-
-	if (atomic_inc_return(&watchdog_cpus) == 1)
-		firstcpu = 1;
-
-	wd_attr = &wd_hw_attr;
-	wd_attr->sample_period = hw_nmi_get_sample_period(watchdog_thresh);
-
-	/* Try to register using hardware perf events */
-	event = perf_event_create_kernel_counter(wd_attr, cpu, NULL, watchdog_overflow_callback, NULL);
-
-	/* save the first cpu's error for future comparision */
-	if (firstcpu && IS_ERR(event))
-		firstcpu_err = PTR_ERR(event);
-
-	if (!IS_ERR(event)) {
-		/* only print for the first cpu initialized */
-		if (firstcpu || firstcpu_err)
-			pr_info("enabled on all CPUs, permanently consumes one hw-PMU counter.\n");
-		goto out_save;
-	}
-
-	/* skip displaying the same error again */
-	if (!firstcpu && (PTR_ERR(event) == firstcpu_err))
-		return PTR_ERR(event);
-
-	/* vary the KERN level based on the returned errno */
-	if (PTR_ERR(event) == -EOPNOTSUPP)
-		pr_info("disabled (cpu%i): not supported (no LAPIC?)\n", cpu);
-	else if (PTR_ERR(event) == -ENOENT)
-		pr_warn("disabled (cpu%i): hardware events not enabled\n",
-			 cpu);
-	else
-		pr_err("disabled (cpu%i): unable to create perf event: %ld\n",
-			cpu, PTR_ERR(event));
-
-	pr_info("Disabling hard lockup detector permanently\n");
-	hardlockup_detector_disabled = true;
-
-	return PTR_ERR(event);
-
-	/* success path */
-out_save:
-	per_cpu(watchdog_ev, cpu) = event;
-out_enable:
-	perf_event_enable(per_cpu(watchdog_ev, cpu));
-out:
-	return 0;
-}
-
 static int hardlockup_detector_event_create(void)
 {
 	unsigned int cpu = smp_processor_id();
@@ -267,6 +189,9 @@ void hardlockup_detector_perf_enable(void)
 	if (hardlockup_detector_event_create())
 		return;
 
+	if (!watchdog_cpus++)
+		pr_info("Enabled. Permanently consumes one hw-PMU counter.\n");
+
 	perf_event_enable(this_cpu_read(watchdog_ev));
 }
 
@@ -282,10 +207,7 @@ void hardlockup_detector_perf_disable(void)
 		this_cpu_write(watchdog_ev, NULL);
 		this_cpu_write(dead_event, event);
 		cpumask_set_cpu(smp_processor_id(), &dead_events_mask);
-
-		/* watchdog_nmi_enable() expects this to be zero initially. */
-		if (atomic_dec_and_test(&watchdog_cpus))
-			firstcpu_err = 0;
+		watchdog_cpus--;
 	}
 }

[toc] | [prev] | [next] | [standalone]

#1731202 — [patch V2 24/29] lockup_detector/perf: Implement init time perf validation

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 24/29] lockup_detector/perf: Implement init time perf validation
Message-ID	<uoZvJ-5zK-47@gated-at.bofh.it>
In reply to	#1731174

The watchdog tries to create perf events even after it figured out that
perf is not functional or the requested event is not supported.

That's braindead as this can be done once at init time and if not supported
the NMI watchdog can be turned off unconditonally.

Implement the perf hardlockup detector functionality for that. This creates
a new event create function, which will replace the unholy mess of the
existing one in later patches.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20170831073054.911861725@linutronix.de

---
 include/linux/nmi.h   |    8 ++++++--
 kernel/watchdog_hld.c |   37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 2 deletions(-)

--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -93,14 +93,18 @@ extern void hardlockup_detector_perf_sto
 extern void hardlockup_detector_perf_restart(void);
 extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_cleanup(void);
+extern int hardlockup_detector_perf_init(void);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
-#if !defined(CONFIG_HAVE_NMI_WATCHDOG)
+# if !defined(CONFIG_HAVE_NMI_WATCHDOG)
+static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
-#endif
+# else
+static int hardlockup_detector_perf_init(void) { return 0; }
+# endif
 #endif
 
 void watchdog_nmi_reconfigure(bool run);
--- a/kernel/watchdog_hld.c
+++ b/kernel/watchdog_hld.c
@@ -238,6 +238,27 @@ int watchdog_nmi_enable(unsigned int cpu
 	return 0;
 }
 
+static int hardlockup_detector_event_create(void)
+{
+	unsigned int cpu = smp_processor_id();
+	struct perf_event_attr *wd_attr;
+	struct perf_event *evt;
+
+	wd_attr = &wd_hw_attr;
+	wd_attr->sample_period = hw_nmi_get_sample_period(watchdog_thresh);
+
+	/* Try to register using hardware perf events */
+	evt = perf_event_create_kernel_counter(wd_attr, cpu, NULL,
+					       watchdog_overflow_callback, NULL);
+	if (IS_ERR(evt)) {
+		pr_info("Perf event create on CPU %d failed with %ld\n", cpu,
+			PTR_ERR(evt));
+		return PTR_ERR(evt);
+	}
+	this_cpu_write(watchdog_ev, evt);
+	return 0;
+}
+
 /**
  * hardlockup_detector_perf_disable - Disable the local event
  */
@@ -315,3 +336,19 @@ void __init hardlockup_detector_perf_res
 			perf_event_enable(event);
 	}
 }
+
+/**
+ * hardlockup_detector_perf_init - Probe whether NMI event is available at all
+ */
+int __init hardlockup_detector_perf_init(void)
+{
+	int ret = hardlockup_detector_event_create();
+
+	if (ret) {
+		pr_info("Perf NMI watchdog permanetely disabled\n");
+	} else {
+		perf_event_release_kernel(this_cpu_read(watchdog_ev));
+		this_cpu_write(watchdog_ev, NULL);
+	}
+	return ret;
+}

[toc] | [prev] | [next] | [standalone]

#1732193 — [tip:core/urgent] watchdog/hardlockup/perf: Implement init time perf validation

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/hardlockup/perf: Implement init time perf validation
Message-ID	<upA2d-3WF-1@gated-at.bofh.it>
In reply to	#1731202

Commit-ID:  178b9f7a36d2c74a38274b66dd89f53611298a19
Gitweb:     http://git.kernel.org/tip/178b9f7a36d2c74a38274b66dd89f53611298a19
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:18 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:08 +0200

watchdog/hardlockup/perf: Implement init time perf validation

The watchdog tries to create perf events even after it figured out that
perf is not functional or the requested event is not supported.

That's braindead as this can be done once at init time and if not supported
the NMI watchdog can be turned off unconditonally.

Implement the perf hardlockup detector functionality for that. This creates
a new event create function, which will replace the unholy mess of the
existing one in later patches.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194148.019090547@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 include/linux/nmi.h   |  8 ++++++--
 kernel/watchdog_hld.c | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/include/linux/nmi.h b/include/linux/nmi.h
index eee255b..72c62a8 100644
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -93,14 +93,18 @@ extern void hardlockup_detector_perf_stop(void);
 extern void hardlockup_detector_perf_restart(void);
 extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_cleanup(void);
+extern int hardlockup_detector_perf_init(void);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
-#if !defined(CONFIG_HAVE_NMI_WATCHDOG)
+# if !defined(CONFIG_HAVE_NMI_WATCHDOG)
+static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
-#endif
+# else
+static inline int hardlockup_detector_perf_init(void) { return 0; }
+# endif
 #endif
 
 void watchdog_nmi_reconfigure(bool run);
diff --git a/kernel/watchdog_hld.c b/kernel/watchdog_hld.c
index 0aa191e..f7e752e 100644
--- a/kernel/watchdog_hld.c
+++ b/kernel/watchdog_hld.c
@@ -238,6 +238,27 @@ out:
 	return 0;
 }
 
+static int hardlockup_detector_event_create(void)
+{
+	unsigned int cpu = smp_processor_id();
+	struct perf_event_attr *wd_attr;
+	struct perf_event *evt;
+
+	wd_attr = &wd_hw_attr;
+	wd_attr->sample_period = hw_nmi_get_sample_period(watchdog_thresh);
+
+	/* Try to register using hardware perf events */
+	evt = perf_event_create_kernel_counter(wd_attr, cpu, NULL,
+					       watchdog_overflow_callback, NULL);
+	if (IS_ERR(evt)) {
+		pr_info("Perf event create on CPU %d failed with %ld\n", cpu,
+			PTR_ERR(evt));
+		return PTR_ERR(evt);
+	}
+	this_cpu_write(watchdog_ev, evt);
+	return 0;
+}
+
 /**
  * hardlockup_detector_perf_disable - Disable the local event
  */
@@ -315,3 +336,19 @@ void __init hardlockup_detector_perf_restart(void)
 			perf_event_enable(event);
 	}
 }
+
+/**
+ * hardlockup_detector_perf_init - Probe whether NMI event is available at all
+ */
+int __init hardlockup_detector_perf_init(void)
+{
+	int ret = hardlockup_detector_event_create();
+
+	if (ret) {
+		pr_info("Perf NMI watchdog permanetely disabled\n");
+	} else {
+		perf_event_release_kernel(this_cpu_read(watchdog_ev));
+		this_cpu_write(watchdog_ev, NULL);
+	}
+	return ret;
+}

[toc] | [prev] | [next] | [standalone]

#1731203 — [patch V2 19/29] lockup_detector: Cleanup header mess

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 19/29] lockup_detector: Cleanup header mess
Message-ID	<uoZvJ-5zK-59@gated-at.bofh.it>
In reply to	#1731174

Having the same #ifdef in various places does not make it more
readable. Collect stuff into one place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20170831073054.505213255@linutronix.de

---
 include/linux/nmi.h |   60 +++++++++++++++++++++-------------------------------
 1 file changed, 25 insertions(+), 35 deletions(-)

--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -14,11 +14,29 @@
 void lockup_detector_init(void);
 void lockup_detector_soft_poweroff(void);
 void lockup_detector_cleanup(void);
+bool is_hardlockup(void);
+
+extern int watchdog_user_enabled;
+extern int nmi_watchdog_enabled;
+extern int soft_watchdog_enabled;
+extern int watchdog_thresh;
+extern unsigned long watchdog_enabled;
+
+extern struct cpumask watchdog_cpumask;
+extern unsigned long *watchdog_cpumask_bits;
+#ifdef CONFIG_SMP
+extern int sysctl_softlockup_all_cpu_backtrace;
+extern int sysctl_hardlockup_all_cpu_backtrace;
 #else
+#define sysctl_softlockup_all_cpu_backtrace 0
+#define sysctl_hardlockup_all_cpu_backtrace 0
+#endif /* !CONFIG_SMP */
+
+#else /* CONFIG_LOCKUP_DETECTOR */
 static inline void lockup_detector_init(void) { }
 static inline void lockup_detector_soft_poweroff(void) { }
 static inline void lockup_detector_cleanup(void) { }
-#endif
+#endif /* !CONFIG_LOCKUP_DETECTOR */
 
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 extern void touch_softlockup_watchdog_sched(void);
@@ -26,28 +44,17 @@ extern void touch_softlockup_watchdog(vo
 extern void touch_softlockup_watchdog_sync(void);
 extern void touch_all_softlockup_watchdogs(void);
 extern unsigned int  softlockup_panic;
-extern int soft_watchdog_enabled;
 #else
-static inline void touch_softlockup_watchdog_sched(void)
-{
-}
-static inline void touch_softlockup_watchdog(void)
-{
-}
-static inline void touch_softlockup_watchdog_sync(void)
-{
-}
-static inline void touch_all_softlockup_watchdogs(void)
-{
-}
+static inline void touch_softlockup_watchdog_sched(void) { }
+static inline void touch_softlockup_watchdog(void) { }
+static inline void touch_softlockup_watchdog_sync(void) { }
+static inline void touch_all_softlockup_watchdogs(void) { }
 #endif
 
 #ifdef CONFIG_DETECT_HUNG_TASK
 void reset_hung_task_detector(void);
 #else
-static inline void reset_hung_task_detector(void)
-{
-}
+static inline void reset_hung_task_detector(void) { }
 #endif
 
 /*
@@ -92,7 +99,7 @@ static inline void arch_touch_nmi_watchd
 
 /**
  * touch_nmi_watchdog - restart NMI watchdog timeout.
- * 
+ *
  * If the architecture supports the NMI watchdog, touch_nmi_watchdog()
  * may be used to reset the timeout - for code which intentionally
  * disables interrupts for a long time. This call is stateless.
@@ -162,21 +169,6 @@ static inline bool trigger_single_cpu_ba
 u64 hw_nmi_get_sample_period(int watchdog_thresh);
 #endif
 
-#ifdef CONFIG_LOCKUP_DETECTOR
-extern int nmi_watchdog_enabled;
-extern int watchdog_user_enabled;
-extern int watchdog_thresh;
-extern unsigned long watchdog_enabled;
-extern struct cpumask watchdog_cpumask;
-extern unsigned long *watchdog_cpumask_bits;
-#ifdef CONFIG_SMP
-extern int sysctl_softlockup_all_cpu_backtrace;
-extern int sysctl_hardlockup_all_cpu_backtrace;
-#else
-#define sysctl_softlockup_all_cpu_backtrace 0
-#define sysctl_hardlockup_all_cpu_backtrace 0
-#endif
-
 #if defined(CONFIG_HARDLOCKUP_CHECK_TIMESTAMP) && \
     defined(CONFIG_HARDLOCKUP_DETECTOR)
 void watchdog_update_hrtimer_threshold(u64 period);
@@ -184,7 +176,6 @@ void watchdog_update_hrtimer_threshold(u
 static inline void watchdog_update_hrtimer_threshold(u64 period) { }
 #endif
 
-extern bool is_hardlockup(void);
 struct ctl_table;
 extern int proc_watchdog(struct ctl_table *, int ,
 			 void __user *, size_t *, loff_t *);
@@ -196,7 +187,6 @@ extern int proc_watchdog_thresh(struct c
 				void __user *, size_t *, loff_t *);
 extern int proc_watchdog_cpumask(struct ctl_table *, int,
 				 void __user *, size_t *, loff_t *);
-#endif
 
 #ifdef CONFIG_HAVE_ACPI_APEI_NMI
 #include <asm/nmi.h>

[toc] | [prev] | [next] | [standalone]

#1732209 — [tip:core/urgent] watchdog/core: Clean up header mess

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/core: Clean up header mess
Message-ID	<upA2f-3WF-39@gated-at.bofh.it>
In reply to	#1731203

Commit-ID:  3b371b5936e7777c819619c00ca60f196a8e13fa
Gitweb:     http://git.kernel.org/tip/3b371b5936e7777c819619c00ca60f196a8e13fa
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:13 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:07 +0200

watchdog/core: Clean up header mess

Having the same #ifdef in various places does not make it more
readable. Collect stuff into one place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194147.627096864@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 include/linux/nmi.h | 60 ++++++++++++++++++++++-------------------------------
 1 file changed, 25 insertions(+), 35 deletions(-)

diff --git a/include/linux/nmi.h b/include/linux/nmi.h
index 91a3a4a..cfebb3b 100644
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -14,11 +14,29 @@
 void lockup_detector_init(void);
 void lockup_detector_soft_poweroff(void);
 void lockup_detector_cleanup(void);
+bool is_hardlockup(void);
+
+extern int watchdog_user_enabled;
+extern int nmi_watchdog_enabled;
+extern int soft_watchdog_enabled;
+extern int watchdog_thresh;
+extern unsigned long watchdog_enabled;
+
+extern struct cpumask watchdog_cpumask;
+extern unsigned long *watchdog_cpumask_bits;
+#ifdef CONFIG_SMP
+extern int sysctl_softlockup_all_cpu_backtrace;
+extern int sysctl_hardlockup_all_cpu_backtrace;
 #else
+#define sysctl_softlockup_all_cpu_backtrace 0
+#define sysctl_hardlockup_all_cpu_backtrace 0
+#endif /* !CONFIG_SMP */
+
+#else /* CONFIG_LOCKUP_DETECTOR */
 static inline void lockup_detector_init(void) { }
 static inline void lockup_detector_soft_poweroff(void) { }
 static inline void lockup_detector_cleanup(void) { }
-#endif
+#endif /* !CONFIG_LOCKUP_DETECTOR */
 
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 extern void touch_softlockup_watchdog_sched(void);
@@ -26,28 +44,17 @@ extern void touch_softlockup_watchdog(void);
 extern void touch_softlockup_watchdog_sync(void);
 extern void touch_all_softlockup_watchdogs(void);
 extern unsigned int  softlockup_panic;
-extern int soft_watchdog_enabled;
 #else
-static inline void touch_softlockup_watchdog_sched(void)
-{
-}
-static inline void touch_softlockup_watchdog(void)
-{
-}
-static inline void touch_softlockup_watchdog_sync(void)
-{
-}
-static inline void touch_all_softlockup_watchdogs(void)
-{
-}
+static inline void touch_softlockup_watchdog_sched(void) { }
+static inline void touch_softlockup_watchdog(void) { }
+static inline void touch_softlockup_watchdog_sync(void) { }
+static inline void touch_all_softlockup_watchdogs(void) { }
 #endif
 
 #ifdef CONFIG_DETECT_HUNG_TASK
 void reset_hung_task_detector(void);
 #else
-static inline void reset_hung_task_detector(void)
-{
-}
+static inline void reset_hung_task_detector(void) { }
 #endif
 
 /*
@@ -92,7 +99,7 @@ static inline void arch_touch_nmi_watchdog(void) {}
 
 /**
  * touch_nmi_watchdog - restart NMI watchdog timeout.
- * 
+ *
  * If the architecture supports the NMI watchdog, touch_nmi_watchdog()
  * may be used to reset the timeout - for code which intentionally
  * disables interrupts for a long time. This call is stateless.
@@ -162,21 +169,6 @@ static inline bool trigger_single_cpu_backtrace(int cpu)
 u64 hw_nmi_get_sample_period(int watchdog_thresh);
 #endif
 
-#ifdef CONFIG_LOCKUP_DETECTOR
-extern int nmi_watchdog_enabled;
-extern int watchdog_user_enabled;
-extern int watchdog_thresh;
-extern unsigned long watchdog_enabled;
-extern struct cpumask watchdog_cpumask;
-extern unsigned long *watchdog_cpumask_bits;
-#ifdef CONFIG_SMP
-extern int sysctl_softlockup_all_cpu_backtrace;
-extern int sysctl_hardlockup_all_cpu_backtrace;
-#else
-#define sysctl_softlockup_all_cpu_backtrace 0
-#define sysctl_hardlockup_all_cpu_backtrace 0
-#endif
-
 #if defined(CONFIG_HARDLOCKUP_CHECK_TIMESTAMP) && \
     defined(CONFIG_HARDLOCKUP_DETECTOR)
 void watchdog_update_hrtimer_threshold(u64 period);
@@ -184,7 +176,6 @@ void watchdog_update_hrtimer_threshold(u64 period);
 static inline void watchdog_update_hrtimer_threshold(u64 period) { }
 #endif
 
-extern bool is_hardlockup(void);
 struct ctl_table;
 extern int proc_watchdog(struct ctl_table *, int ,
 			 void __user *, size_t *, loff_t *);
@@ -196,7 +187,6 @@ extern int proc_watchdog_thresh(struct ctl_table *, int ,
 				void __user *, size_t *, loff_t *);
 extern int proc_watchdog_cpumask(struct ctl_table *, int,
 				 void __user *, size_t *, loff_t *);
-#endif
 
 #ifdef CONFIG_HAVE_ACPI_APEI_NMI
 #include <asm/nmi.h>

[toc] | [prev] | [next] | [standalone]

#1731205 — [patch V2 22/29] lockup_detector: Make watchdog_nmi_reconfigure() two stage

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 22/29] lockup_detector: Make watchdog_nmi_reconfigure() two stage
Message-ID	<uoZvJ-5zK-61@gated-at.bofh.it>
In reply to	#1731174

Both the perf reconfiguration and the powerpc watchdog_nmi_reconfigure()
need to be done in two steps.

     1) Stop all NMIs
     2) Read the new parameters and start NMIs

Right now watchdog_nmi_reconfigure() is a combination of both. To allow a
clean reconfiguration add a 'run' argument and split the functionality in
powerpc.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/20170831073054.740462115@linutronix.de

---
 arch/powerpc/kernel/watchdog.c |   17 +++++++++--------
 include/linux/nmi.h            |    2 ++
 kernel/watchdog.c              |   31 ++++++++++++++++++++++---------
 3 files changed, 33 insertions(+), 17 deletions(-)

--- a/arch/powerpc/kernel/watchdog.c
+++ b/arch/powerpc/kernel/watchdog.c
@@ -355,17 +355,18 @@ static void watchdog_calc_timeouts(void)
 	wd_timer_period_ms = watchdog_thresh * 1000 * 2 / 5;
 }
 
-void watchdog_nmi_reconfigure(void)
+void watchdog_nmi_reconfigure(bool run)
 {
 	int cpu;
 
-	watchdog_calc_timeouts();
-
-	for_each_cpu(cpu, &wd_cpus_enabled)
-		stop_wd_on_cpu(cpu);
-
-	for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask)
-		start_wd_on_cpu(cpu);
+	if (!run) {
+		for_each_cpu(cpu, &wd_cpus_enabled)
+			stop_wd_on_cpu(cpu);
+	} else {
+		watchdog_calc_timeouts();
+		for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask)
+			start_wd_on_cpu(cpu);
+	}
 }
 
 /*
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -103,6 +103,8 @@ static inline void arch_touch_nmi_watchd
 #endif
 #endif
 
+void watchdog_nmi_reconfigure(bool run);
+
 /**
  * touch_nmi_watchdog - restart NMI watchdog timeout.
  *
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -112,17 +112,25 @@ void __weak watchdog_nmi_disable(unsigne
 	hardlockup_detector_perf_disable();
 }
 
-/*
- * watchdog_nmi_reconfigure can be implemented to be notified after any
- * watchdog configuration change. The arch hardlockup watchdog should
- * respond to the following variables:
+/**
+ * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
+ * @run:	If false stop the watchdogs on all enabled CPUs
+ *		If true start the watchdogs on all enabled CPUs
+ *
+ * The core call order is:
+ * watchdog_nmi_reconfigure(false);
+ * update_variables();
+ * watchdog_nmi_reconfigure(true);
+ *
+ * The second call which starts the watchdogs again guarantees that the
+ * following variables are stable across the call.
  * - watchdog_enabled
  * - watchdog_thresh
  * - watchdog_cpumask
- * - sysctl_hardlockup_all_cpu_backtrace
- * - hardlockup_panic
+ *
+ * After the call the variables can be changed again.
  */
-void __weak watchdog_nmi_reconfigure(void) { }
+void __weak watchdog_nmi_reconfigure(bool run) { }
 
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 
@@ -515,10 +523,12 @@ static void softlockup_unpark_threads(vo
 
 static void softlockup_reconfigure_threads(bool enabled)
 {
+	watchdog_nmi_reconfigure(false);
 	softlockup_park_all_threads();
 	set_sample_period();
 	if (enabled)
 		softlockup_unpark_threads();
+	watchdog_nmi_reconfigure(true);
 }
 
 /*
@@ -559,7 +569,11 @@ static inline void watchdog_unpark_threa
 static inline int watchdog_enable_all_cpus(void) { return 0; }
 static inline void watchdog_disable_all_cpus(void) { }
 static inline void softlockup_init_threads(void) { }
-static inline void softlockup_reconfigure_threads(bool enabled) { }
+static void softlockup_reconfigure_threads(bool enabled)
+{
+	watchdog_nmi_reconfigure(false);
+	watchdog_nmi_reconfigure(true);
+}
 #endif /* !CONFIG_SOFTLOCKUP_DETECTOR */
 
 static void __lockup_detector_cleanup(void)
@@ -599,7 +613,6 @@ static void proc_watchdog_update(void)
 	/* Remove impossible cpus to keep sysctl output clean. */
 	cpumask_and(&watchdog_cpumask, &watchdog_cpumask, cpu_possible_mask);
 	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
-	watchdog_nmi_reconfigure();
 }
 
 /*

[toc] | [prev] | [next] | [standalone]

#1732195 — [tip:core/urgent] watchdog/core, powerpc: Make watchdog_nmi_reconfigure() two stage

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/core, powerpc: Make watchdog_nmi_reconfigure() two stage
Message-ID	<upA2d-3WF-5@gated-at.bofh.it>
In reply to	#1731205

Commit-ID:  6592ad2fcc8f15b4f99b36c1db7d9f65510c203b
Gitweb:     http://git.kernel.org/tip/6592ad2fcc8f15b4f99b36c1db7d9f65510c203b
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:16 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:07 +0200

watchdog/core, powerpc: Make watchdog_nmi_reconfigure() two stage

Both the perf reconfiguration and the powerpc watchdog_nmi_reconfigure()
need to be done in two steps.

     1) Stop all NMIs
     2) Read the new parameters and start NMIs

Right now watchdog_nmi_reconfigure() is a combination of both. To allow a
clean reconfiguration add a 'run' argument and split the functionality in
powerpc.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/20170912194147.862865570@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/powerpc/kernel/watchdog.c | 17 +++++++++--------
 include/linux/nmi.h            |  2 ++
 kernel/watchdog.c              | 31 ++++++++++++++++++++++---------
 3 files changed, 33 insertions(+), 17 deletions(-)

diff --git a/arch/powerpc/kernel/watchdog.c b/arch/powerpc/kernel/watchdog.c
index 5ded171..291af79 100644
--- a/arch/powerpc/kernel/watchdog.c
+++ b/arch/powerpc/kernel/watchdog.c
@@ -355,17 +355,18 @@ static void watchdog_calc_timeouts(void)
 	wd_timer_period_ms = watchdog_thresh * 1000 * 2 / 5;
 }
 
-void watchdog_nmi_reconfigure(void)
+void watchdog_nmi_reconfigure(bool run)
 {
 	int cpu;
 
-	watchdog_calc_timeouts();
-
-	for_each_cpu(cpu, &wd_cpus_enabled)
-		stop_wd_on_cpu(cpu);
-
-	for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask)
-		start_wd_on_cpu(cpu);
+	if (!run) {
+		for_each_cpu(cpu, &wd_cpus_enabled)
+			stop_wd_on_cpu(cpu);
+	} else {
+		watchdog_calc_timeouts();
+		for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask)
+			start_wd_on_cpu(cpu);
+	}
 }
 
 /*
diff --git a/include/linux/nmi.h b/include/linux/nmi.h
index 4a8d103..eee255b 100644
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -103,6 +103,8 @@ static inline void arch_touch_nmi_watchdog(void) {}
 #endif
 #endif
 
+void watchdog_nmi_reconfigure(bool run);
+
 /**
  * touch_nmi_watchdog - restart NMI watchdog timeout.
  *
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index baae9fc..5693afd 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -112,17 +112,25 @@ void __weak watchdog_nmi_disable(unsigned int cpu)
 	hardlockup_detector_perf_disable();
 }
 
-/*
- * watchdog_nmi_reconfigure can be implemented to be notified after any
- * watchdog configuration change. The arch hardlockup watchdog should
- * respond to the following variables:
+/**
+ * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
+ * @run:	If false stop the watchdogs on all enabled CPUs
+ *		If true start the watchdogs on all enabled CPUs
+ *
+ * The core call order is:
+ * watchdog_nmi_reconfigure(false);
+ * update_variables();
+ * watchdog_nmi_reconfigure(true);
+ *
+ * The second call which starts the watchdogs again guarantees that the
+ * following variables are stable across the call.
  * - watchdog_enabled
  * - watchdog_thresh
  * - watchdog_cpumask
- * - sysctl_hardlockup_all_cpu_backtrace
- * - hardlockup_panic
+ *
+ * After the call the variables can be changed again.
  */
-void __weak watchdog_nmi_reconfigure(void) { }
+void __weak watchdog_nmi_reconfigure(bool run) { }
 
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 
@@ -515,10 +523,12 @@ static void softlockup_unpark_threads(void)
 
 static void softlockup_reconfigure_threads(bool enabled)
 {
+	watchdog_nmi_reconfigure(false);
 	softlockup_park_all_threads();
 	set_sample_period();
 	if (enabled)
 		softlockup_unpark_threads();
+	watchdog_nmi_reconfigure(true);
 }
 
 /*
@@ -559,7 +569,11 @@ static inline void watchdog_unpark_threads(void) { }
 static inline int watchdog_enable_all_cpus(void) { return 0; }
 static inline void watchdog_disable_all_cpus(void) { }
 static inline void softlockup_init_threads(void) { }
-static inline void softlockup_reconfigure_threads(bool enabled) { }
+static void softlockup_reconfigure_threads(bool enabled)
+{
+	watchdog_nmi_reconfigure(false);
+	watchdog_nmi_reconfigure(true);
+}
 #endif /* !CONFIG_SOFTLOCKUP_DETECTOR */
 
 static void __lockup_detector_cleanup(void)
@@ -599,7 +613,6 @@ static void proc_watchdog_update(void)
 	/* Remove impossible cpus to keep sysctl output clean. */
 	cpumask_and(&watchdog_cpumask, &watchdog_cpumask, cpu_possible_mask);
 	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
-	watchdog_nmi_reconfigure();
 }
 
 /*

[toc] | [prev] | [next] | [standalone]

#1731206 — [patch V2 25/29] lockup_detector: Implement init time detection of perf

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 25/29] lockup_detector: Implement init time detection of perf
Message-ID	<uoZvK-5zK-65@gated-at.bofh.it>
In reply to	#1731174

Use the init time detection of the perf NMI watchdog to determine whether
the perf NMI watchdog is functional. If not disable it permanentely. It
won't come back magically at runtime.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20170831073054.997264122@linutronix.de

---
 kernel/watchdog.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -44,6 +44,7 @@ int __read_mostly watchdog_user_enabled
 int __read_mostly nmi_watchdog_user_enabled = NMI_WATCHDOG_DEFAULT;
 int __read_mostly soft_watchdog_user_enabled = 1;
 int __read_mostly watchdog_thresh = 10;
+int __read_mostly nmi_watchdog_available;
 
 struct cpumask watchdog_allowed_mask __read_mostly;
 static bool softlockup_threads_initialized __read_mostly;
@@ -114,6 +115,12 @@ void __weak watchdog_nmi_disable(unsigne
 	hardlockup_detector_perf_disable();
 }
 
+/* Return 0, if a NMI watchdog is available. Error code otherwise */
+int __weak __init void watchdog_nmi_probe(void)
+{
+	return hardlockup_detector_perf_init();
+}
+
 /**
  * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
  * @run:	If false stop the watchdogs on all enabled CPUs
@@ -145,7 +152,7 @@ static void lockup_detector_update_enabl
 	watchdog_enabled = 0;
 	if (!watchdog_user_enabled)
 		return;
-	if (nmi_watchdog_user_enabled)
+	if (nmi_watchdog_available && nmi_watchdog_user_enabled)
 		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
 	if (soft_watchdog_user_enabled)
 		watchdog_enabled |= SOFT_WATCHDOG_ENABLED;
@@ -692,6 +699,8 @@ int proc_watchdog(struct ctl_table *tabl
 int proc_nmi_watchdog(struct ctl_table *table, int write,
 		      void __user *buffer, size_t *lenp, loff_t *ppos)
 {
+	if (!nmi_watchdog_available && write)
+		return -ENOTSUPP;
 	return proc_watchdog_common(NMI_WATCHDOG_ENABLED,
 				    table, write, buffer, lenp, ppos);
 }
@@ -764,5 +773,7 @@ void __init lockup_detector_init(void)
 	cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
 #endif
 
+	if (!watchdog_nmi_probe())
+		nmi_watchdog_available = true;
 	softlockup_init_threads();
 }

[toc] | [prev] | [next] | [standalone]

#1731763 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

From	Don Zickus <dzickus@redhat.com>
Date	2017-09-13 20:10 +0200
Subject	Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf
Message-ID	<upkgO-2kQ-11@gated-at.bofh.it>
In reply to	#1731206

On Tue, Sep 12, 2017 at 09:37:19PM +0200, Thomas Gleixner wrote:
> Use the init time detection of the perf NMI watchdog to determine whether
> the perf NMI watchdog is functional. If not disable it permanentely. It
> won't come back magically at runtime.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Don Zickus <dzickus@redhat.com>
> Cc: Chris Metcalf <cmetcalf@mellanox.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Ulrich Obergfell <uobergfe@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Link: http://lkml.kernel.org/r/20170831073054.997264122@linutronix.de
> 
> ---
>  kernel/watchdog.c |   13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -44,6 +44,7 @@ int __read_mostly watchdog_user_enabled
>  int __read_mostly nmi_watchdog_user_enabled = NMI_WATCHDOG_DEFAULT;
>  int __read_mostly soft_watchdog_user_enabled = 1;
>  int __read_mostly watchdog_thresh = 10;
> +int __read_mostly nmi_watchdog_available;
>  
>  struct cpumask watchdog_allowed_mask __read_mostly;
>  static bool softlockup_threads_initialized __read_mostly;
> @@ -114,6 +115,12 @@ void __weak watchdog_nmi_disable(unsigne
>  	hardlockup_detector_perf_disable();
>  }
>  
> +/* Return 0, if a NMI watchdog is available. Error code otherwise */
> +int __weak __init void watchdog_nmi_probe(void)

^^^ compile error, can't have 'int' and 'void'.  Removing 'void' fixes it.

Cheers,
Don

> +{
> +	return hardlockup_detector_perf_init();
> +}
> +
>  /**
>   * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
>   * @run:	If false stop the watchdogs on all enabled CPUs
> @@ -145,7 +152,7 @@ static void lockup_detector_update_enabl
>  	watchdog_enabled = 0;
>  	if (!watchdog_user_enabled)
>  		return;
> -	if (nmi_watchdog_user_enabled)
> +	if (nmi_watchdog_available && nmi_watchdog_user_enabled)
>  		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
>  	if (soft_watchdog_user_enabled)
>  		watchdog_enabled |= SOFT_WATCHDOG_ENABLED;
> @@ -692,6 +699,8 @@ int proc_watchdog(struct ctl_table *tabl
>  int proc_nmi_watchdog(struct ctl_table *table, int write,
>  		      void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> +	if (!nmi_watchdog_available && write)
> +		return -ENOTSUPP;
>  	return proc_watchdog_common(NMI_WATCHDOG_ENABLED,
>  				    table, write, buffer, lenp, ppos);
>  }
> @@ -764,5 +773,7 @@ void __init lockup_detector_init(void)
>  	cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
>  #endif
>  
> +	if (!watchdog_nmi_probe())
> +		nmi_watchdog_available = true;
>  	softlockup_init_threads();
>  }
> 
>

[toc] | [prev] | [next] | [standalone]

#1731765 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-13 20:10 +0200
Subject	Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf
Message-ID	<upkgO-2kQ-15@gated-at.bofh.it>
In reply to	#1731763

On Wed, 13 Sep 2017, Don Zickus wrote:
> > +/* Return 0, if a NMI watchdog is available. Error code otherwise */
> > +int __weak __init void watchdog_nmi_probe(void)
> 
> ^^^ compile error, can't have 'int' and 'void'.  Removing 'void' fixes it.

Yes, I know. I feel stupid because I'm 100% sure that I compiled it ....

[toc] | [prev] | [next] | [standalone]

#1732047 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

From	Ingo Molnar <mingo@kernel.org>
Date	2017-09-14 07:30 +0200
Subject	Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf
Message-ID	<upuSR-Qy-7@gated-at.bofh.it>
In reply to	#1731765

* Thomas Gleixner <tglx@linutronix.de> wrote:

> On Wed, 13 Sep 2017, Don Zickus wrote:
> > > +/* Return 0, if a NMI watchdog is available. Error code otherwise */
> > > +int __weak __init void watchdog_nmi_probe(void)
> > 
> > ^^^ compile error, can't have 'int' and 'void'.  Removing 'void' fixes it.
> 
> Yes, I know. I feel stupid because I'm 100% sure that I compiled it ....

So there a Kconfig dependency on that - defconfig won't show it, which might be 
why you didn't see it?

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1732207 — [tip:core/urgent] watchdog/hardlockup/perf: Implement init time detection of perf

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/hardlockup/perf: Implement init time detection of perf
Message-ID	<upA2f-3WF-37@gated-at.bofh.it>
In reply to	#1731206

Commit-ID:  a994a3147e4c0c9c50a46e6cace7586254975e20
Gitweb:     http://git.kernel.org/tip/a994a3147e4c0c9c50a46e6cace7586254975e20
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:19 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:08 +0200

watchdog/hardlockup/perf: Implement init time detection of perf

Use the init time detection of the perf NMI watchdog to determine whether
the perf NMI watchdog is functional. If not disable it permanentely. It
won't come back magically at runtime.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194148.099799541@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/watchdog.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 8488631..fd8a998 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -44,6 +44,7 @@ int __read_mostly watchdog_user_enabled = 1;
 int __read_mostly nmi_watchdog_user_enabled = NMI_WATCHDOG_DEFAULT;
 int __read_mostly soft_watchdog_user_enabled = 1;
 int __read_mostly watchdog_thresh = 10;
+int __read_mostly nmi_watchdog_available;
 
 struct cpumask watchdog_allowed_mask __read_mostly;
 static bool softlockup_threads_initialized __read_mostly;
@@ -114,6 +115,12 @@ void __weak watchdog_nmi_disable(unsigned int cpu)
 	hardlockup_detector_perf_disable();
 }
 
+/* Return 0, if a NMI watchdog is available. Error code otherwise */
+int __weak __init watchdog_nmi_probe(void)
+{
+	return hardlockup_detector_perf_init();
+}
+
 /**
  * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
  * @run:	If false stop the watchdogs on all enabled CPUs
@@ -145,7 +152,7 @@ static void lockup_detector_update_enable(void)
 	watchdog_enabled = 0;
 	if (!watchdog_user_enabled)
 		return;
-	if (nmi_watchdog_user_enabled)
+	if (nmi_watchdog_available && nmi_watchdog_user_enabled)
 		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
 	if (soft_watchdog_user_enabled)
 		watchdog_enabled |= SOFT_WATCHDOG_ENABLED;
@@ -692,6 +699,8 @@ int proc_watchdog(struct ctl_table *table, int write,
 int proc_nmi_watchdog(struct ctl_table *table, int write,
 		      void __user *buffer, size_t *lenp, loff_t *ppos)
 {
+	if (!nmi_watchdog_available && write)
+		return -ENOTSUPP;
 	return proc_watchdog_common(NMI_WATCHDOG_ENABLED,
 				    table, write, buffer, lenp, ppos);
 }
@@ -764,5 +773,7 @@ void __init lockup_detector_init(void)
 	cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
 #endif
 
+	if (!watchdog_nmi_probe())
+		nmi_watchdog_available = true;
 	softlockup_init_threads();
 }

[toc] | [prev] | [next] | [standalone]

#1731207 — [patch V2 23/29] lockup_detector: Get rid of the racy update loop

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-12 22:00 +0200
Subject	[patch V2 23/29] lockup_detector: Get rid of the racy update loop
Message-ID	<uoZvK-5zK-67@gated-at.bofh.it>
In reply to	#1731174

Letting user space poke directly at variables which are used at run time is
stupid and causes a lot of race conditions and other issues.

Seperate the user variables and on change invoke the reconfiguration, which
then stops the watchdogs, reevaluates the new user value and restarts the
watchdogs with the new parameters.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20170831073054.832604378@linutronix.de

---
 kernel/watchdog.c |   95 ++++++++++++++++++++++++++----------------------------
 1 file changed, 47 insertions(+), 48 deletions(-)

--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -32,15 +32,17 @@
 static DEFINE_MUTEX(watchdog_mutex);
 
 #if defined(CONFIG_HARDLOCKUP_DETECTOR) || defined(CONFIG_HAVE_NMI_WATCHDOG)
-unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED |
-						NMI_WATCHDOG_ENABLED;
+# define WATCHDOG_DEFAULT	(SOFT_WATCHDOG_ENABLED | NMI_WATCHDOG_ENABLED)
+# define NMI_WATCHDOG_DEFAULT	1
 #else
-unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED;
+# define WATCHDOG_DEFAULT	(SOFT_WATCHDOG_ENABLED)
+# define NMI_WATCHDOG_DEFAULT	0
 #endif
 
-int __read_mostly nmi_watchdog_user_enabled;
-int __read_mostly soft_watchdog_user_enabled;
-int __read_mostly watchdog_user_enabled;
+unsigned long __read_mostly watchdog_enabled;
+int __read_mostly watchdog_user_enabled = 1;
+int __read_mostly nmi_watchdog_user_enabled = NMI_WATCHDOG_DEFAULT;
+int __read_mostly soft_watchdog_user_enabled = 1;
 int __read_mostly watchdog_thresh = 10;
 
 struct cpumask watchdog_allowed_mask __read_mostly;
@@ -65,7 +67,7 @@ unsigned int __read_mostly hardlockup_pa
  */
 void __init hardlockup_detector_disable(void)
 {
-	watchdog_enabled &= ~NMI_WATCHDOG_ENABLED;
+	nmi_watchdog_user_enabled = 0;
 }
 
 static int __init hardlockup_panic_setup(char *str)
@@ -75,9 +77,9 @@ static int __init hardlockup_panic_setup
 	else if (!strncmp(str, "nopanic", 7))
 		hardlockup_panic = 0;
 	else if (!strncmp(str, "0", 1))
-		watchdog_enabled &= ~NMI_WATCHDOG_ENABLED;
+		nmi_watchdog_user_enabled = 0;
 	else if (!strncmp(str, "1", 1))
-		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
+		nmi_watchdog_user_enabled = 1;
 	return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
@@ -132,6 +134,23 @@ void __weak watchdog_nmi_disable(unsigne
  */
 void __weak watchdog_nmi_reconfigure(bool run) { }
 
+/**
+ * lockup_detector_update_enable - Update the sysctl enable bit
+ *
+ * Caller needs to make sure that the NMI/perf watchdogs are off, so this
+ * can't race with watchdog_nmi_disable().
+ */
+static void lockup_detector_update_enable(void)
+{
+	watchdog_enabled = 0;
+	if (!watchdog_user_enabled)
+		return;
+	if (nmi_watchdog_user_enabled)
+		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
+	if (soft_watchdog_user_enabled)
+		watchdog_enabled |= SOFT_WATCHDOG_ENABLED;
+}
+
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 
 /* Global variables, exported for sysctl */
@@ -160,14 +179,14 @@ static int __init softlockup_panic_setup
 
 static int __init nowatchdog_setup(char *str)
 {
-	watchdog_enabled = 0;
+	watchdog_user_enabled = 0;
 	return 1;
 }
 __setup("nowatchdog", nowatchdog_setup);
 
 static int __init nosoftlockup_setup(char *str)
 {
-	watchdog_enabled &= ~SOFT_WATCHDOG_ENABLED;
+	soft_watchdog_user_enabled = 0;
 	return 1;
 }
 __setup("nosoftlockup", nosoftlockup_setup);
@@ -521,12 +540,13 @@ static void softlockup_unpark_threads(vo
 	softlockup_update_smpboot_threads();
 }
 
-static void softlockup_reconfigure_threads(bool enabled)
+static void softlockup_reconfigure_threads(void)
 {
 	watchdog_nmi_reconfigure(false);
 	softlockup_park_all_threads();
 	set_sample_period();
-	if (enabled)
+	lockup_detector_update_enable();
+	if (watchdog_enabled && watchdog_thresh)
 		softlockup_unpark_threads();
 	watchdog_nmi_reconfigure(true);
 }
@@ -546,6 +566,8 @@ static __init void softlockup_init_threa
 	 * If sysctl is off and watchdog got disabled on the command line,
 	 * nothing to do here.
 	 */
+	lockup_detector_update_enable();
+
 	if (!IS_ENABLED(CONFIG_SYSCTL) &&
 	    !(watchdog_enabled && watchdog_thresh))
 		return;
@@ -559,7 +581,7 @@ static __init void softlockup_init_threa
 
 	mutex_lock(&watchdog_mutex);
 	softlockup_threads_initialized = true;
-	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
+	softlockup_reconfigure_threads();
 	mutex_unlock(&watchdog_mutex);
 }
 
@@ -569,9 +591,10 @@ static inline void watchdog_unpark_threa
 static inline int watchdog_enable_all_cpus(void) { return 0; }
 static inline void watchdog_disable_all_cpus(void) { }
 static inline void softlockup_init_threads(void) { }
-static void softlockup_reconfigure_threads(bool enabled)
+static void softlockup_reconfigure_threads(void)
 {
 	watchdog_nmi_reconfigure(false);
+	lockup_detector_update_enable();
 	watchdog_nmi_reconfigure(true);
 }
 #endif /* !CONFIG_SOFTLOCKUP_DETECTOR */
@@ -612,7 +635,7 @@ static void proc_watchdog_update(void)
 {
 	/* Remove impossible cpus to keep sysctl output clean. */
 	cpumask_and(&watchdog_cpumask, &watchdog_cpumask, cpu_possible_mask);
-	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
+	softlockup_reconfigure_threads();
 }
 
 /*
@@ -630,48 +653,24 @@ static void proc_watchdog_update(void)
 static int proc_watchdog_common(int which, struct ctl_table *table, int write,
 				void __user *buffer, size_t *lenp, loff_t *ppos)
 {
-	int err, old, new;
-	int *watchdog_param = (int *)table->data;
+	int err, old, *param = table->data;
 
 	cpu_hotplug_disable();
 	mutex_lock(&watchdog_mutex);
 
-	/*
-	 * If the parameter is being read return the state of the corresponding
-	 * bit(s) in 'watchdog_enabled', else update 'watchdog_enabled' and the
-	 * run state of the lockup detectors.
-	 */
 	if (!write) {
-		*watchdog_param = (watchdog_enabled & which) != 0;
+		/*
+		 * On read synchronize the userspace interface. This is a
+		 * racy snapshot.
+		 */
+		*param = (watchdog_enabled & which) != 0;
 		err = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 	} else {
+		old = READ_ONCE(*param);
 		err = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
-		if (err)
-			goto out;
-
-		/*
-		 * There is a race window between fetching the current value
-		 * from 'watchdog_enabled' and storing the new value. During
-		 * this race window, watchdog_nmi_enable() can sneak in and
-		 * clear the NMI_WATCHDOG_ENABLED bit in 'watchdog_enabled'.
-		 * The 'cmpxchg' detects this race and the loop retries.
-		 */
-		do {
-			old = watchdog_enabled;
-			/*
-			 * If the parameter value is not zero set the
-			 * corresponding bit(s), else clear it(them).
-			 */
-			if (*watchdog_param)
-				new = old | which;
-			else
-				new = old & ~which;
-		} while (cmpxchg(&watchdog_enabled, old, new) != old);
-
-		if (old != new)
+		if (!err && old != READ_ONCE(*param))
 			proc_watchdog_update();
 	}
-out:
 	mutex_unlock(&watchdog_mutex);
 	cpu_hotplug_enable();
 	return err;

[toc] | [prev] | [next] | [standalone]

#1732206 — [tip:core/urgent] watchdog/core: Get rid of the racy update loop

From	tip-bot for Thomas Gleixner <tipbot@zytor.com>
Date	2017-09-14 13:00 +0200
Subject	[tip:core/urgent] watchdog/core: Get rid of the racy update loop
Message-ID	<upA2f-3WF-35@gated-at.bofh.it>
In reply to	#1731207

Commit-ID:  091549858ed881e5f3054374af4f5b1cac681d50
Gitweb:     http://git.kernel.org/tip/091549858ed881e5f3054374af4f5b1cac681d50
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Tue, 12 Sep 2017 21:37:17 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 14 Sep 2017 11:41:07 +0200

watchdog/core: Get rid of the racy update loop

Letting user space poke directly at variables which are used at run time is
stupid and causes a lot of race conditions and other issues.

Seperate the user variables and on change invoke the reconfiguration, which
then stops the watchdogs, reevaluates the new user value and restarts the
watchdogs with the new parameters.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Don Zickus <dzickus@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Ulrich Obergfell <uobergfe@redhat.com>
Link: http://lkml.kernel.org/r/20170912194147.939985640@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/watchdog.c | 95 +++++++++++++++++++++++++++----------------------------
 1 file changed, 47 insertions(+), 48 deletions(-)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 5693afd..8488631 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -32,15 +32,17 @@
 static DEFINE_MUTEX(watchdog_mutex);
 
 #if defined(CONFIG_HARDLOCKUP_DETECTOR) || defined(CONFIG_HAVE_NMI_WATCHDOG)
-unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED |
-						NMI_WATCHDOG_ENABLED;
+# define WATCHDOG_DEFAULT	(SOFT_WATCHDOG_ENABLED | NMI_WATCHDOG_ENABLED)
+# define NMI_WATCHDOG_DEFAULT	1
 #else
-unsigned long __read_mostly watchdog_enabled = SOFT_WATCHDOG_ENABLED;
+# define WATCHDOG_DEFAULT	(SOFT_WATCHDOG_ENABLED)
+# define NMI_WATCHDOG_DEFAULT	0
 #endif
 
-int __read_mostly nmi_watchdog_user_enabled;
-int __read_mostly soft_watchdog_user_enabled;
-int __read_mostly watchdog_user_enabled;
+unsigned long __read_mostly watchdog_enabled;
+int __read_mostly watchdog_user_enabled = 1;
+int __read_mostly nmi_watchdog_user_enabled = NMI_WATCHDOG_DEFAULT;
+int __read_mostly soft_watchdog_user_enabled = 1;
 int __read_mostly watchdog_thresh = 10;
 
 struct cpumask watchdog_allowed_mask __read_mostly;
@@ -65,7 +67,7 @@ unsigned int __read_mostly hardlockup_panic =
  */
 void __init hardlockup_detector_disable(void)
 {
-	watchdog_enabled &= ~NMI_WATCHDOG_ENABLED;
+	nmi_watchdog_user_enabled = 0;
 }
 
 static int __init hardlockup_panic_setup(char *str)
@@ -75,9 +77,9 @@ static int __init hardlockup_panic_setup(char *str)
 	else if (!strncmp(str, "nopanic", 7))
 		hardlockup_panic = 0;
 	else if (!strncmp(str, "0", 1))
-		watchdog_enabled &= ~NMI_WATCHDOG_ENABLED;
+		nmi_watchdog_user_enabled = 0;
 	else if (!strncmp(str, "1", 1))
-		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
+		nmi_watchdog_user_enabled = 1;
 	return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
@@ -132,6 +134,23 @@ void __weak watchdog_nmi_disable(unsigned int cpu)
  */
 void __weak watchdog_nmi_reconfigure(bool run) { }
 
+/**
+ * lockup_detector_update_enable - Update the sysctl enable bit
+ *
+ * Caller needs to make sure that the NMI/perf watchdogs are off, so this
+ * can't race with watchdog_nmi_disable().
+ */
+static void lockup_detector_update_enable(void)
+{
+	watchdog_enabled = 0;
+	if (!watchdog_user_enabled)
+		return;
+	if (nmi_watchdog_user_enabled)
+		watchdog_enabled |= NMI_WATCHDOG_ENABLED;
+	if (soft_watchdog_user_enabled)
+		watchdog_enabled |= SOFT_WATCHDOG_ENABLED;
+}
+
 #ifdef CONFIG_SOFTLOCKUP_DETECTOR
 
 /* Global variables, exported for sysctl */
@@ -160,14 +179,14 @@ __setup("softlockup_panic=", softlockup_panic_setup);
 
 static int __init nowatchdog_setup(char *str)
 {
-	watchdog_enabled = 0;
+	watchdog_user_enabled = 0;
 	return 1;
 }
 __setup("nowatchdog", nowatchdog_setup);
 
 static int __init nosoftlockup_setup(char *str)
 {
-	watchdog_enabled &= ~SOFT_WATCHDOG_ENABLED;
+	soft_watchdog_user_enabled = 0;
 	return 1;
 }
 __setup("nosoftlockup", nosoftlockup_setup);
@@ -521,12 +540,13 @@ static void softlockup_unpark_threads(void)
 	softlockup_update_smpboot_threads();
 }
 
-static void softlockup_reconfigure_threads(bool enabled)
+static void softlockup_reconfigure_threads(void)
 {
 	watchdog_nmi_reconfigure(false);
 	softlockup_park_all_threads();
 	set_sample_period();
-	if (enabled)
+	lockup_detector_update_enable();
+	if (watchdog_enabled && watchdog_thresh)
 		softlockup_unpark_threads();
 	watchdog_nmi_reconfigure(true);
 }
@@ -546,6 +566,8 @@ static __init void softlockup_init_threads(void)
 	 * If sysctl is off and watchdog got disabled on the command line,
 	 * nothing to do here.
 	 */
+	lockup_detector_update_enable();
+
 	if (!IS_ENABLED(CONFIG_SYSCTL) &&
 	    !(watchdog_enabled && watchdog_thresh))
 		return;
@@ -559,7 +581,7 @@ static __init void softlockup_init_threads(void)
 
 	mutex_lock(&watchdog_mutex);
 	softlockup_threads_initialized = true;
-	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
+	softlockup_reconfigure_threads();
 	mutex_unlock(&watchdog_mutex);
 }
 
@@ -569,9 +591,10 @@ static inline void watchdog_unpark_threads(void) { }
 static inline int watchdog_enable_all_cpus(void) { return 0; }
 static inline void watchdog_disable_all_cpus(void) { }
 static inline void softlockup_init_threads(void) { }
-static void softlockup_reconfigure_threads(bool enabled)
+static void softlockup_reconfigure_threads(void)
 {
 	watchdog_nmi_reconfigure(false);
+	lockup_detector_update_enable();
 	watchdog_nmi_reconfigure(true);
 }
 #endif /* !CONFIG_SOFTLOCKUP_DETECTOR */
@@ -612,7 +635,7 @@ static void proc_watchdog_update(void)
 {
 	/* Remove impossible cpus to keep sysctl output clean. */
 	cpumask_and(&watchdog_cpumask, &watchdog_cpumask, cpu_possible_mask);
-	softlockup_reconfigure_threads(watchdog_enabled && watchdog_thresh);
+	softlockup_reconfigure_threads();
 }
 
 /*
@@ -630,48 +653,24 @@ static void proc_watchdog_update(void)
 static int proc_watchdog_common(int which, struct ctl_table *table, int write,
 				void __user *buffer, size_t *lenp, loff_t *ppos)
 {
-	int err, old, new;
-	int *watchdog_param = (int *)table->data;
+	int err, old, *param = table->data;
 
 	cpu_hotplug_disable();
 	mutex_lock(&watchdog_mutex);
 
-	/*
-	 * If the parameter is being read return the state of the corresponding
-	 * bit(s) in 'watchdog_enabled', else update 'watchdog_enabled' and the
-	 * run state of the lockup detectors.
-	 */
 	if (!write) {
-		*watchdog_param = (watchdog_enabled & which) != 0;
+		/*
+		 * On read synchronize the userspace interface. This is a
+		 * racy snapshot.
+		 */
+		*param = (watchdog_enabled & which) != 0;
 		err = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 	} else {
+		old = READ_ONCE(*param);
 		err = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
-		if (err)
-			goto out;
-
-		/*
-		 * There is a race window between fetching the current value
-		 * from 'watchdog_enabled' and storing the new value. During
-		 * this race window, watchdog_nmi_enable() can sneak in and
-		 * clear the NMI_WATCHDOG_ENABLED bit in 'watchdog_enabled'.
-		 * The 'cmpxchg' detects this race and the loop retries.
-		 */
-		do {
-			old = watchdog_enabled;
-			/*
-			 * If the parameter value is not zero set the
-			 * corresponding bit(s), else clear it(them).
-			 */
-			if (*watchdog_param)
-				new = old | which;
-			else
-				new = old & ~which;
-		} while (cmpxchg(&watchdog_enabled, old, new) != old);
-
-		if (old != new)
+		if (!err && old != READ_ONCE(*param))
 			proc_watchdog_update();
 	}
-out:
 	mutex_unlock(&watchdog_mutex);
 	cpu_hotplug_enable();
 	return err;

[toc] | [prev] | [next] | [standalone]

#1731761 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

From	Don Zickus <dzickus@redhat.com>
Date	2017-09-13 20:10 +0200
Subject	Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape
Message-ID	<upkgO-2kQ-9@gated-at.bofh.it>
In reply to	#1731174

On Tue, Sep 12, 2017 at 09:36:54PM +0200, Thomas Gleixner wrote:
> The lockup detector is broken is several ways:
> 
>     - It's deadlock prone vs. CPU hotplug in various ways. Some of these
>       are due to recursive cpus_read_lock() others are due to
>       cpus_read_lock() from CPU hotplug callbacks which immediately lock
>       the machine because cpus are write locked.
> 
>     - The handling of the cpu hotplug threads happens sideways to the
>       smpboot thread infrastructure, which is racy and pointless
> 
>     - The handling of the user space sysctl interface is a complete
>       trainwreck as it fiddles directly with variables which can be
>       modified or evaluated by the running watchdogs.
> 
>     - The perf event initialization is a trainwreck as it tries to create
>       perf events over and over even if perf is not functional (no
>       hardware, ....). To avoid excessive dmesg spam it contains magic
>       printk ratelimiting along with either wrong or useless messages.
> 
>     - The code structure is horrible as ifdef sections are scattered all
>       over the place which makes it unreadable
> 
>     - There is more wreckage, but see the changelogs for the ugly details.
>

Aside from the simple compile issue in patch 25.  I have no issues with this
patchset.  Thanks Thomas!

Reviewed-by: Don Zickus <dzickus@redhat.com>

 
> The following series sanitizes the facility and addresses the problems.
> 
> Changes since V1:
> 
>     - Wrapped the perf specific calls into the weak watchdog_nmi_*
>       functions
> 
>     - Fixed the compile error pointed out by Don
> 
>     - Fixed the reconfiguration parameter inconsistency which broke
>       powerpc
> 
>     - Picked up the updated version of patch 11/29
> 
> Delta patch below.
> 
> Thanks,
> 
>         tglx
> ---
> Diffstat for the series:
> 
>  arch/parisc/kernel/process.c   |    2 
>  arch/powerpc/kernel/watchdog.c |   22 -
>  arch/x86/events/intel/core.c   |   11 
>  include/linux/nmi.h            |  121 +++----
>  include/linux/smpboot.h        |    4 
>  kernel/cpu.c                   |    6 
>  kernel/smpboot.c               |   22 -
>  kernel/sysctl.c                |   22 -
>  kernel/watchdog.c              |  633 ++++++++++++++---------------------------
>  kernel/watchdog_hld.c          |  193 ++++++------
>  10 files changed, 434 insertions(+), 602 deletions(-)
> 
> Delta patch vs. V1
> 8<------------------------
> --- a/arch/powerpc/kernel/watchdog.c
> +++ b/arch/powerpc/kernel/watchdog.c
> @@ -355,12 +355,12 @@ static void watchdog_calc_timeouts(void)
>  	wd_timer_period_ms = watchdog_thresh * 1000 * 2 / 5;
>  }
>  
> -void watchdog_nmi_reconfigure(bool stop)
> +void watchdog_nmi_reconfigure(bool run)
>  {
>  	int cpu;
>  
>  	cpus_read_lock();
> -	if (stop) {
> +	if (!run) {
>  		for_each_cpu(cpu, &wd_cpus_enabled)
>  			stop_wd_on_cpu(cpu);
>  	} else {
> --- a/include/linux/nmi.h
> +++ b/include/linux/nmi.h
> @@ -102,7 +102,7 @@ static inline void hardlockup_detector_p
>  static inline void hardlockup_detector_perf_enable(void) { }
>  static inline void hardlockup_detector_perf_cleanup(void) { }
>  # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
> -static int hardlockup_detector_perf_init(void) { return -ENODEV; }
> +static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
>  static inline void arch_touch_nmi_watchdog(void) {}
>  # else
>  static int hardlockup_detector_perf_init(void) { return 0; }
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -105,18 +105,32 @@ static int __init hardlockup_all_cpu_bac
>   * softlockup watchdog threads start and stop. The arch must select the
>   * SOFTLOCKUP_DETECTOR Kconfig.
>   */
> -int __weak watchdog_nmi_enable(unsigned int cpu) { return 0; }
> -void __weak watchdog_nmi_disable(unsigned int cpu) { }
> +int __weak watchdog_nmi_enable(unsigned int cpu)
> +{
> +	hardlockup_detector_perf_enable();
> +	return 0;
> +}
> +
> +void __weak watchdog_nmi_disable(unsigned int cpu)
> +{
> +	hardlockup_detector_perf_disable();
> +}
> +
> +/* Return 0, if a NMI watchdog is available. Error code otherwise */
> +int __weak __init void watchdog_nmi_probe(void)
> +{
> +	return hardlockup_detector_perf_init();
> +}
>  
>  /**
>   * watchdog_nmi_reconfigure - Optional function to reconfigure NMI watchdogs
> - * @stop:	If true stop the watchdogs on all enabled CPUs
> - *		If false start the watchdogs on all enabled CPUs
> + * @run:	If false stop the watchdogs on all enabled CPUs
> + *		If true start the watchdogs on all enabled CPUs
>   *
>   * The core call order is:
> - * watchdog_nmi_reconfigure(true);
> - * update_variables();
>   * watchdog_nmi_reconfigure(false);
> + * update_variables();
> + * watchdog_nmi_reconfigure(true);
>   *
>   * The second call which starts the watchdogs again guarantees that the
>   * following variables are stable across the call.
> @@ -126,13 +140,13 @@ void __weak watchdog_nmi_disable(unsigne
>   *
>   * After the call the variables can be changed again.
>   */
> -void __weak watchdog_nmi_reconfigure(bool stop) { }
> +void __weak watchdog_nmi_reconfigure(bool run) { }
>  
>  /**
>   * lockup_detector_update_enable - Update the sysctl enable bit
>   *
>   * Caller needs to make sure that the NMI/perf watchdogs are off, so this
> - * can't race with hardlockup_detector_disable().
> + * can't race with watchdog_nmi_disable().
>   */
>  static void lockup_detector_update_enable(void)
>  {
> @@ -453,8 +467,7 @@ static void watchdog_enable(unsigned int
>  	__touch_watchdog();
>  	/* Enable the perf event */
>  	if (watchdog_enabled & NMI_WATCHDOG_ENABLED)
> -		hardlockup_detector_perf_enable();
> -	watchdog_nmi_enable(cpu);
> +		watchdog_nmi_enable(cpu);
>  
>  	watchdog_set_prio(SCHED_FIFO, MAX_RT_PRIO - 1);
>  }
> @@ -469,7 +482,6 @@ static void watchdog_disable(unsigned in
>  	 * between disabling the timer and disabling the perf event causes
>  	 * the perf NMI to detect a false positive.
>  	 */
> -	hardlockup_detector_perf_disable();
>  	watchdog_nmi_disable(cpu);
>  	hrtimer_cancel(hrtimer);
>  }
> @@ -745,12 +757,6 @@ int proc_watchdog_cpumask(struct ctl_tab
>  }
>  #endif /* CONFIG_SYSCTL */
>  
> -static __init void detect_nmi_watchdog(void)
> -{
> -	if (!hardlockup_detector_perf_init())
> -		nmi_watchdog_available = true;
> -}
> -
>  void __init lockup_detector_init(void)
>  {
>  #ifdef CONFIG_NO_HZ_FULL
> @@ -763,6 +769,7 @@ void __init lockup_detector_init(void)
>  	cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
>  #endif
>  
> -	detect_nmi_watchdog();
> +	if (!watchdog_nmi_probe())
> +		nmi_watchdog_available = true;
>  	softlockup_init_threads();
>  }
> 
> 
>

[toc] | [prev] | [next] | [standalone]

#1732046 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

From	Ingo Molnar <mingo@kernel.org>
Date	2017-09-14 07:30 +0200
Subject	Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape
Message-ID	<upuSR-Qy-9@gated-at.bofh.it>
In reply to	#1731761

* Don Zickus <dzickus@redhat.com> wrote:

> On Tue, Sep 12, 2017 at 09:36:54PM +0200, Thomas Gleixner wrote:
> > The lockup detector is broken is several ways:
> > 
> >     - It's deadlock prone vs. CPU hotplug in various ways. Some of these
> >       are due to recursive cpus_read_lock() others are due to
> >       cpus_read_lock() from CPU hotplug callbacks which immediately lock
> >       the machine because cpus are write locked.
> > 
> >     - The handling of the cpu hotplug threads happens sideways to the
> >       smpboot thread infrastructure, which is racy and pointless
> > 
> >     - The handling of the user space sysctl interface is a complete
> >       trainwreck as it fiddles directly with variables which can be
> >       modified or evaluated by the running watchdogs.
> > 
> >     - The perf event initialization is a trainwreck as it tries to create
> >       perf events over and over even if perf is not functional (no
> >       hardware, ....). To avoid excessive dmesg spam it contains magic
> >       printk ratelimiting along with either wrong or useless messages.
> > 
> >     - The code structure is horrible as ifdef sections are scattered all
> >       over the place which makes it unreadable
> > 
> >     - There is more wreckage, but see the changelogs for the ugly details.
> >
> 
> Aside from the simple compile issue in patch 25.  I have no issues with this
> patchset.  Thanks Thomas!
> 
> Reviewed-by: Don Zickus <dzickus@redhat.com>

Thanks guys! I've applied the series and will push it out after some testing.

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1732108 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

From	Thomas Gleixner <tglx@linutronix.de>
Date	2017-09-14 10:20 +0200
Subject	Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape
Message-ID	<upxxn-2xM-11@gated-at.bofh.it>
In reply to	#1731761

On Wed, 13 Sep 2017, Don Zickus wrote:
> 
> Aside from the simple compile issue in patch 25.  I have no issues with this
> patchset.  Thanks Thomas!
> 
> Reviewed-by: Don Zickus <dzickus@redhat.com>

Thanks for your time and feedback!

       tglx

[toc] | [prev] | [standalone]

Page 3 of 3 — ← Prev page 1 2 [3]

csiph-web

[patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

Contents

#1732213 — [tip:core/urgent] watchdog/sysctl: Get rid of the #ifdeffery

#1731201 — [patch V2 27/29] lockup_detector: Use new perf CPU enable mechanism

#1732199 — [tip:core/urgent] watchdog/hardlockup/perf: Use new perf CPU enable mechanism

#1731202 — [patch V2 24/29] lockup_detector/perf: Implement init time perf validation

#1732193 — [tip:core/urgent] watchdog/hardlockup/perf: Implement init time perf validation

#1731203 — [patch V2 19/29] lockup_detector: Cleanup header mess

#1732209 — [tip:core/urgent] watchdog/core: Clean up header mess

#1731205 — [patch V2 22/29] lockup_detector: Make watchdog_nmi_reconfigure() two stage

#1732195 — [tip:core/urgent] watchdog/core, powerpc: Make watchdog_nmi_reconfigure() two stage

#1731206 — [patch V2 25/29] lockup_detector: Implement init time detection of perf

#1731763 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

#1731765 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

#1732047 — Re: [patch V2 25/29] lockup_detector: Implement init time detection of perf

#1732207 — [tip:core/urgent] watchdog/hardlockup/perf: Implement init time detection of perf

#1731207 — [patch V2 23/29] lockup_detector: Get rid of the racy update loop

#1732206 — [tip:core/urgent] watchdog/core: Get rid of the racy update loop

#1731761 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

#1732046 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

#1732108 — Re: [patch V2 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape