Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1318151 > unrolled thread

[PATCH v2] Add hard/soft lockup debugger entry points

Back to article view | Back to linux.kernel

Contents

  [PATCH v2] Add hard/soft lockup debugger entry points Jeff Merkey <linux.mdb@gmail.com> - 2016-01-26 18:10 +0100
    Re: [PATCH v2] Add hard/soft lockup debugger entry points Chris Metcalf <cmetcalf@ezchip.com> - 2016-01-26 18:30 +0100
      Re: [PATCH v2] Add hard/soft lockup debugger entry points Jeff Merkey <linux.mdb@gmail.com> - 2016-01-26 18:30 +0100
    Re: [PATCH v2] Add hard/soft lockup debugger entry points kbuild test robot <lkp@intel.com> - 2016-01-26 19:50 +0100
      Re: [PATCH v2] Add hard/soft lockup debugger entry points Jeff Merkey <linux.mdb@gmail.com> - 2016-01-26 20:40 +0100

#1318151 — [PATCH v2] Add hard/soft lockup debugger entry points

This patch adds an export which can be set by system debuggers to direct
the hard lockup and soft lockup detector to trigger a breakpoint exception
and enter a debugger if one is active.  It is assumed that if someone
sets this variable, then an breakpoint handler of some sort will be actively
loaded or registered via the notify die handler chain.

This addition is extremely useful for debugging hard and soft lockups
real time and quickly from a console debugger.

Signed-off-by: Jeff Merkey <linux.mdb@gmail.com>
---
 kernel/watchdog.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index b3ace6e..c28e58c 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -23,6 +23,7 @@
 #include <linux/workqueue.h>
 
 #include <asm/irq_regs.h>
+#include <linux/kgdb.h>
 #include <linux/kvm_para.h>
 #include <linux/perf_event.h>
 #include <linux/kthread.h>
@@ -108,6 +109,9 @@ static DEFINE_PER_CPU(struct perf_event *, watchdog_ev);
 #endif
 static unsigned long soft_lockup_nmi_warn;
 
+int debug_watchdog_lockups;
+EXPORT_SYMBOL_GPL(debug_watchdog_lockups);
+
 /* boot commands */
 /*
  * Should we panic when a soft-lockup or hard-lockup occurs:
@@ -358,6 +362,9 @@ static void watchdog_overflow_callback(struct perf_event *event,
 		else
 			dump_stack();
 
+		if (debug_watchdog_lockups)
+			arch_kgdb_breakpoint();
+
 		/*
 		 * Perform all-CPU dump only once to avoid multiple hardlockups
 		 * generating interleaving traces
@@ -478,6 +485,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
 		else
 			dump_stack();
 
+		if (debug_watchdog_lockups)
+			arch_kgdb_breakpoint();
+
 		if (softlockup_all_cpu_backtrace) {
 			/* Avoid generating two back traces for current
 			 * given that one is already made above
-- 
1.8.3.1

[toc] | [next] | [standalone]

#1318189

On 01/26/2016 12:04 PM, Jeff Merkey wrote:
> This patch adds an export which can be set by system debuggers to direct
> the hard lockup and soft lockup detector to trigger a breakpoint exception
> and enter a debugger if one is active.  It is assumed that if someone
> sets this variable, then an breakpoint handler of some sort will be actively
> loaded or registered via the notify die handler chain.
>
> This addition is extremely useful for debugging hard and soft lockups
> real time and quickly from a console debugger.
>
> Signed-off-by: Jeff Merkey<linux.mdb@gmail.com>
> ---
>   kernel/watchdog.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)

You probably should reach out to someone who uses this stuff more
regularly - I actually wonder if the kgdb_breakpoint() API is the
right thing, though, not the internal arch_kgdb_breakpoint().

Of course any of these strategies also assume you are building
the kernel with CONFIG_KGDB set, and I'm pretty sure will cause
your build to fail if it isn't.  You likely need to guard this
stuff locally within watchdog.c for !CONFIG_KGDB.

-- 
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

[toc] | [prev] | [next] | [standalone]

#1318198

On 1/26/16, Chris Metcalf <cmetcalf@ezchip.com> wrote:
> On 01/26/2016 12:04 PM, Jeff Merkey wrote:
>> This patch adds an export which can be set by system debuggers to direct
>> the hard lockup and soft lockup detector to trigger a breakpoint
>> exception
>> and enter a debugger if one is active.  It is assumed that if someone
>> sets this variable, then an breakpoint handler of some sort will be
>> actively
>> loaded or registered via the notify die handler chain.
>>
>> This addition is extremely useful for debugging hard and soft lockups
>> real time and quickly from a console debugger.
>>
>> Signed-off-by: Jeff Merkey<linux.mdb@gmail.com>
>> ---
>>   kernel/watchdog.c | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>
> You probably should reach out to someone who uses this stuff more
> regularly - I actually wonder if the kgdb_breakpoint() API is the
> right thing, though, not the internal arch_kgdb_breakpoint().
>
> Of course any of these strategies also assume you are building
> the kernel with CONFIG_KGDB set, and I'm pretty sure will cause
> your build to fail if it isn't.  You likely need to guard this
> stuff locally within watchdog.c for !CONFIG_KGDB.
>
> --
> Chris Metcalf, EZChip Semiconductor
> http://www.ezchip.com
>
>

I'll go review that code carefully.  What would ideal is to have the
breakpoint abstraction independent of kgdb entirely.  That would not
be a simple patch but more involved.  We really just need the
breakpoint instruction for each arch, not all of kgdb and it should
not depend on KGB being enabled.  I looked over those includes and
this may work, but you are correct, it's not squeaky clean to do it
that way.  I'll get to work on v3.

:-)

Jeff

[toc] | [prev] | [next] | [standalone]

#1318249

[Multipart message — attachments visible in raw view] — view raw

Hi Jeff,

[auto build test ERROR on v4.5-rc1]
[also build test ERROR on next-20160125]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160127-010930
config: sparc64-defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sparc64 

All errors (new ones prefixed by >>):

   kernel/built-in.o: In function `watchdog_timer_fn':
>> watchdog.c:(.text+0x736ec): undefined reference to `arch_kgdb_breakpoint'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[toc] | [prev] | [next] | [standalone]

#1318312

On 1/26/16, kbuild test robot <lkp@intel.com> wrote:
> Hi Jeff,
>
> [auto build test ERROR on v4.5-rc1]
> [also build test ERROR on next-20160125]
> [if your patch is applied to the wrong git tree, please drop us a note to
> help improving the system]
>
> url:
> https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160127-010930
> config: sparc64-defconfig (attached as .config)
> reproduce:
>         wget
> https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross
> -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         make.cross ARCH=sparc64
>
> All errors (new ones prefixed by >>):
>
>    kernel/built-in.o: In function `watchdog_timer_fn':
>>> watchdog.c:(.text+0x736ec): undefined reference to
>>> `arch_kgdb_breakpoint'
>
> ---
> 0-DAY kernel test infrastructure                Open Source Technology
> Center
> https://lists.01.org/pipermail/kbuild-all                   Intel
> Corporation
>

I love this test robot.  What an incredibly useful tool.  Well, looks
like a I quite a bit of work to do.

Jeff

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web