Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1320552
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code |
| Date | 2016-01-28 12:00 +0100 |
| Message-ID | <qVSsW-2cx-19@gated-at.bofh.it> (permalink) |
| References | (2 earlier) <qVKF3-4Q6-1@gated-at.bofh.it> <qVMQz-6q1-21@gated-at.bofh.it> <qVNWh-7D7-9@gated-at.bofh.it> <qVQ7P-FX-67@gated-at.bofh.it> <qVSjg-28z-25@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
two small corrections.
On (01/28/16 19:41), Sergey Senozhatsky wrote:
[..]
> > Unfortunately, it's not reproduced anymore.
> >
> > If it's clearly a spinlock caller's bug as you said, modifying the
> > spinlock debug code does not help it at all. But I found there's a
> > possiblity in the debug code *itself* to cause a lockup. So I tried
> > to fix it. What do you think about it?
>
> ah... silly me... you mean the first CPU that triggers the spin_dump() will
^^^ this, of course, is true for
console_sem->lock and logbuf_lock
only.
> deadlock itself, so the rest of CPUs will see endless recursive
> spin_lock()->spin_dump()->spin_lock()->spin_dump() calls?
>
> like the one below?
>
>
> CPUZ is doing vprintk_emit()->spin_lock(), CPUA is the spin_lock's owner
>
> CPUZ -> vprintk_emit()
> __spin_lock_debug()
> for (i = 0; i < `loops_per_jiffy * HZ'; i++) { << wait for the lock
> if (arch_spin_trylock())
> return;
> __delay(1);
> }
> spin_dump() << lock is still owned by CPUA
> { -> vprintk_emit()
> __spin_lock_debug()
> for (...) {
> if (arch_spin_trylock())
> return;
> __delay(1);
> }
- << CPUA unlocked the lock
> spin_dump()
> { -> vprintk_emit()
> __spin_lock_debug()
the "<< CPUA unlocked the lock" line better be here. to make it correct.
+ << CPUA unlocked the lock
> for (...) {
> if (arch_spin_trylock()) << success!!
> /* CPUZ now owns the lock */
> return;
> }
> }
>
> << we return here with the spin_lock being owned by this CPUZ
>
> trigger_all_cpu_backtrace()
>
> << and... now it does the arch_spin_lock()
> /*
> * The trylock above was causing a livelock. Give the lower level arch
> * specific lock code a chance to acquire the lock. We have already
> * printed a warning/backtrace at this point. The non-debug arch
> * specific code might actually succeed in acquiring the lock. If it is
> * not successful, the end-result is the same - there is no forward
> * progress.
> */
> arch_spin_lock(&lock->raw_lock);
>
> << which obviously dealocks this CPU...
> }
>
> trigger_all_cpu_backtrace()
>
> arch_spin_lock()
>
>
>
>
> so
> "the CPUZ is now keeping the lock forever, and not going to release it"
> and
> "CPUA-CPUX will do vprintk_emit()->spin_lock()->spin_dump()->vprintk_emit()->..."
>
>
>
> My apologies for not getting it right the first time. Sorry!
>
> Can you please update your bug description in the commit message?
> It's the deadlock that is causing the recursion on other CPUs in the
> first place.
-ss
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-28 03:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-28 03:40 +0100
RE: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code "byungchul.park" <byungchul.park@lge.com> - 2016-01-28 06:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-28 07:10 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-28 09:30 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-28 11:50 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-28 12:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky@gmail.com> - 2016-01-28 16:50 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Peter Hurley <peter@hurleysoftware.com> - 2016-01-29 00:10 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-29 01:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 02:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-29 04:10 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 05:10 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-29 13:20 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 01:30 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Peter Hurley <peter@hurleysoftware.com> - 2016-01-29 05:40 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 06:30 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Peter Hurley <peter@hurleysoftware.com> - 2016-01-29 06:50 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 07:20 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 07:40 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-31 13:30 +0100
[PATCH 2/3] printk: introduce reset_console_drivers() Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-31 13:40 +0100
Re: [PATCH 2/3] printk: introduce reset_console_drivers() kbuild test robot <lkp@intel.com> - 2016-01-31 13:50 +0100
[PATCH 1/3] printk: introduce console_reset_on_panic() function Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-31 13:40 +0100
[PATCH 3/3] spinlock_debug: panic on recursive lock spin_dump() Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-31 13:40 +0100
Re: [PATCH 3/3] spinlock_debug: panic on recursive lock spin_dump() Sergey Senozhatsky <sergey.senozhatsky@gmail.com> - 2016-02-01 17:20 +0100
Re: [PATCH 3/3] spinlock_debug: panic on recursive lock spin_dump() Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-02-02 09:00 +0100
Re: [PATCH 1/3] printk: introduce console_reset_on_panic() function kbuild test robot <lkp@intel.com> - 2016-01-31 13:50 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-29 08:00 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-01-29 08:20 +0100
Re: [PATCH v4] lib/spinlock_debug.c: prevent a recursive cycle in the debug code Byungchul Park <byungchul.park@lge.com> - 2016-01-29 09:20 +0100
csiph-web