Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1409517

[PATCH 4.6 076/100] locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait()

Show all headers | View raw

4.6-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Peter Zijlstra <peterz@infradead.org>

commit 54cf809b9512be95f53ed4a5e3b631d1ac42f0fa upstream.

Similar to commits:

  51d7d5205d33 ("powerpc: Add smp_mb() to arch_spin_is_locked()")
  d86b8da04dfa ("arm64: spinlock: serialise spin_unlock_wait against concurrent lockers")

qspinlock suffers from the fact that the _Q_LOCKED_VAL store is
unordered inside the ACQUIRE of the lock.

And while this is not a problem for the regular mutual exclusive
critical section usage of spinlocks, it breaks creative locking like:

	spin_lock(A)			spin_lock(B)
	spin_unlock_wait(B)		if (!spin_is_locked(A))
	do_something()			  do_something()

In that both CPUs can end up running do_something at the same time,
because our _Q_LOCKED_VAL store can drop past the spin_unlock_wait()
spin_is_locked() loads (even on x86!!).

To avoid making the normal case slower, add smp_mb()s to the less used
spin_unlock_wait() / spin_is_locked() side of things to avoid this
problem.

Reported-and-tested-by: Davidlohr Bueso <dave@stgolabs.net>
Reported-by: Giovanni Gherdovich <ggherdovich@suse.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 include/asm-generic/qspinlock.h |   27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

--- a/include/asm-generic/qspinlock.h
+++ b/include/asm-generic/qspinlock.h
@@ -28,7 +28,30 @@
  */
 static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
 {
-	return atomic_read(&lock->val);
+	/*
+	 * queued_spin_lock_slowpath() can ACQUIRE the lock before
+	 * issuing the unordered store that sets _Q_LOCKED_VAL.
+	 *
+	 * See both smp_cond_acquire() sites for more detail.
+	 *
+	 * This however means that in code like:
+	 *
+	 *   spin_lock(A)		spin_lock(B)
+	 *   spin_unlock_wait(B)	spin_is_locked(A)
+	 *   do_something()		do_something()
+	 *
+	 * Both CPUs can end up running do_something() because the store
+	 * setting _Q_LOCKED_VAL will pass through the loads in
+	 * spin_unlock_wait() and/or spin_is_locked().
+	 *
+	 * Avoid this by issuing a full memory barrier between the spin_lock()
+	 * and the loads in spin_unlock_wait() and spin_is_locked().
+	 *
+	 * Note that regular mutual exclusion doesn't care about this
+	 * delayed store.
+	 */
+	smp_mb();
+	return atomic_read(&lock->val) & _Q_LOCKED_MASK;
 }
 
 /**
@@ -108,6 +131,8 @@ static __always_inline void queued_spin_
  */
 static inline void queued_spin_unlock_wait(struct qspinlock *lock)
 {
+	/* See queued_spin_is_locked() */
+	smp_mb();
 	while (atomic_read(&lock->val) & _Q_LOCKED_MASK)
 		cpu_relax();
 }

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 4.6 000/100] 4.6.1-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 074/100] serial: 8250_mid: recognize interrupt source in handler Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 070/100] tty/serial: atmel: fix hardware handshake selection Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 077/100] clk: bcm2835: add locking to pll*_on/off methods Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 069/100] TTY: n_gsm, fix false positive WARN_ON Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 055/100] usb: misc: usbtest: fix pattern tests for scatterlists. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 073/100] serial: 8250_mid: use proper bar for DNV platform Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 079/100] watchdog: sp5100_tco: properly check for new register layouts Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 071/100] Fix OpenSSH pty regression on close Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 056/100] USB: leave LPM alone if possible when binding/unbinding interface drivers Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 022/100] fscrypto/f2fs: allow fs-specific key prefix for fs encryption Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 076/100] locking,qspinlock: Fix spin_is_locked() and spin_unlock_wait() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 007/100] kvm: arm64: Fix EC field in inject_abt64 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 068/100] tty: vt, return error when con_startup fails Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 067/100] x86/cpufeature, x86/mm/pkeys: Fix broken compile-time disabling of pkeys Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 042/100] mei: amthif: discard not read messages Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 008/100] perf/x86/intel/uncore: Remove WARN_ON_ONCE in uncore_pci_probe Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 066/100] xen/x86: actually allocate legacy interrupts on PV guests Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 043/100] mei: bus: call mei_cl_read_start under device lock Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 081/100] Fixing oops in callback path Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 059/100] usb: host: xhci-rcar: Avoid long wait in xhci_reset() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 044/100] USB: serial: cp210x: fix hardware flow-control disable Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 060/100] staging: comedi: das1800: fix possible NULL dereference Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:30 +0200
  [PATCH 4.6 039/100] Bluetooth: vhci: purge unhandled skbs Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:40 +0200
  [PATCH 4.6 038/100] Bluetooth: vhci: fix open_timeout vs. hdev race Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:40 +0200
  [PATCH 4.6 016/100] ring-buffer: Prevent overflow of size in ring_buffer_resize() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:40 +0200
  [PATCH 4.6 033/100] mmc: sdhci-acpi: Remove MMC_CAP_BUS_WIDTH_TEST for Intel controllers Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 037/100] mmc: sdhci-pci: Remove MMC_CAP_BUS_WIDTH_TEST for Intel controllers Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 031/100] ACPI / PM: Export acpi_device_fix_up_power() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 027/100] irqchip/gic-v3: Configure all interrupts as non-secure Group-1 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 036/100] mmc: longer timeout for long read time quirk Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 020/100] clk: qcom: msm8916: Fix crypto clock flags Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 024/100] mfd: omap-usb-tll: Fix scheduling while atomic BUG Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 018/100] crypto: talitos - fix ahash algorithms registration Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 002/100] arm64: Ensure pmd_present() returns false after pmd_mknotpresent() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 025/100] Input: pwm-beeper - fix - scheduling while atomic Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 026/100] irqchip/gic: Ensure ordering between read of INTACK and shared data Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 030/100] mmc: mmc: Fix partition switch timeout for some eMMCs Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 003/100] arm64: Implement ptep_set_access_flags() for hardware AF/DBM Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 019/100] crypto: sun4i-ss - Replace spinlock_bh by spin_lock_irq{save|restore} Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 023/100] sched/loadavg: Fix loadavg artifacts on fully idle and on fully loaded systems Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 017/100] crypto: caam - fix caam_jr_alloc() ret code Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  [PATCH 4.6 034/100] ACPI / osi: Fix an issue that acpi_osi=!* cannot disable ACPICA internal strings Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-05-30 23:50 +0200
  Re: [PATCH 4.6 000/100] 4.6.1-stable review Guenter Roeck <linux@roeck-us.net> - 2016-06-01 07:40 +0200
    Re: [PATCH 4.6 000/100] 4.6.1-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-06-01 18:20 +0200
      Re: [PATCH 4.6 000/100] 4.6.1-stable review "Xuetao Guan" <gxt@mprc.pku.edu.cn> - 2016-06-02 06:40 +0200
  Re: [PATCH 4.6 000/100] 4.6.1-stable review Shuah Khan <shuahkh@osg.samsung.com> - 2016-06-01 16:30 +0200

csiph-web