Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1268785
| From | Luis Henriques <luis.henriques@canonical.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH 3.16.y-ckt 94/94] sched/preempt: Fix cond_resched_lock() and cond_resched_softirq() |
| Date | 2015-11-13 11:20 +0100 |
| Message-ID | <qujCA-56Q-75@gated-at.bofh.it> (permalink) |
| References | <qujCx-56Q-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
3.16.7-ckt20 -stable review patch. If anyone has any objections, please let me know.
------------------
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
commit fe32d3cd5e8eb0f82e459763374aa80797023403 upstream.
These functions check should_resched() before unlocking spinlock/bh-enable:
preempt_count always non-zero => should_resched() always returns false.
cond_resched_lock() worked iff spin_needbreak is set.
This patch adds argument "preempt_offset" to should_resched().
preempt_count offset constants for that:
PREEMPT_DISABLE_OFFSET - offset after preempt_disable()
PREEMPT_LOCK_OFFSET - offset after spin_lock()
SOFTIRQ_DISABLE_OFFSET - offset after local_bh_distable()
SOFTIRQ_LOCK_OFFSET - offset after spin_lock_bh()
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alexander Graf <agraf@suse.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: bdb438065890 ("sched: Extract the basic add/sub preempt_count modifiers")
Link: http://lkml.kernel.org/r/20150715095204.12246.98268.stgit@buzz
Signed-off-by: Ingo Molnar <mingo@kernel.org>
[ luis: backported to 3.16:
- some of the changes done in preempt_mask.h instead of preempt.h
- adjusted context ]
Signed-off-by: Luis Henriques <luis.henriques@canonical.com>
---
arch/x86/include/asm/preempt.h | 4 ++--
include/asm-generic/preempt.h | 5 +++--
include/linux/preempt.h | 5 +++--
include/linux/preempt_mask.h | 14 +++++++++++---
include/linux/sched.h | 6 ------
kernel/sched/core.c | 6 +++---
6 files changed, 22 insertions(+), 18 deletions(-)
diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h
index 7024c12f7bfe..cebe6d05891e 100644
--- a/arch/x86/include/asm/preempt.h
+++ b/arch/x86/include/asm/preempt.h
@@ -93,9 +93,9 @@ static __always_inline bool __preempt_count_dec_and_test(void)
/*
* Returns true when we need to resched and can (barring IRQ state).
*/
-static __always_inline bool should_resched(void)
+static __always_inline bool should_resched(int preempt_offset)
{
- return unlikely(!raw_cpu_read_4(__preempt_count));
+ return unlikely(raw_cpu_read_4(__preempt_count) == preempt_offset);
}
#ifdef CONFIG_PREEMPT
diff --git a/include/asm-generic/preempt.h b/include/asm-generic/preempt.h
index 1cd3f5d767a8..54352f4dde1a 100644
--- a/include/asm-generic/preempt.h
+++ b/include/asm-generic/preempt.h
@@ -74,9 +74,10 @@ static __always_inline bool __preempt_count_dec_and_test(void)
/*
* Returns true when we need to resched and can (barring IRQ state).
*/
-static __always_inline bool should_resched(void)
+static __always_inline bool should_resched(int preempt_offset)
{
- return unlikely(!preempt_count() && tif_need_resched());
+ return unlikely(preempt_count() == preempt_offset &&
+ tif_need_resched());
}
#ifdef CONFIG_PREEMPT
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index de83b4eb1642..8cd6725c5758 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -20,7 +20,8 @@
#if defined(CONFIG_DEBUG_PREEMPT) || defined(CONFIG_PREEMPT_TRACER)
extern void preempt_count_add(int val);
extern void preempt_count_sub(int val);
-#define preempt_count_dec_and_test() ({ preempt_count_sub(1); should_resched(); })
+#define preempt_count_dec_and_test() \
+ ({ preempt_count_sub(1); should_resched(0); })
#else
#define preempt_count_add(val) __preempt_count_add(val)
#define preempt_count_sub(val) __preempt_count_sub(val)
@@ -59,7 +60,7 @@ do { \
#define preempt_check_resched() \
do { \
- if (should_resched()) \
+ if (should_resched(0)) \
__preempt_schedule(); \
} while (0)
diff --git a/include/linux/preempt_mask.h b/include/linux/preempt_mask.h
index 1f654ee836b7..5cb25f17331a 100644
--- a/include/linux/preempt_mask.h
+++ b/include/linux/preempt_mask.h
@@ -71,13 +71,21 @@
*/
#define in_nmi() (preempt_count() & NMI_MASK)
+/*
+ * The preempt_count offset after preempt_disable();
+ */
#if defined(CONFIG_PREEMPT_COUNT)
-# define PREEMPT_DISABLE_OFFSET 1
+# define PREEMPT_DISABLE_OFFSET PREEMPT_OFFSET
#else
-# define PREEMPT_DISABLE_OFFSET 0
+# define PREEMPT_DISABLE_OFFSET 0
#endif
/*
+ * The preempt_count offset after spin_lock()
+ */
+#define PREEMPT_LOCK_OFFSET PREEMPT_DISABLE_OFFSET
+
+/*
* The preempt_count offset needed for things like:
*
* spin_lock_bh()
@@ -90,7 +98,7 @@
*
* Work as expected.
*/
-#define SOFTIRQ_LOCK_OFFSET (SOFTIRQ_DISABLE_OFFSET + PREEMPT_DISABLE_OFFSET)
+#define SOFTIRQ_LOCK_OFFSET (SOFTIRQ_DISABLE_OFFSET + PREEMPT_LOCK_OFFSET)
/*
* Are we running in atomic context? WARNING: this macro cannot
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 4bab946551e9..a632aaad1e59 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2736,12 +2736,6 @@ extern int _cond_resched(void);
extern int __cond_resched_lock(spinlock_t *lock);
-#ifdef CONFIG_PREEMPT_COUNT
-#define PREEMPT_LOCK_OFFSET PREEMPT_OFFSET
-#else
-#define PREEMPT_LOCK_OFFSET 0
-#endif
-
#define cond_resched_lock(lock) ({ \
__might_sleep(__FILE__, __LINE__, PREEMPT_LOCK_OFFSET); \
__cond_resched_lock(lock); \
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index c1d7818dade9..c80729679806 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4172,7 +4172,7 @@ static void __cond_resched(void)
int __sched _cond_resched(void)
{
- if (should_resched()) {
+ if (should_resched(0)) {
__cond_resched();
return 1;
}
@@ -4190,7 +4190,7 @@ EXPORT_SYMBOL(_cond_resched);
*/
int __cond_resched_lock(spinlock_t *lock)
{
- int resched = should_resched();
+ int resched = should_resched(PREEMPT_LOCK_OFFSET);
int ret = 0;
lockdep_assert_held(lock);
@@ -4212,7 +4212,7 @@ int __sched __cond_resched_softirq(void)
{
BUG_ON(!in_softirq());
- if (should_resched()) {
+ if (should_resched(SOFTIRQ_DISABLE_OFFSET)) {
local_bh_enable();
__cond_resched();
local_bh_disable();
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[3.16.y-ckt stable] Linux 3.16.7-ckt20 stable review Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 05/94] svcrdma: handle rdma read with a non-zero initial page offset Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 92/94] sfc: Fix memcpy() with const destination compiler warning. Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 89/94] ppp, slip: Validate VJ compression slot parameters completely Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 88/94] isdn_ppp: Add checks for allocation failure in isdn_ppp_open() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 52/94] ASoC: Add info callback for SX_TLV controls Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 46/94] ARM: orion: Fix DSA platform device after mvmdio conversion Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 43/94] iwlwifi: mvm: fix D3 firmware PN programming Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 35/94] i2c: designware: Do not use parameters from ACPI on Dell Inspiron 7348 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 14/94] usb: Add device quirk for Logitech PTZ cameras Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 22/94] drm/radeon: add pm sysfs files late Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 39/94] rbd: fix double free on rbd_dev->header_name Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 71/94] i2c: mv64xxx: really allow I2C offloading Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 91/94] power: bq24190_charger: suppress build warning Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 90/94] staging/dgnc: fix info leak in ioctl Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 33/94] i2c: s3c2410: enable RuntimePM before registering to the core Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 86/94] serial: 8250_pci: Add support for 12 port Exar boards Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 94/94] sched/preempt: Fix cond_resched_lock() and cond_resched_softirq() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 93/94] sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:20 +0100 [PATCH 3.16.y-ckt 65/94] iwlwifi: mvm: init card correctly on ctkill exit check Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 70/94] iommu/amd: Don't clear DTE flags when modifying it Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 77/94] drm/radeon: fix dpms when driver backlight control is disabled Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 69/94] md/raid10: submit_bio_wait() returns 0 on success Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 67/94] crypto: api - Only abort operations on fatal signal Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 66/94] module: Fix locking in symbol_put_addr() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 74/94] Revert "ARM64: unwind: Fix PC calculation" Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 82/94] ppp: fix pppoe_dev deletion condition in pppoe_release() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 72/94] drm/radeon: don't try to recreate sysfs entries on resume Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 63/94] dm btree: fix leak of bufio-backed block in btree_split_beneath error path Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 55/94] xhci: Add spurious wakeup quirk for LynxPoint-LP controllers Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 80/94] Revert "md: allow a partially recovered device to be hot-added to an array." Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 76/94] drm/radeon: move bl encoder assignment into bl init Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 83/94] ipv6: gre: support SIT encapsulation Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:30 +0100 [PATCH 3.16.y-ckt 50/94] drm/i915: Deny wrapping an userptr into a framebuffer Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 58/94] powerpc/rtas: Validate rtas.entry before calling enter_rtas() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 53/94] xhci: don't finish a TD if we get a short transfer event mid TD Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 48/94] iio: mxs-lradc: Fix temperature offset Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 62/94] dm btree remove: fix a bug when rebalancing nodes after removal Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 57/94] drm/nouveau/gem: return only valid domain when there's only one Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 59/94] mm: make sendfile(2) killable Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 56/94] ASoC: wm8904: Correct number of EQ registers Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 44/94] iwlwifi: fix firmware filename for 3160 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 45/94] iwlwifi: pci: add a few more PCI subvendor IDs for the 7265 series Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 54/94] xhci: handle no ping response error properly Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 47/94] xen-blkfront: check for null drvdata in blkback_changed (XenbusStateClosing) Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 51/94] x86/efi: Fix multiple GOP device support Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 49/94] ALSA: hda - Fix inverted internal mic on Lenovo G50-80 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 61/94] rbd: prevent kernel stack blow up on rbd map Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:40 +0100 [PATCH 3.16.y-ckt 23/94] drm/radeon: add quirk for MSI R7 370 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 34/94] i2c: designware-platdrv: enable RuntimePM before registering to the core Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 36/94] btrfs: fix possible leak in btrfs_ioctl_balance() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 27/94] arm64: errata: use KBUILD_CFLAGS_MODULE for erratum #843419 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 41/94] iio: accel: sca3000: memory corruption in sca3000_read_first_n_hw_rb() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 31/94] btrfs: fix use after free iterating extrefs Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 29/94] dm thin: fix missing pool reference count decrement in pool_ctr error path Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 38/94] usb: xhci: Add support for URB_ZERO_PACKET to bulk/sg transfers Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 30/94] btrfs: check unsupported filters in balance arguments Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 42/94] iwlwifi: dvm: fix D3 firmware PN programming Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 28/94] crypto: ahash - ensure statesize is non-zero Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 40/94] ath9k: declare required extra tx headroom Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 11:50 +0100 [PATCH 3.16.y-ckt 25/94] crypto: sparc - initialize blkcipher.ivsize Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 21/94] workqueue: make sure delayed work run in local cpu Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 04/94] s390/boot/decompression: disable floating point in decompressor Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 10/94] dm raid: fix round up of default region size Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 11/94] ALSA: hda: Add dock support for ThinkPad T550 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 16/94] ALSA: synth: Fix conflicting OSS device registration on AWE32 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 13/94] USB: Add reset-resume quirk for two Plantronics usb headphones. Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 03/94] x86/xen: Support kexec/kdump in HVM guests by doing a soft reset Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 02/94] regmap: debugfs: Don't bother actually printing when calculating max length Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 09/94] ASoC: dwc: correct irq clear method Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 12/94] ALSA: hda - Apply SPDIF pin ctl to MacBookPro 12,1 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 06/94] dm: fix AB-BA deadlock in __dm_destroy() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 26/94] drm/nouveau/fbcon: take runpm reference when userspace has an open fd Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 15/94] staging: speakup: fix speakup-r regression Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 20/94] 3w-9xxx: don't unmap bounce buffered commands Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 17/94] arm64: readahead: fault retry breaks mmap file read random detection Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 01/94] regmap: debugfs: Ensure we don't underflow when printing access masks Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 24/94] drm/radeon: add quirk for ASUS R7 370 Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 07/94] [SMB3] Do not fall back to SMBWriteX in set_file_size error cases Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100 [PATCH 3.16.y-ckt 18/94] sched/core: Fix TASK_DEAD race in finish_task_switch() Luis Henriques <luis.henriques@canonical.com> - 2015-11-13 12:00 +0100
csiph-web