Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1721200
| From | Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH 4.12 75/99] timers: Fix excessive granularity of new timers after a nohz idle |
| Date | 2017-08-28 10:20 +0200 |
| Message-ID | <ujnr7-3du-67@gated-at.bofh.it> (permalink) |
| References | <ujnhn-38X-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
4.12-stable review patch. If anyone has any objections, please let me know.
------------------
From: Nicholas Piggin <npiggin@gmail.com>
commit 2fe59f507a65dbd734b990a11ebc7488f6f87a24 upstream.
When a timer base is idle, it is forwarded when a new timer is added
to ensure that granularity does not become excessive. When not idle,
the timer tick is expected to increment the base.
However there are several problems:
- If an existing timer is modified, the base is forwarded only after
the index is calculated.
- The base is not forwarded by add_timer_on.
- There is a window after a timer is restarted from a nohz idle, after
it is marked not-idle and before the timer tick on this CPU, where a
timer may be added but the ancient base does not get forwarded.
These result in excessive granularity (a 1 jiffy timeout can blow out
to 100s of jiffies), which cause the rcu lockup detector to trigger,
among other things.
Fix this by keeping track of whether the timer base has been idle
since it was last run or forwarded, and if so then forward it before
adding a new timer.
There is still a case where mod_timer optimises the case of a pending
timer mod with the same expiry time, where the timer can see excessive
granularity relative to the new, shorter interval. A comment is added,
but it's not changed because it is an important fastpath for
networking.
This has been tested and found to fix the RCU softlockup messages.
Testing was also done with tracing to measure requested versus
achieved wakeup latencies for all non-deferrable timers in an idle
system (with no lockup watchdogs running). Wakeup latency relative to
absolute latency is calculated (note this suffers from round-up skew
at low absolute times) and analysed:
max avg std
upstream 506.0 1.20 4.68
patched 2.0 1.08 0.15
The bug was noticed due to the lockup detector Kconfig changes
dropping it out of people's .configs and resulting in larger base
clk skew When the lockup detectors are enabled, no CPU can go idle for
longer than 4 seconds, which limits the granularity errors.
Sub-optimal timer behaviour is observable on a smaller scale in that
case:
max avg std
upstream 9.0 1.05 0.19
patched 2.0 1.04 0.11
Fixes: Fixes: a683f390b93f ("timers: Forward the wheel clock whenever possible")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Tested-by: David Miller <davem@davemloft.net>
Cc: dzickus@redhat.com
Cc: sfr@canb.auug.org.au
Cc: mpe@ellerman.id.au
Cc: Stephen Boyd <sboyd@codeaurora.org>
Cc: linuxarm@huawei.com
Cc: abdhalee@linux.vnet.ibm.com
Cc: John Stultz <john.stultz@linaro.org>
Cc: akpm@linux-foundation.org
Cc: paulmck@linux.vnet.ibm.com
Cc: torvalds@linux-foundation.org
Link: http://lkml.kernel.org/r/20170822084348.21436-1-npiggin@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
kernel/time/timer.c | 50 +++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 41 insertions(+), 9 deletions(-)
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -203,6 +203,7 @@ struct timer_base {
bool migration_enabled;
bool nohz_active;
bool is_idle;
+ bool must_forward_clk;
DECLARE_BITMAP(pending_map, WHEEL_SIZE);
struct hlist_head vectors[WHEEL_SIZE];
} ____cacheline_aligned;
@@ -856,13 +857,19 @@ get_target_base(struct timer_base *base,
static inline void forward_timer_base(struct timer_base *base)
{
- unsigned long jnow = READ_ONCE(jiffies);
+ unsigned long jnow;
/*
- * We only forward the base when it's idle and we have a delta between
- * base clock and jiffies.
+ * We only forward the base when we are idle or have just come out of
+ * idle (must_forward_clk logic), and have a delta between base clock
+ * and jiffies. In the common case, run_timers will take care of it.
*/
- if (!base->is_idle || (long) (jnow - base->clk) < 2)
+ if (likely(!base->must_forward_clk))
+ return;
+
+ jnow = READ_ONCE(jiffies);
+ base->must_forward_clk = base->is_idle;
+ if ((long)(jnow - base->clk) < 2)
return;
/*
@@ -938,6 +945,11 @@ __mod_timer(struct timer_list *timer, un
* same array bucket then just return:
*/
if (timer_pending(timer)) {
+ /*
+ * The downside of this optimization is that it can result in
+ * larger granularity than you would get from adding a new
+ * timer with this expiry.
+ */
if (timer->expires == expires)
return 1;
@@ -948,6 +960,7 @@ __mod_timer(struct timer_list *timer, un
* dequeue/enqueue dance.
*/
base = lock_timer_base(timer, &flags);
+ forward_timer_base(base);
clk = base->clk;
idx = calc_wheel_index(expires, clk);
@@ -964,6 +977,7 @@ __mod_timer(struct timer_list *timer, un
}
} else {
base = lock_timer_base(timer, &flags);
+ forward_timer_base(base);
}
ret = detach_if_pending(timer, base, false);
@@ -991,12 +1005,10 @@ __mod_timer(struct timer_list *timer, un
spin_lock(&base->lock);
WRITE_ONCE(timer->flags,
(timer->flags & ~TIMER_BASEMASK) | base->cpu);
+ forward_timer_base(base);
}
}
- /* Try to forward a stale timer base clock */
- forward_timer_base(base);
-
timer->expires = expires;
/*
* If 'idx' was calculated above and the base time did not advance
@@ -1112,6 +1124,7 @@ void add_timer_on(struct timer_list *tim
WRITE_ONCE(timer->flags,
(timer->flags & ~TIMER_BASEMASK) | cpu);
}
+ forward_timer_base(base);
debug_activate(timer, timer->expires);
internal_add_timer(base, timer);
@@ -1497,10 +1510,16 @@ u64 get_next_timer_interrupt(unsigned lo
if (!is_max_delta)
expires = basem + (u64)(nextevt - basej) * TICK_NSEC;
/*
- * If we expect to sleep more than a tick, mark the base idle:
+ * If we expect to sleep more than a tick, mark the base idle.
+ * Also the tick is stopped so any added timer must forward
+ * the base clk itself to keep granularity small. This idle
+ * logic is only maintained for the BASE_STD base, deferrable
+ * timers may still see large granularity skew (by design).
*/
- if ((expires - basem) > TICK_NSEC)
+ if ((expires - basem) > TICK_NSEC) {
+ base->must_forward_clk = true;
base->is_idle = true;
+ }
}
spin_unlock(&base->lock);
@@ -1611,6 +1630,19 @@ static __latent_entropy void run_timer_s
{
struct timer_base *base = this_cpu_ptr(&timer_bases[BASE_STD]);
+ /*
+ * must_forward_clk must be cleared before running timers so that any
+ * timer functions that call mod_timer will not try to forward the
+ * base. idle trcking / clock forwarding logic is only used with
+ * BASE_STD timers.
+ *
+ * The deferrable base does not do idle tracking at all, so we do
+ * not forward it. This can result in very large variations in
+ * granularity for deferrable timers, but they can be deferred for
+ * long periods due to idle.
+ */
+ base->must_forward_clk = false;
+
__run_timers(base);
if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && base->nohz_active)
__run_timers(this_cpu_ptr(&timer_bases[BASE_DEF]));
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH 4.12 00/99] 4.12.10-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 90/99] iio: magnetometer: st_magn: fix status register address for LSM303AGR Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 23/99] irda: do not leak initialized list.dev to userspace Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 40/99] ALSA: firewire: fix NULL pointer dereference when releasing uninitialized data of iso-resource Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 38/99] ALSA: core: Fix unexpected error at replacing user TLV Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 20/99] ipv6: repair fib6 tree in failure case Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 44/99] ARCv2: PAE40: set MSB even if !CONFIG_ARC_HAS_PAE40 but PAE exists in SoC Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 17/99] sctp: fully initialize the IPv6 address in sctp_v6_to_addr() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 68/99] cifs: Fix df output for users with quota limits Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 11/99] net: igmp: Use ingress interface rather than vrf device Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 18/99] tipc: fix use-after-free Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 13/99] ptr_ring: use kmalloc_array() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 36/99] KVM: x86: block guest protection keys unless the host has them enabled Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 31/99] Input: ALPS - fix two-finger scroll breakage in right side on ALPS touchpad Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 10/99] bpf: fix bpf_trace_printk on 32 bit archs Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:10 +0200
[PATCH 4.12 93/99] iommu: Fix wrong freeing of iommu_device->dev Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 75/99] timers: Fix excessive granularity of new timers after a nohz idle Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 79/99] netfilter: nat: fix src map lookup Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 77/99] net: sunrpc: svcsock: fix NULL-pointer exception Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 99/99] powerpc/mm: Ensure cpumask update is ordered Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 81/99] Bluetooth: hidp: fix possible might sleep error in hidp_session_thread Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 10:20 +0200
[PATCH 4.12 91/99] iio: magnetometer: st_magn: remove ihl property for LSM303AGR Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 76/99] x86/mm: Fix use-after-free of ldt_struct Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 80/99] netfilter: nfnetlink: Improve input length sanitization in nfnetlink_rcv Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 69/99] cifs: return ENAMETOOLONG for overlong names in cifs_open()/cifs_lookup() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 96/99] ntb: transport shouldnt disable link due to bogus values in SPADs Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 89/99] iio: hid-sensor-trigger: Fix the race with user space powering up sensors Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 73/99] ftrace: Check for null ret_stack on profile function graph entry function Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 95/99] ntb: ntb_test: ensure the link is up before trying to configure the mws Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 63/99] tracing: Missing error code in tracer_alloc_buffers() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 82/99] Bluetooth: cmtp: fix possible might sleep error in cmtp_session Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 94/99] Clarify (and fix) MAX_LFS_FILESIZE macros Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 74/99] perf/core: Fix group {cpu,task} validation Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 71/99] ring-buffer: Have ring_buffer_alloc_read_page() return error on offline CPU Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 35/99] KVM, pkeys: do not use PKRU value in vcpu->arch.guest_fpu.state Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 83/99] Bluetooth: bnep: fix possible might sleep error in bnep_session Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 72/99] virtio_pci: fix cpu affinity support Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 97/99] ACPI: APD: Fix HID for Hisilicon Hip07/08 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 87/99] ANDROID: binder: fix proc->tsk check. Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 37/99] ALSA: usb-audio: Add delay quirk for H650e/Jabra 550a USB headsets Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 85/99] binder: use group leader instead of open thread Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 78/99] netfilter: expect: fix crash when putting uninited expectation Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 98/99] ACPI: EC: Fix regression related to wrong ECDT initialization order Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 70/99] nfsd: Limit end of page list when decoding NFSv4 WRITE Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 39/99] ALSA: hda - Add stereo mic quirk for Lenovo G50-70 (17aa:3978) Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:10 +0200
[PATCH 4.12 33/99] KVM: s390: sthyi: fix specification exception detection Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 64/99] tracing: Fix kmemleak in tracing_map_array_free() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 52/99] arm64: fpsimd: Prevent registers leaking across exec Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 03/99] bonding: ratelimit failed speed/duplex update warning Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 07/99] ipv4: fix NULL dereference in free_fib_info_rcu() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 47/99] dax: fix deadlock due to misaligned PMD faults Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 57/99] drm/atomic: If the atomic check fails, return its value first Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 45/99] PM/hibernate: touch NMI watchdog when creating snapshot Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 43/99] ARCv2: PAE40: Explicitly set MSB counterpart of SLC region ops addresses Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 08/99] net_sched/sfq: update hierarchical backlog when drop packet Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 61/99] ACPI: device property: Fix node lookup in acpi_graph_get_child_prop_value() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 48/99] i2c: designware: Fix system suspend Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 66/99] RDMA/uverbs: Initialize cq_context appropriately Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 30/99] Input: elan_i2c - add ELAN0602 ACPI ID to support Lenovo Yoga310 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 46/99] mm, shmem: fix handling /sys/kernel/mm/transparent_hugepage/shmem_enabled Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 60/99] Revert "drm/amdgpu: fix vblank_time when displays are off" Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 32/99] KVM: s390: sthyi: fix sthyi inline assembly Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 09/99] net_sched: remove warning from qdisc_hash_add Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 65/99] tracing: Fix freeing of filter in create_filter() when set_str is false Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 41/99] ALSA: firewire-motu: destroy stream data surely at failure of card initialization Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 05/99] dccp: purge write queue in dccp_destroy_sock() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 42/99] ARCv2: SLC: Make sure busy bit is set properly for region ops Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 49/99] mm/madvise.c: fix freeing of locked page with MADV_FREE Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:20 +0200
[PATCH 4.12 21/99] tcp: when rearming RTO, if RTO time is in past then fire RTO ASAP Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 12/99] openvswitch: fix skb_panic due to the incorrect actions attrlen Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 16/99] tun: handle register_netdevice() failures properly Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 24/99] net: sched: fix NULL pointer dereference when action calls some targets Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 02/99] bonding: require speed/duplex only for 802.3ad, alb and tlb Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 28/99] bpf/verifier: fix min/max handling in BPF_SUB Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 26/99] bpf, verifier: add additional patterns to evaluate_reg_imm_alu Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 14/99] ipv4: better IP_MAX_MTU enforcement Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
[PATCH 4.12 19/99] ipv6: reset fn->rr_ptr when replacing route Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-28 11:30 +0200
Re: [PATCH 4.12 00/99] 4.12.10-stable review Shuah Khan <shuahkh@osg.samsung.com> - 2017-08-28 21:50 +0200
Re: [PATCH 4.12 00/99] 4.12.10-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-29 07:00 +0200
Re: [PATCH 4.12 00/99] 4.12.10-stable review Guenter Roeck <linux@roeck-us.net> - 2017-08-29 02:20 +0200
Re: [PATCH 4.12 00/99] 4.12.10-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2017-08-29 07:00 +0200
csiph-web