[PATCH 3.19.y-ckt 15/86] x86/apic: Serialize LVTT and TSC_DEADLINE writes

From	Kamal Mostafa <kamal@canonical.com>
Newsgroups	linux.kernel
Subject	[PATCH 3.19.y-ckt 15/86] x86/apic: Serialize LVTT and TSC_DEADLINE writes
Date	2015-10-27 23:10 +0100
Message-ID	<qokBl-2oh-65@gated-at.bofh.it> (permalink)
References	<qok8i-1XT-9@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

3.19.8-ckt9 -stable review patch.  If anyone has any objections, please let me know.

------------------

From: Shaohua Li <shli@fb.com>

commit 5d7c631d926b59aa16f3c56eaeb83f1036c81dc7 upstream.

The APIC LVTT register is MMIO mapped but the TSC_DEADLINE register is an
MSR. The write to the TSC_DEADLINE MSR is not serializing, so it's not
guaranteed that the write to LVTT has reached the APIC before the
TSC_DEADLINE MSR is written. In such a case the write to the MSR is
ignored and as a consequence the local timer interrupt never fires.

The SDM decribes this issue for xAPIC and x2APIC modes. The
serialization methods recommended by the SDM differ.

xAPIC:
 "1. Memory-mapped write to LVT Timer Register, setting bits 18:17 to 10b.
  2. WRMSR to the IA32_TSC_DEADLINE MSR a value much larger than current time-stamp counter.
  3. If RDMSR of the IA32_TSC_DEADLINE MSR returns zero, go to step 2.
  4. WRMSR to the IA32_TSC_DEADLINE MSR the desired deadline."

x2APIC:
 "To allow for efficient access to the APIC registers in x2APIC mode,
  the serializing semantics of WRMSR are relaxed when writing to the
  APIC registers. Thus, system software should not use 'WRMSR to APIC
  registers in x2APIC mode' as a serializing instruction. Read and write
  accesses to the APIC registers will occur in program order. A WRMSR to
  an APIC register may complete before all preceding stores are globally
  visible; software can prevent this by inserting a serializing
  instruction, an SFENCE, or an MFENCE before the WRMSR."

The xAPIC method is to just wait for the memory mapped write to hit
the LVTT by checking whether the MSR write has reached the hardware.
There is no reason why a proper MFENCE after the memory mapped write would
not do the same. Andi Kleen confirmed that MFENCE is sufficient for the
xAPIC case as well.

Issue MFENCE before writing to the TSC_DEADLINE MSR. This can be done
unconditionally as all CPUs which have TSC_DEADLINE also have MFENCE
support.

[ tglx: Massaged the changelog ]

Signed-off-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Cc: <Kernel-team@fb.com>
Cc: <lenb@kernel.org>
Cc: <fenghua.yu@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: http://lkml.kernel.org/r/20150909041352.GA2059853@devbig257.prn2.facebook.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Kamal Mostafa <kamal@canonical.com>
---
 arch/x86/kernel/apic/apic.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 29b5b18..c803cda 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -366,6 +366,13 @@ static void __setup_APIC_LVTT(unsigned int clocks, int oneshot, int irqen)
 	apic_write(APIC_LVTT, lvtt_value);
 
 	if (lvtt_value & APIC_LVT_TIMER_TSCDEADLINE) {
+		/*
+		 * See Intel SDM: TSC-Deadline Mode chapter. In xAPIC mode,
+		 * writing to the APIC LVTT and TSC_DEADLINE MSR isn't serialized.
+		 * According to Intel, MFENCE can do the serialization here.
+		 */
+		asm volatile("mfence" : : : "memory");
+
 		printk_once(KERN_DEBUG "TSC deadline timer enabled\n");
 		return;
 	}
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Thread

[3.19.y-ckt stable] Linux 3.19.8-ckt9 stable review Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:40 +0100
  [PATCH 3.19.y-ckt 86/86] fib_rules: fix fib rule dumps across multiple skbs Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:40 +0100
  [PATCH 3.19.y-ckt 85/86] net/ipv6: Correct PIM6 mrt_lock handling Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:40 +0100
  [PATCH 3.19.y-ckt 05/86] spi: spi-pxa2xx: Check status register to determine if SSSR_TINT is disabled Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:40 +0100
  [PATCH 3.19.y-ckt 83/86] usbnet: Get EVENT_NO_RUNTIME_PM bit before it is cleared Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:50 +0100
  [PATCH 3.19.y-ckt 81/86] netlink, mmap: transform mmap skb into full skb on taps Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:50 +0100
  [PATCH 3.19.y-ckt 82/86] nfs: fix pg_test page count calculation Kamal Mostafa <kamal@canonical.com> - 2015-10-27 22:50 +0100
  [PATCH 3.19.y-ckt 54/86] irqchip/atmel-aic5: Use per chip mask caches in mask/unmask() Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 69/86] netfilter: nft_compat: skip family comparison in case of NFPROTO_UNSPEC Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 71/86] netfilter: nf_log: wait for rcu grace after logger unregistration Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 55/86] spi: xtensa-xtfpga: fix register endianness Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 80/86] ARM: dts: omap5-uevm.dts: fix i2c5 pinctrl offsets Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 78/86] ASoC: fix broken pxa SoC support Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 60/86] ASoC: pxa: pxa2xx-ac97: fix dma requestor lines Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 61/86] ocfs2/dlm: fix deadlock when dispatch assert master Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 72/86] ARM: OMAP2+: board-generic: Remove stale of_irq macros Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 62/86] drm/i915/bios: handle MIPI Sequence Block v3+ gracefully Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 74/86] ARM: 8425/1: kgdb: Don't try to stop the machine when setting breakpoints Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 66/86] PCI: Use function 0 VPD for identical functions, regular VPD for others Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 79/86] netfilter: nf_log: don't zap all loggers on unregister Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 73/86] vxlan: set needed headroom correctly Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 59/86] x86/nmi/64: Fix a paravirt stack-clobbering bug in the NMI code Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 68/86] perf header: Fixup reading of HEADER_NRCPUS feature Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 58/86] x86/paravirt: Replace the paravirt nop with a bona fide empty function Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 76/86] net/mlx4_en: really allow to change RSS key Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 67/86] spi: spidev: fix possible NULL dereference Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 65/86] PCI: Fix devfn for VPD access through function 0 Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 64/86] [SMB3] Fix sec=krb5 on smb3 mounts Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 75/86] tcp: add proper TS val into RST packets Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 77/86] macvtap: fix TUNSETSNDBUF values > 64k Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 63/86] drm/qxl: only report first monitor as connected if we have no state Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 50/86] usb: xhci: exit early in xhci_setup_device() if we're halted or dying Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 70/86] openvswitch: Zero flows on allocation. Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:00 +0100
  [PATCH 3.19.y-ckt 44/86] sched: access local runqueue directly in single_task_running Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 46/86] usb: Use the USB_SS_MULT() macro to get the burst multiplier. Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 53/86] xhci: init command timeout timer earlier to avoid deleting it uninitialized Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 33/86] arm: KVM: Fix incorrect device to IPA mapping Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 39/86] arm64: KVM: Disable virtual timer even if the guest is not using it Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 30/86] powerpc/boot: Specify ABI v2 when building an LE boot wrapper Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 37/86] arm64: errata: add module build workaround for erratum #843419 Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 34/86] x86/platform: Fix Geode LX timekeeping in the generic x86 build Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 35/86] arm/arm64: KVM: vgic: Check for !irqchip_in_kernel() when mapping resources Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 42/86] hp-wmi: limit hotkey enable Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 49/86] usb: xhci: Clear XHCI_STATE_DYING on start Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 45/86] KVM: x86: trap AMD MSRs for the TSeg base and mask Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 36/86] arm64: compat: fix vfp save/restore across signal handlers in big-endian Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 48/86] usb: xhci: lock mutex on xhci_stop Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 41/86] staging: ion: fix corruption of ion_import_dma_buf Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 43/86] zram: fix possible use after free in zcomp_create() Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 32/86] KVM: vmx: fix VPID is 0000H in non-root operation Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 52/86] xhci-mem: Use setup_timer Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 26/86] net: mvneta: fix DMA buffer unmapping in mvneta_rx() Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 15/86] x86/apic: Serialize LVTT and TSC_DEADLINE writes Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 11/86] time: Fix timekeeping_freqadjust()'s incorrect use of abs() instead of abs64() Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 31/86] powerpc/mm: Recompute hash value after a failed update Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 51/86] xhci: change xhci 1.0 only restrictions to support xhci 1.1 Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 38/86] s390/compat: correct uc_sigmask of the compat signal frame Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 47/86] xhci: give command abortion one more chance before killing xhci Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 40/86] arm: KVM: Disable virtual timer even if the guest is not using it Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:10 +0100
  [PATCH 3.19.y-ckt 09/86] CIFS: fix type confusion in copy offload ioctl Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 07/86] ARM: 8429/1: disable GCC SRA optimization Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 03/86] KEYS: Fix crash when attempt to garbage collect an uninstantiated keyring Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 14/86] KVM: arm64: add workaround for Cortex-A57 erratum #852523 Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 20/86] cxl: Fix unbalanced pci_dev_get in cxl_probe Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 08/86] powerpc/MSI: Fix race condition in tearing down MSI interrupts Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 29/86] ARM: fix Thumb2 signal handling when ARMv6 is enabled Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 17/86] Btrfs: fix read corruption of compressed and shared extents Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 27/86] iser-target: remove command with state ISTATE_REMOVE Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 25/86] kvm: fix zero length mmio searching Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 13/86] drm/vmwgfx: Fix up user_dmabuf refcounting Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
    Re: [PATCH 3.19.y-ckt 13/86] drm/vmwgfx: Fix up user_dmabuf  refcounting Thomas Hellstrom <thellstrom@vmware.com> - 2015-10-28 08:40 +0100
      Re: [PATCH 3.19.y-ckt 13/86] drm/vmwgfx: Fix up user_dmabuf  refcounting Kamal Mostafa <kamal@canonical.com> - 2015-10-28 20:20 +0100
        Re: [PATCH 3.19.y-ckt 13/86] drm/vmwgfx: Fix up user_dmabuf  refcounting Kamal Mostafa <kamal@canonical.com> - 2015-10-28 20:30 +0100
  [PATCH 3.19.y-ckt 16/86] ARM: dts: omap3-beagle: make i2c3, ddc and tfp410 gpio work again Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 06/86] spi: Fix documentation of spi_alloc_master() Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 21/86] arm64: head.S: initialise mdcr_el2 in el2_setup Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 28/86] iser-target: Put the reference on commands waiting for unsol data Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 02/86] KEYS: Fix race between key destruction and finding a keyring by name Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 01/86] sctp: donot reset the overall_error_count in SHUTDOWN_RECEIVE state Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 24/86] kvm: fix double free for fast mmio eventfd Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 12/86] USB: option: add ZTE PIDs Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 10/86] hwmon: (nct6775) Swap STEP_UP_TIME and STEP_DOWN_TIME registers for most chips Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:20 +0100
  [PATCH 3.19.y-ckt 04/86] KEYS: Don't permit request_key() to construct a new keyring Kamal Mostafa <kamal@canonical.com> - 2015-10-27 23:30 +0100

csiph-web