Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1525994 > unrolled thread

[patch 0/8] x86/tsc: Utilize TSC_ADJUST MSR

Back to article view | Back to linux.kernel

Contents

  [patch 0/8] x86/tsc: Utilize TSC_ADJUST MSR Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
    [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
      Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle Peter Zijlstra <peterz@infradead.org> - 2016-11-20 14:20 +0100
        Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle Thomas Gleixner <tglx@linutronix.de> - 2016-11-21 09:20 +0100
          Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle Peter Zijlstra <peterz@infradead.org> - 2016-11-21 12:10 +0100
      Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle Andi Kleen <andi@firstfloor.org> - 2016-11-22 00:00 +0100
    [patch 5/8] x86/tsc: Sync test only for the first cpu in a package Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
    [patch 3/8] x86/tsc: Store and check TSC ADJUST MSR Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
    [patch 1/8] x86/tsc: Use X86_FEATURE_TSC_ADJUST in detect_art() Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
    [patch 2/8] x86/tsc: Detect random warps Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100
    [patch 7/8] x86/tsc: Prepare warp test for TSC adjustment Thomas Gleixner <tglx@linutronix.de> - 2016-11-19 15:00 +0100

#1525994 — [patch 0/8] x86/tsc: Utilize TSC_ADJUST MSR

The TSC_ADJUST MSR shows whether the TSC has been modified. This is helpful
in two aspects:

1) It allows to detect BIOS wreckage, where SMM code tries to 'hide' the
   cycles spent by storing the TSC value at SMM entry and restoring it at
   SMM exit. On affected machines the TSCs run slowly out of sync up to the
   point where the clocksource watchdog (if available) detects it.

   The TSC_ADJUST MSR allows to detect the TSC modification before that and
   eventually restore it. This is also important for SoCs which have no
   watchdog clocksource and therefore TSC wreckage cannot be detected and
   acted upon.

2) All threads in a package are required to have the same TSC_ADJUST
   value. Broken BIOSes break that and as a result the TSC synchronization
   check fails.

   The TSC_ADJUST MSR allows to detect the deviation when a CPU comes
   online. If detected set it to the value of an already online CPU in the
   same package. This also allows to reduce the number of sync tests
   because with that in place the test is only required for the first CPU
   in a package.

   In principle all CPUs in a system should have the same TSC_ADJUST value
   even across packages, but with physical CPU hotplug this assumption is
   not true because the TSC starts with power on, so physical hotplug has
   to do some trickery to bring the TSC into sync with already running
   packages, which requires to use an TSC_ADJUST value different from CPUs
   which got powered earlier.

   A final enhancement is the opportunity to compensate for unsynced TSCs
   accross nodes at boot time and make the TSC usable that way. It won't
   help for TSCs which run apart due to frequency skew between packages,
   but this gets detected by the clocksource watchdog later.

This patch series implements all of the avove.

Thanks,

	tglx

---
 include/asm/tsc.h |    8 +
 kernel/Makefile   |    2 
 kernel/process.c  |    1 
 kernel/tsc.c      |   16 ++-
 kernel/tsc_sync.c |  237 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 5 files changed, 244 insertions(+), 20 deletions(-)

[toc] | [next] | [standalone]

#1525995 — [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle

When entering idle, it's a good oportunity to verify that the TSC_ADJUST
MSR has not been tampered with (BIOS hiding SMM cycles). If tampering is
detected, emit a warning and restore it to the previous value.

This is especially important for machines, which mark the TSC reliable
because there is no watchdog clocksource available (SoCs).

This is not sufficient for HPC (NOHZ_FULL) situations where a CPU never
goes idle, but adding a timer to do the check periodically is not an option
either. On a machine, which has this issue, the check triggeres right
during boot, so there is a decent chance that the sysadmin will notice.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/tsc.h |    2 ++
 arch/x86/kernel/process.c  |    1 +
 arch/x86/kernel/tsc_sync.c |   20 +++++++++++++++++++-
 3 files changed, 22 insertions(+), 1 deletion(-)

--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -50,8 +50,10 @@ extern void check_tsc_sync_target(void);
 
 #ifdef CONFIG_X86_TSC
 extern void tsc_store_and_check_tsc_adjust(void);
+extern void tsc_verify_tsc_adjust(void);
 #else
 static inline void tsc_store_and_check_tsc_adjust(void) { }
+static inline void tsc_verify_tsc_adjust(void) { }
 #endif
 
 extern int notsc_setup(char *);
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -277,6 +277,7 @@ void exit_idle(void)
 
 void arch_cpu_idle_enter(void)
 {
+	tsc_verify_tsc_adjust();
 	local_touch_nmi();
 	enter_idle();
 }
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -28,6 +28,24 @@ struct tsc_adjust {
 
 static DEFINE_PER_CPU(struct tsc_adjust, tsc_adjust);
 
+void tsc_verify_tsc_adjust(void)
+{
+	struct tsc_adjust *adj = this_cpu_ptr(&tsc_adjust);
+	s64 curval;
+
+	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
+		return;
+
+	rdmsrl(MSR_IA32_TSC_ADJUST, curval);
+	if (adj->adjusted == curval)
+		return;
+
+	pr_warn(FW_BUG "TSC ADJUST differs: CPU%u %lld --> %lld. Restoring\n",
+		smp_processor_id(), adj->adjusted, curval);
+
+	wrmsrl(MSR_IA32_TSC_ADJUST, adj->adjusted);
+}
+
 #ifndef CONFIG_SMP
 void __init tsc_store_and_check_tsc_adjust(void)
 {
@@ -40,7 +58,7 @@ void __init tsc_store_and_check_tsc_adju
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
 	cur->adjusted = bootval;
-	pr_info("TSC ADJUST: Boot CPU%u: %lld\n",cpu,  bootval);
+	pr_info("TSC ADJUST: Boot CPU%u: %lld\n", cpu,  bootval);
 }
 
 #else /* !CONFIG_SMP */

[toc] | [prev] | [next] | [standalone]

#1526236 — Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle

On Sat, Nov 19, 2016 at 01:47:37PM -0000, Thomas Gleixner wrote:
> When entering idle, it's a good oportunity to verify that the TSC_ADJUST
> MSR has not been tampered with (BIOS hiding SMM cycles). If tampering is
> detected, emit a warning and restore it to the previous value.

> +++ b/arch/x86/kernel/process.c
> @@ -277,6 +277,7 @@ void exit_idle(void)
>  
>  void arch_cpu_idle_enter(void)
>  {
> +	tsc_verify_tsc_adjust();
>  	local_touch_nmi();
>  	enter_idle();
>  }

Doing a RDMSR on the idle path isn't going to be popular. That path is
already way too slow.

[toc] | [prev] | [next] | [standalone]

#1526442 — Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle

On Sun, 20 Nov 2016, Peter Zijlstra wrote:
> On Sat, Nov 19, 2016 at 01:47:37PM -0000, Thomas Gleixner wrote:
> > When entering idle, it's a good oportunity to verify that the TSC_ADJUST
> > MSR has not been tampered with (BIOS hiding SMM cycles). If tampering is
> > detected, emit a warning and restore it to the previous value.
> 
> > +++ b/arch/x86/kernel/process.c
> > @@ -277,6 +277,7 @@ void exit_idle(void)
> >  
> >  void arch_cpu_idle_enter(void)
> >  {
> > +	tsc_verify_tsc_adjust();
> >  	local_touch_nmi();
> >  	enter_idle();
> >  }
> 
> Doing a RDMSR on the idle path isn't going to be popular. That path is
> already way too slow.

Of course we can ratelimit that MSR read with jiffies, but do you have any
better suggestion aside of doing it timer based?

Thanks,

	tglx

[toc] | [prev] | [next] | [standalone]

#1526576 — Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle

On Mon, Nov 21, 2016 at 09:16:44AM +0100, Thomas Gleixner wrote:
> On Sun, 20 Nov 2016, Peter Zijlstra wrote:
> > On Sat, Nov 19, 2016 at 01:47:37PM -0000, Thomas Gleixner wrote:
> > > When entering idle, it's a good oportunity to verify that the TSC_ADJUST
> > > MSR has not been tampered with (BIOS hiding SMM cycles). If tampering is
> > > detected, emit a warning and restore it to the previous value.
> > 
> > > +++ b/arch/x86/kernel/process.c
> > > @@ -277,6 +277,7 @@ void exit_idle(void)
> > >  
> > >  void arch_cpu_idle_enter(void)
> > >  {
> > > +	tsc_verify_tsc_adjust();
> > >  	local_touch_nmi();
> > >  	enter_idle();
> > >  }
> > 
> > Doing a RDMSR on the idle path isn't going to be popular. That path is
> > already way too slow.
> 
> Of course we can ratelimit that MSR read with jiffies, but do you have any
> better suggestion aside of doing it timer based?

Not really :/ 

[toc] | [prev] | [next] | [standalone]

#1527112 — Re: [patch 4/8] x86/tsc: Verify TSC_ADJUST from idle

Thomas Gleixner <tglx@linutronix.de> writes:

> When entering idle, it's a good oportunity to verify that the TSC_ADJUST
> MSR has not been tampered with (BIOS hiding SMM cycles). If tampering is
> detected, emit a warning and restore it to the previous value.

idle entry is a time critical code path too, because idle periods are
often very short.  rdmsr is a us+ at least. You'll likely make
workloads that do a lot of short sleeps noticeable slower.

If anything load limit it aggressively.

BTW I don't think this situation of SMM tampering with TSC is common
anyways, so it may be simply not worth checking, or perhaps
only with a debug boot option.

-Andi

[toc] | [prev] | [next] | [standalone]

#1525996 — [patch 5/8] x86/tsc: Sync test only for the first cpu in a package

If the TSC_ADJUST MSR is available all CPUs in a package are forced to the
same value. So TSCs cannot be out of sync when the first CPU in the package
was in sync.

That allows to skip the sync test for all CPUs except the first starting
CPU in a package.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/tsc.h |    4 ++--
 arch/x86/kernel/tsc_sync.c |   37 ++++++++++++++++++++++++++++---------
 2 files changed, 30 insertions(+), 11 deletions(-)

--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -49,10 +49,10 @@ extern void check_tsc_sync_source(int cp
 extern void check_tsc_sync_target(void);
 
 #ifdef CONFIG_X86_TSC
-extern void tsc_store_and_check_tsc_adjust(void);
+extern bool tsc_store_and_check_tsc_adjust(void);
 extern void tsc_verify_tsc_adjust(void);
 #else
-static inline void tsc_store_and_check_tsc_adjust(void) { }
+static inline bool tsc_store_and_check_tsc_adjust(void) { }
 static inline void tsc_verify_tsc_adjust(void) { }
 #endif
 
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -47,18 +47,19 @@ void tsc_verify_tsc_adjust(void)
 }
 
 #ifndef CONFIG_SMP
-void __init tsc_store_and_check_tsc_adjust(void)
+bool __init tsc_store_and_check_tsc_adjust(void)
 {
 	struct tsc_adjust *ref, *cur = this_cpu_ptr(&tsc_adjust);
 	s64 bootval;
 
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
-		return;
+		return false;
 
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
 	cur->adjusted = bootval;
 	pr_info("TSC ADJUST: Boot CPU%u: %lld\n", cpu,  bootval);
+	return false;
 }
 
 #else /* !CONFIG_SMP */
@@ -66,14 +67,14 @@ void __init tsc_store_and_check_tsc_adju
 /*
  * Store and check the TSC ADJUST MSR if available
  */
-void tsc_store_and_check_tsc_adjust(void)
+bool tsc_store_and_check_tsc_adjust(void)
 {
 	struct tsc_adjust *ref, *cur = this_cpu_ptr(&tsc_adjust);
 	unsigned int refcpu, cpu = smp_processor_id();
 	s64 bootval;
 
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
-		return;
+		return false;
 
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
@@ -95,7 +96,7 @@ void tsc_store_and_check_tsc_adjust(void
 		 */
 		cur->adjusted = bootval;
 		pr_info_once("TSC ADJUST: Boot CPU%u: %lld\n", cpu,  bootval);
-		return;
+		return false;
 	}
 
 	ref = per_cpu_ptr(&tsc_adjust, refcpu);
@@ -119,6 +120,11 @@ void tsc_store_and_check_tsc_adjust(void
 		cur->adjusted = ref->adjusted;
 		wrmsrl(MSR_IA32_TSC_ADJUST, ref->adjusted);
 	}
+	/*
+	 * We have the TSCs forced to be in sync on this package. Skip sync
+	 * test:
+	 */
+	return true;
 }
 
 /*
@@ -127,6 +133,7 @@ void tsc_store_and_check_tsc_adjust(void
  */
 static atomic_t start_count;
 static atomic_t stop_count;
+static atomic_t skip_test;
 
 /*
  * We use a raw spinlock in this exceptional case, because
@@ -250,10 +257,16 @@ void check_tsc_sync_source(int cpu)
 	atomic_set(&stop_count, 0);
 
 	/*
-	 * Wait for the target to arrive:
+	 * Wait for the target to start or to skip the test:
 	 */
-	while (atomic_read(&start_count) != cpus-1)
+	while (atomic_read(&start_count) != cpus - 1) {
+		if (atomic_read(&skip_test) > 0) {
+			atomic_set(&skip_test, 0);
+			return;
+		}
 		cpu_relax();
+	}
+
 	/*
 	 * Trigger the target to continue into the measurement too:
 	 */
@@ -303,8 +316,14 @@ void check_tsc_sync_target(void)
 	if (unsynchronized_tsc() || tsc_clocksource_reliable)
 		return;
 
-	/* Store and check the TSC ADJUST MSR */
-	tsc_store_and_check_tsc_adjust();
+	/*
+	 * Store, verify and sanitize the TSC adjust register. If
+	 * successful skip the test.
+	 */
+	if (tsc_store_and_check_tsc_adjust()) {
+		atomic_inc(&skip_test);
+		return;
+	}
 
 	/*
 	 * Register this CPU's participation and wait for the

[toc] | [prev] | [next] | [standalone]

#1525997 — [patch 3/8] x86/tsc: Store and check TSC ADJUST MSR

The TSC_ADJUST MSR shows whether the TSC has been modified. This is helpful
in a two aspects:

1) It allows to detect BIOS wreckage, where SMM code tries to 'hide' the
   cycles spent by storing the TSC value at SMM entry and restoring it at
   SMM exit. On affected machines the TSCs run slowly out of sync up to the
   point where the clocksource watchdog (if available) detects it.

   The TSC_ADJUST MSR allows to detect the TSC modification before that and
   eventually restore it. This is also important for SoCs which have no
   watchdog clocksource and therefore TSC wreckage cannot be detected and
   acted upon.

2) All threads in a package are required to have the same TSC_ADJUST
   value. Broken BIOSes break that and as a result the TSC synchronization
   check fails.

   The TSC_ADJUST MSR allows to detect the deviation when a CPU comes
   online. If detected set it to the value of an already online CPU in the
   same package. This also allows to reduce the number of sync tests
   because with that in place the test is only required for the first CPU
   in a package.

   In principle all CPUs in a system should have the same TSC_ADJUST value
   even across packages, but with physical CPU hotplug this assumption is
   not true because the TSC starts with power on, so physical hotplug has
   to do some trickery to bring the TSC into sync with already running
   packages, which requires to use an TSC_ADJUST value different from CPUs
   which got powered earlier.

   A final enhancement is the opportunity to compensate for unsynced TSCs
   accross nodes at boot time and make the TSC usable that way. It won't
   help for TSCs which run apart due to frequency skew between packages,
   but this gets detected by the clocksource watchdog later.

The first step toward this is to store the TSC_ADJUST value of a starting
CPU and compare it with the value of an already online CPU in the same
package. If they differ, emit a warning and adjust it to the reference
value. The !SMP version just stores the boot value for later verification.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/tsc.h |    6 +++
 arch/x86/kernel/Makefile   |    2 -
 arch/x86/kernel/tsc.c      |    2 +
 arch/x86/kernel/tsc_sync.c |   88 +++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 97 insertions(+), 1 deletion(-)

--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -48,6 +48,12 @@ extern int tsc_clocksource_reliable;
 extern void check_tsc_sync_source(int cpu);
 extern void check_tsc_sync_target(void);
 
+#ifdef CONFIG_X86_TSC
+extern void tsc_store_and_check_tsc_adjust(void);
+#else
+static inline void tsc_store_and_check_tsc_adjust(void) { }
+#endif
+
 extern int notsc_setup(char *);
 extern void tsc_save_sched_clock_state(void);
 extern void tsc_restore_sched_clock_state(void);
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -75,7 +75,7 @@ apm-y				:= apm_32.o
 obj-$(CONFIG_APM)		+= apm.o
 obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_SMP)		+= smpboot.o
-obj-$(CONFIG_SMP)		+= tsc_sync.o
+obj-$(CONFIG_X86_TSC)		+= tsc_sync.o
 obj-$(CONFIG_SMP)		+= setup_percpu.o
 obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
 obj-y				+= apic/
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1365,6 +1365,8 @@ void __init tsc_init(void)
 
 	if (unsynchronized_tsc())
 		mark_tsc_unstable("TSCs unsynchronized");
+	else
+		tsc_store_and_check_tsc_adjust();
 
 	check_system_tsc_reliable();
 
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -14,12 +14,95 @@
  * ( The serial nature of the boot logic and the CPU hotplug lock
  *   protects against more than 2 CPUs entering this code. )
  */
+#include <linux/topology.h>
 #include <linux/spinlock.h>
 #include <linux/kernel.h>
 #include <linux/smp.h>
 #include <linux/nmi.h>
 #include <asm/tsc.h>
 
+struct tsc_adjust {
+	s64	bootval;
+	s64	adjusted;
+};
+
+static DEFINE_PER_CPU(struct tsc_adjust, tsc_adjust);
+
+#ifndef CONFIG_SMP
+void __init tsc_store_and_check_tsc_adjust(void)
+{
+	struct tsc_adjust *ref, *cur = this_cpu_ptr(&tsc_adjust);
+	s64 bootval;
+
+	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
+		return;
+
+	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
+	cur->bootval = bootval;
+	cur->adjusted = bootval;
+	pr_info("TSC ADJUST: Boot CPU%u: %lld\n",cpu,  bootval);
+}
+
+#else /* !CONFIG_SMP */
+
+/*
+ * Store and check the TSC ADJUST MSR if available
+ */
+void tsc_store_and_check_tsc_adjust(void)
+{
+	struct tsc_adjust *ref, *cur = this_cpu_ptr(&tsc_adjust);
+	unsigned int refcpu, cpu = smp_processor_id();
+	s64 bootval;
+
+	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
+		return;
+
+	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
+	cur->bootval = bootval;
+
+	/*
+	 * Check whether this CPU is the first in a package to come up. In
+	 * this case do not check the boot value against another package
+	 * because the package might have been physically hotplugged, where
+	 * TSC_ADJUST is expected to be different.
+	 */
+	refcpu = cpumask_any_but(topology_core_cpumask(cpu), cpu);
+
+	if (refcpu >= nr_cpu_ids) {
+		/*
+		 * First online CPU in a package stores the boot value in
+		 * the adjustment value. This value might change later via
+		 * the sync mechanism. If that fails we still can yell
+		 * about boot values not being consistent.
+		 */
+		cur->adjusted = bootval;
+		pr_info_once("TSC ADJUST: Boot CPU%u: %lld\n", cpu,  bootval);
+		return;
+	}
+
+	ref = per_cpu_ptr(&tsc_adjust, refcpu);
+	/*
+	 * Compare the boot value and complain if it differs in the
+	 * package.
+	 */
+	if (bootval != ref->bootval) {
+		pr_warn("TSC ADJUST differs: Reference CPU%u: %lld CPU%u: %lld\n",
+			refcpu, ref->bootval, cpu, bootval);
+	}
+	/*
+	 * The TSC_ADJUST values in a package must be the same. If the boot
+	 * value on this newly upcoming CPU differs from the adjustment
+	 * value of the already online CPU in this package, set it to that
+	 * adjusted value.
+	 */
+	if (bootval != ref->adjusted) {
+		pr_warn("TSC ADJUST synchronize: Reference CPU%u: %lld CPU%u: %lld\n",
+			refcpu, ref->adjusted, cpu, bootval);
+		cur->adjusted = ref->adjusted;
+		wrmsrl(MSR_IA32_TSC_ADJUST, ref->adjusted);
+	}
+}
+
 /*
  * Entry/exit counters that make sure that both CPUs
  * run the measurement code at once:
@@ -202,6 +285,9 @@ void check_tsc_sync_target(void)
 	if (unsynchronized_tsc() || tsc_clocksource_reliable)
 		return;
 
+	/* Store and check the TSC ADJUST MSR */
+	tsc_store_and_check_tsc_adjust();
+
 	/*
 	 * Register this CPU's participation and wait for the
 	 * source CPU to start the measurement:
@@ -223,3 +309,5 @@ void check_tsc_sync_target(void)
 	while (atomic_read(&stop_count) != cpus)
 		cpu_relax();
 }
+
+#endif /* CONFIG_SMP */

[toc] | [prev] | [next] | [standalone]

#1525998 — [patch 1/8] x86/tsc: Use X86_FEATURE_TSC_ADJUST in detect_art()

The art detection uses rdmsrl_safe() to detect the availablity of the
TSC_ADJUST MSR.

That's pointless because we have a feature bit for this. Use it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/tsc.c |   14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1043,18 +1043,20 @@ static void detect_art(void)
 	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
 		return;
 
-	cpuid(ART_CPUID_LEAF, &art_to_tsc_denominator,
-	      &art_to_tsc_numerator, unused, unused+1);
-
-	/* Don't enable ART in a VM, non-stop TSC required */
+	/* Don't enable ART in a VM, non-stop TSC and TSC_ADJUST required */
 	if (boot_cpu_has(X86_FEATURE_HYPERVISOR) ||
 	    !boot_cpu_has(X86_FEATURE_NONSTOP_TSC) ||
-	    art_to_tsc_denominator < ART_MIN_DENOMINATOR)
+	    !boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return;
 
-	if (rdmsrl_safe(MSR_IA32_TSC_ADJUST, &art_to_tsc_offset))
+	cpuid(ART_CPUID_LEAF, &art_to_tsc_denominator,
+	      &art_to_tsc_numerator, unused, unused+1);
+
+	if (art_to_tsc_denominator < ART_MIN_DENOMINATOR)
 		return;
 
+	rdmsrl(MSR_IA32_TSC_ADJUST, art_to_tsc_offset);
+
 	/* Make this sticky over multiple CPU init calls */
 	setup_force_cpu_cap(X86_FEATURE_ART);
 }

[toc] | [prev] | [next] | [standalone]

#1526000 — [patch 2/8] x86/tsc: Detect random warps

If time warps can be observed then they should only ever be observed on one
CPU. If they are observed on both CPUs then the system is completely hosed.

Add a check for this condition and notify if it happens.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/tsc_sync.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -37,6 +37,7 @@ static arch_spinlock_t sync_lock = __ARC
 static cycles_t last_tsc;
 static cycles_t max_warp;
 static int nr_warps;
+static int random_warps;
 
 /*
  * TSC-warp measurement loop running on both CPUs.  This is not called
@@ -45,7 +46,7 @@ static int nr_warps;
 static void check_tsc_warp(unsigned int timeout)
 {
 	cycles_t start, now, prev, end;
-	int i;
+	int i, cur_warps = 0;
 
 	start = rdtsc_ordered();
 	/*
@@ -85,7 +86,14 @@ static void check_tsc_warp(unsigned int
 		if (unlikely(prev > now)) {
 			arch_spin_lock(&sync_lock);
 			max_warp = max(max_warp, prev - now);
+			/*
+			 * Check whether this bounces back and forth. Only
+			 * one CPU should observe time going backwards.
+			 */
+			if (cur_warps != nr_warps)
+				random_warps++;
 			nr_warps++;
+			cur_warps = nr_warps;
 			arch_spin_unlock(&sync_lock);
 		}
 	}
@@ -160,6 +168,8 @@ void check_tsc_sync_source(int cpu)
 			smp_processor_id(), cpu);
 		pr_warning("Measured %Ld cycles TSC warp between CPUs, "
 			   "turning off TSC clock.\n", max_warp);
+		if (random_warps)
+			pr_warning("TSC warped randomly between CPUs\n");
 		mark_tsc_unstable("check_tsc_sync_source failed");
 	} else {
 		pr_debug("TSC synchronization [CPU#%d -> CPU#%d]: passed\n",
@@ -170,6 +180,7 @@ void check_tsc_sync_source(int cpu)
 	 * Reset it - just in case we boot another CPU later:
 	 */
 	atomic_set(&start_count, 0);
+	random_warps = 0;
 	nr_warps = 0;
 	max_warp = 0;
 	last_tsc = 0;

[toc] | [prev] | [next] | [standalone]

#1526001 — [patch 7/8] x86/tsc: Prepare warp test for TSC adjustment

To allow TSC compensation cross nodes its necessary to know in which
direction the TSC warp was observed. Return the maximum observed value on
the calling CPU so the caller can determine the direction later.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/tsc_sync.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -151,9 +151,9 @@ static int random_warps;
  * TSC-warp measurement loop running on both CPUs.  This is not called
  * if there is no TSC.
  */
-static void check_tsc_warp(unsigned int timeout)
+static cycles_t check_tsc_warp(unsigned int timeout)
 {
-	cycles_t start, now, prev, end;
+	cycles_t start, now, prev, end, cur_max_warp = 0;
 	int i, cur_warps = 0;
 
 	start = rdtsc_ordered();
@@ -194,6 +194,7 @@ static void check_tsc_warp(unsigned int
 		if (unlikely(prev > now)) {
 			arch_spin_lock(&sync_lock);
 			max_warp = max(max_warp, prev - now);
+			cur_max_warp = max_warp;
 			/*
 			 * Check whether this bounces back and forth. Only
 			 * one CPU should observe time going backwards.
@@ -208,6 +209,7 @@ static void check_tsc_warp(unsigned int
 	WARN(!(now-start),
 		"Warning: zero tsc calibration delta: %Ld [max: %Ld]\n",
 			now-start, end-start);
+	return cur_max_warp;
 }
 
 /*

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web