Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1477856 > unrolled thread

[PATCH v2 0/2] if running under KVM, use kvmclock to compute TSC deadline value

Started byPaolo Bonzini <pbonzini@redhat.com>
First post2016-09-07 00:40 +0200
Last post2016-09-07 08:40 +0200
Articles 6 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 0/2] if running under KVM, use kvmclock to compute TSC deadline value Paolo Bonzini <pbonzini@redhat.com> - 2016-09-07 00:40 +0200
    [PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value Paolo Bonzini <pbonzini@redhat.com> - 2016-09-07 00:40 +0200
      Re: [PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value David Matlack <dmatlack@google.com> - 2016-09-09 00:20 +0200
    [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops Paolo Bonzini <pbonzini@redhat.com> - 2016-09-07 00:40 +0200
      Re: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to  pv_ops kbuild test robot <lkp@intel.com> - 2016-09-07 08:30 +0200
      Re: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to  pv_ops kbuild test robot <lkp@intel.com> - 2016-09-07 08:40 +0200

#1477856 — [PATCH v2 0/2] if running under KVM, use kvmclock to compute TSC deadline value

FromPaolo Bonzini <pbonzini@redhat.com>
Date2016-09-07 00:40 +0200
Subject[PATCH v2 0/2] if running under KVM, use kvmclock to compute TSC deadline value
Message-ID<sexc5-8bZ-5@gated-at.bofh.it>
v1 posted here: https://patchwork.kernel.org/patch/9214993/

The motivation for this patch is in patch 2 (or you can read it from v1).

This version is larger but the hooks into apic.c are cleaner than in
v1.  Instead of arranging for kvmclock to replace only a small part of
setup_apic_timer, it registers its own clockevent.  The downside is that
kvmclock now needs to hook into the LAPIC timer interrupt to invoke the
event_handler of the new clockevent, but this is pretty straightforward
with a new pvop (assuming the introduction of new pvops is straightforward
at all).

Thanks,

Paolo

Paolo Bonzini (2):
  x86: paravirt: add local_apic_timer_interrupt to pv_ops
  x86, kvm: use kvmclock to compute TSC deadline value

 arch/x86/include/asm/apic.h           |   2 +
 arch/x86/include/asm/paravirt.h       |   5 ++
 arch/x86/include/asm/paravirt_types.h |   1 +
 arch/x86/kernel/apic/apic.c           |   4 +-
 arch/x86/kernel/kvmclock.c            | 156 ++++++++++++++++++++++++++++++++++
 arch/x86/kernel/paravirt.c            |   1 +
 6 files changed, 167 insertions(+), 2 deletions(-)

-- 
1.8.3.1

[toc] | [next] | [standalone]

#1477857 — [PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value

FromPaolo Bonzini <pbonzini@redhat.com>
Date2016-09-07 00:40 +0200
Subject[PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value
Message-ID<sexc5-8bZ-7@gated-at.bofh.it>
In reply to#1477856
Bad things happen if a guest using the TSC deadline timer is migrated.
The guest doesn't re-calibrate the TSC after migration, and the
TSC frequency can and will change unless your processor supports TSC
scaling (on Intel this is only Skylake) or your data center is perfectly
homogeneous.

The solution in this patch is to skip tsc_khz, and instead derive the
frequency from kvmclock's (mult, shift) pair.  Because kvmclock
parameters convert from tsc to nanoseconds, this needs a division
but that's the least of our problems when the TSC_DEADLINE_MSR write
costs 2000 clock cycles.  Luckily tsc_khz is really used by very little
outside the tsc clocksource (which kvmclock replaces) and the TSC
deadline timer.  Because KVM's local APIC doesn't need quirks, we
provide a paravirt clockevent that still uses the deadline timer
under the hood (as suggested by Andy Lutomirski).

This patch does not handle the very first deadline, hoping that it
is masked by the migration downtime (i.e. that the timer fires late
anyway).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/include/asm/apic.h |   1 +
 arch/x86/kernel/apic/apic.c |   2 +-
 arch/x86/kernel/kvmclock.c  | 156 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 158 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index f6e0bad1cde2..c88b0dcfdf3a 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -53,6 +53,7 @@ extern unsigned int apic_verbosity;
 extern int local_apic_timer_c2_ok;
 
 extern int disable_apic;
+extern int disable_apic_timer;
 extern unsigned int lapic_timer_frequency;
 
 #ifdef CONFIG_SMP
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 5b63bec7d0af..d0c6d1e3d627 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -169,7 +169,7 @@ __setup("apicpmtimer", setup_apicpmtimer);
 unsigned long mp_lapic_addr;
 int disable_apic;
 /* Disable local APIC timer from the kernel commandline or via dmi quirk */
-static int disable_apic_timer __initdata;
+int disable_apic_timer __initdata;
 /* Local APIC timer works in C2 */
 int local_apic_timer_c2_ok;
 EXPORT_SYMBOL_GPL(local_apic_timer_c2_ok);
diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 1d39bfbd26bb..365fa6494dd3 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -17,6 +17,7 @@
 */
 
 #include <linux/clocksource.h>
+#include <linux/clockchips.h>
 #include <linux/kvm_para.h>
 #include <asm/pvclock.h>
 #include <asm/msr.h>
@@ -245,6 +246,155 @@ static void kvm_shutdown(void)
 	native_machine_shutdown();
 }
 
+#ifdef CONFIG_X86_LOCAL_APIC
+/*
+ * kvmclock-based clock event implementation, used only together with the
+ * TSC deadline timer.  A subset of the normal LAPIC clockevent, but it
+ * uses kvmclock to convert nanoseconds to TSC.  This is necessary to
+ * handle changes to the TSC frequency, e.g. from live migration.
+ */
+
+static void kvmclock_lapic_timer_setup(unsigned lvtt_value)
+{
+	lvtt_value |= LOCAL_TIMER_VECTOR | APIC_LVT_TIMER_TSCDEADLINE;
+	apic_write(APIC_LVTT, lvtt_value);
+}
+
+static int kvmclock_lapic_timer_set_oneshot(struct clock_event_device *evt)
+{
+	kvmclock_lapic_timer_setup(0);
+	printk_once(KERN_DEBUG "kvmclock: TSC deadline timer enabled\n");
+
+	/*
+	 * See Intel SDM: TSC-Deadline Mode chapter. In xAPIC mode,
+	 * writing to the APIC LVTT and TSC_DEADLINE MSR isn't serialized.
+	 * According to Intel, MFENCE can do the serialization here.
+	 */
+	asm volatile("mfence" : : : "memory");
+	return 0;
+}
+
+static int kvmclock_lapic_timer_stop(struct clock_event_device *evt)
+{
+	kvmclock_lapic_timer_setup(APIC_LVT_MASKED);
+	wrmsrl(MSR_IA32_TSC_DEADLINE, -1);
+	return 0;
+}
+
+/*
+ * We already have the inverse of the (mult,shift) pair, though this means
+ * we need a division.  To avoid it we could compute a multiplicative inverse
+ * every time src->version changes.
+ */
+#define KVMCLOCK_TSC_DEADLINE_MAX_BITS	38
+#define KVMCLOCK_TSC_DEADLINE_MAX	((1ull << KVMCLOCK_TSC_DEADLINE_MAX_BITS) - 1)
+
+static int kvmclock_lapic_next_ktime(ktime_t expires,
+				     struct clock_event_device *evt)
+{
+	u64 ns, tsc;
+	u32 version;
+	int cpu;
+	struct pvclock_vcpu_time_info *src;
+
+	cpu = smp_processor_id();
+	src = &hv_clock[cpu].pvti;
+	ns = ktime_to_ns(expires);
+
+	do {
+		u64 delta_ns;
+		int shift;
+
+		version = pvclock_read_begin(src);
+		if (unlikely(ns < src->system_time)) {
+			tsc = src->tsc_timestamp;
+			virt_rmb();
+			continue;
+		}
+
+		delta_ns = ns - src->system_time;
+
+		/* Cap the wait to avoid overflow.  */
+		if (unlikely(delta_ns > KVMCLOCK_TSC_DEADLINE_MAX))
+			delta_ns = KVMCLOCK_TSC_DEADLINE_MAX;
+
+		/*
+		 * delta_tsc = delta_ns << (32-tsc_shift) / tsc_to_system_mul.
+		 * The shift is split in two steps so that a 38 bits (275 s)
+		 * deadline fits into the 64-bit dividend.
+		 */
+		shift = 32 - src->tsc_shift;
+		
+		/* First shift step... */
+		delta_ns <<= 64 - KVMCLOCK_TSC_DEADLINE_MAX_BITS;
+		shift -= 64 - KVMCLOCK_TSC_DEADLINE_MAX_BITS;
+
+		/* ... division... */
+		tsc = div_u64(delta_ns, src->tsc_to_system_mul);
+
+		/* ... and second shift step for the remaining bits.  */
+		if (shift >= 0)
+			tsc <<= shift;
+		else
+			tsc >>= -shift;
+
+		tsc += src->tsc_timestamp;
+	} while (pvclock_read_retry(src, version));
+
+	wrmsrl(MSR_IA32_TSC_DEADLINE, tsc);
+	return 0;
+}
+
+/*
+ * The local apic timer can be used for any function which is CPU local.
+ */
+static struct clock_event_device kvm_clockevent = {
+	.name			= "lapic",
+	/* Under KVM the LAPIC timer always runs in deep C-states.  */
+	.features		= CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_KTIME,
+	.set_state_shutdown	= kvmclock_lapic_timer_stop,
+	.set_state_oneshot	= kvmclock_lapic_timer_set_oneshot,
+	.set_next_ktime		= kvmclock_lapic_next_ktime,
+	.mult			= 1,
+	/* Make LAPIC timer preferrable over percpu HPET */
+	.rating			= 150,
+	.irq			= -1,
+};
+static DEFINE_PER_CPU(struct clock_event_device, kvm_events);
+
+static void kvmclock_local_apic_timer_interrupt(void)
+{
+	int cpu = smp_processor_id();
+	struct clock_event_device *evt = &per_cpu(kvm_events, cpu);
+
+	/*
+	 * Defer to the native clockevent if ours hasn't been setup yet.
+	 */
+	if (!evt->event_handler) {
+		native_local_apic_timer_interrupt();
+		return;
+	}
+
+	inc_irq_stat(apic_timer_irqs);
+	evt->event_handler(evt);
+}
+
+/*
+ * Setup the local APIC timer for this CPU. Copy the initialized values
+ * of the boot CPU and register the clock event in the framework.
+ */
+static void setup_kvmclock_timer(void)
+{
+	struct clock_event_device *evt = this_cpu_ptr(&kvm_events);
+
+	kvmclock_lapic_timer_stop(evt);
+
+	memcpy(evt, &kvm_clockevent, sizeof(*evt));
+	evt->cpumask = cpumask_of(smp_processor_id());
+	clockevents_register_device(evt);
+}
+#endif
+
 void __init kvmclock_init(void)
 {
 	struct pvclock_vcpu_time_info *vcpu_time;
@@ -292,6 +442,12 @@ void __init kvmclock_init(void)
 	x86_platform.get_wallclock = kvm_get_wallclock;
 	x86_platform.set_wallclock = kvm_set_wallclock;
 #ifdef CONFIG_X86_LOCAL_APIC
+	if (boot_cpu_has(X86_FEATURE_TSC_DEADLINE_TIMER) &&
+	    !disable_apic && !disable_apic_timer) {
+		pv_time_ops.local_apic_timer_interrupt = kvmclock_local_apic_timer_interrupt;
+		x86_init.timers.setup_percpu_clockev = setup_kvmclock_timer;
+		x86_cpuinit.setup_percpu_clockev = setup_kvmclock_timer;
+	}
 	x86_cpuinit.early_percpu_clock_init =
 		kvm_setup_secondary_clock;
 #endif
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1479548 — Re: [PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value

FromDavid Matlack <dmatlack@google.com>
Date2016-09-09 00:20 +0200
SubjectRe: [PATCH 2/2] x86, kvm: use kvmclock to compute TSC deadline value
Message-ID<sffPQ-3bV-17@gated-at.bofh.it>
In reply to#1477857
Hi Paolo,

On Tue, Sep 6, 2016 at 3:29 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Bad things happen if a guest using the TSC deadline timer is migrated.
> The guest doesn't re-calibrate the TSC after migration, and the
> TSC frequency can and will change unless your processor supports TSC
> scaling (on Intel this is only Skylake) or your data center is perfectly
> homogeneous.

Sorry, I forgot to follow up on our discussion in v1. One thing we
discussed there was using the APIC Timer to workaround a changing TSC
rate. You pointed out KVM's TSC deadline timer got a nice performance
boost recently, which makes it preferable. Does it makes sense to
apply the same optimization (using the VMX preemption timer) to the
APIC Timer, instead of creating a new dependency on kvmclock?

>
> The solution in this patch is to skip tsc_khz, and instead derive the
> frequency from kvmclock's (mult, shift) pair.  Because kvmclock
> parameters convert from tsc to nanoseconds, this needs a division
> but that's the least of our problems when the TSC_DEADLINE_MSR write
> costs 2000 clock cycles.  Luckily tsc_khz is really used by very little
> outside the tsc clocksource (which kvmclock replaces) and the TSC
> deadline timer.  Because KVM's local APIC doesn't need quirks, we
> provide a paravirt clockevent that still uses the deadline timer
> under the hood (as suggested by Andy Lutomirski).
>
> This patch does not handle the very first deadline, hoping that it
> is masked by the migration downtime (i.e. that the timer fires late
> anyway).
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  arch/x86/include/asm/apic.h |   1 +
>  arch/x86/kernel/apic/apic.c |   2 +-
>  arch/x86/kernel/kvmclock.c  | 156 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 158 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
> index f6e0bad1cde2..c88b0dcfdf3a 100644
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -53,6 +53,7 @@ extern unsigned int apic_verbosity;
>  extern int local_apic_timer_c2_ok;
>
>  extern int disable_apic;
> +extern int disable_apic_timer;
>  extern unsigned int lapic_timer_frequency;
>
>  #ifdef CONFIG_SMP
> diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
> index 5b63bec7d0af..d0c6d1e3d627 100644
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -169,7 +169,7 @@ __setup("apicpmtimer", setup_apicpmtimer);
>  unsigned long mp_lapic_addr;
>  int disable_apic;
>  /* Disable local APIC timer from the kernel commandline or via dmi quirk */
> -static int disable_apic_timer __initdata;
> +int disable_apic_timer __initdata;
>  /* Local APIC timer works in C2 */
>  int local_apic_timer_c2_ok;
>  EXPORT_SYMBOL_GPL(local_apic_timer_c2_ok);
> diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
> index 1d39bfbd26bb..365fa6494dd3 100644
> --- a/arch/x86/kernel/kvmclock.c
> +++ b/arch/x86/kernel/kvmclock.c
> @@ -17,6 +17,7 @@
>  */
>
>  #include <linux/clocksource.h>
> +#include <linux/clockchips.h>
>  #include <linux/kvm_para.h>
>  #include <asm/pvclock.h>
>  #include <asm/msr.h>
> @@ -245,6 +246,155 @@ static void kvm_shutdown(void)
>         native_machine_shutdown();
>  }
>
> +#ifdef CONFIG_X86_LOCAL_APIC
> +/*
> + * kvmclock-based clock event implementation, used only together with the
> + * TSC deadline timer.  A subset of the normal LAPIC clockevent, but it
> + * uses kvmclock to convert nanoseconds to TSC.  This is necessary to
> + * handle changes to the TSC frequency, e.g. from live migration.
> + */
> +
> +static void kvmclock_lapic_timer_setup(unsigned lvtt_value)
> +{
> +       lvtt_value |= LOCAL_TIMER_VECTOR | APIC_LVT_TIMER_TSCDEADLINE;
> +       apic_write(APIC_LVTT, lvtt_value);
> +}
> +
> +static int kvmclock_lapic_timer_set_oneshot(struct clock_event_device *evt)
> +{
> +       kvmclock_lapic_timer_setup(0);
> +       printk_once(KERN_DEBUG "kvmclock: TSC deadline timer enabled\n");
> +
> +       /*
> +        * See Intel SDM: TSC-Deadline Mode chapter. In xAPIC mode,
> +        * writing to the APIC LVTT and TSC_DEADLINE MSR isn't serialized.
> +        * According to Intel, MFENCE can do the serialization here.
> +        */
> +       asm volatile("mfence" : : : "memory");
> +       return 0;
> +}
> +
> +static int kvmclock_lapic_timer_stop(struct clock_event_device *evt)
> +{
> +       kvmclock_lapic_timer_setup(APIC_LVT_MASKED);
> +       wrmsrl(MSR_IA32_TSC_DEADLINE, -1);
> +       return 0;
> +}
> +
> +/*
> + * We already have the inverse of the (mult,shift) pair, though this means
> + * we need a division.  To avoid it we could compute a multiplicative inverse
> + * every time src->version changes.
> + */
> +#define KVMCLOCK_TSC_DEADLINE_MAX_BITS 38
> +#define KVMCLOCK_TSC_DEADLINE_MAX      ((1ull << KVMCLOCK_TSC_DEADLINE_MAX_BITS) - 1)
> +
> +static int kvmclock_lapic_next_ktime(ktime_t expires,
> +                                    struct clock_event_device *evt)
> +{
> +       u64 ns, tsc;
> +       u32 version;
> +       int cpu;
> +       struct pvclock_vcpu_time_info *src;
> +
> +       cpu = smp_processor_id();
> +       src = &hv_clock[cpu].pvti;
> +       ns = ktime_to_ns(expires);
> +
> +       do {
> +               u64 delta_ns;
> +               int shift;
> +
> +               version = pvclock_read_begin(src);
> +               if (unlikely(ns < src->system_time)) {
> +                       tsc = src->tsc_timestamp;
> +                       virt_rmb();
> +                       continue;
> +               }
> +
> +               delta_ns = ns - src->system_time;
> +
> +               /* Cap the wait to avoid overflow.  */
> +               if (unlikely(delta_ns > KVMCLOCK_TSC_DEADLINE_MAX))
> +                       delta_ns = KVMCLOCK_TSC_DEADLINE_MAX;
> +
> +               /*
> +                * delta_tsc = delta_ns << (32-tsc_shift) / tsc_to_system_mul.
> +                * The shift is split in two steps so that a 38 bits (275 s)
> +                * deadline fits into the 64-bit dividend.
> +                */
> +               shift = 32 - src->tsc_shift;
> +
> +               /* First shift step... */
> +               delta_ns <<= 64 - KVMCLOCK_TSC_DEADLINE_MAX_BITS;
> +               shift -= 64 - KVMCLOCK_TSC_DEADLINE_MAX_BITS;
> +
> +               /* ... division... */
> +               tsc = div_u64(delta_ns, src->tsc_to_system_mul);
> +
> +               /* ... and second shift step for the remaining bits.  */
> +               if (shift >= 0)
> +                       tsc <<= shift;
> +               else
> +                       tsc >>= -shift;
> +
> +               tsc += src->tsc_timestamp;
> +       } while (pvclock_read_retry(src, version));
> +
> +       wrmsrl(MSR_IA32_TSC_DEADLINE, tsc);
> +       return 0;
> +}
> +
> +/*
> + * The local apic timer can be used for any function which is CPU local.
> + */
> +static struct clock_event_device kvm_clockevent = {
> +       .name                   = "lapic",

Should we encode "kvm" or "kvmclock" in the name? I'm not sure how
this name gets used, but it might be nice to distinguish it from the
native TSC deadline timer clock_event_device.

> +       /* Under KVM the LAPIC timer always runs in deep C-states.  */
> +       .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_KTIME,
> +       .set_state_shutdown     = kvmclock_lapic_timer_stop,
> +       .set_state_oneshot      = kvmclock_lapic_timer_set_oneshot,
> +       .set_next_ktime         = kvmclock_lapic_next_ktime,
> +       .mult                   = 1,
> +       /* Make LAPIC timer preferrable over percpu HPET */
> +       .rating                 = 150,
> +       .irq                    = -1,
> +};
> +static DEFINE_PER_CPU(struct clock_event_device, kvm_events);
> +
> +static void kvmclock_local_apic_timer_interrupt(void)
> +{
> +       int cpu = smp_processor_id();
> +       struct clock_event_device *evt = &per_cpu(kvm_events, cpu);
> +
> +       /*
> +        * Defer to the native clockevent if ours hasn't been setup yet.
> +        */
> +       if (!evt->event_handler) {
> +               native_local_apic_timer_interrupt();
> +               return;
> +       }
> +
> +       inc_irq_stat(apic_timer_irqs);
> +       evt->event_handler(evt);
> +}
> +
> +/*
> + * Setup the local APIC timer for this CPU. Copy the initialized values
> + * of the boot CPU and register the clock event in the framework.
> + */
> +static void setup_kvmclock_timer(void)
> +{
> +       struct clock_event_device *evt = this_cpu_ptr(&kvm_events);
> +
> +       kvmclock_lapic_timer_stop(evt);
> +
> +       memcpy(evt, &kvm_clockevent, sizeof(*evt));
> +       evt->cpumask = cpumask_of(smp_processor_id());
> +       clockevents_register_device(evt);
> +}
> +#endif
> +
>  void __init kvmclock_init(void)
>  {
>         struct pvclock_vcpu_time_info *vcpu_time;
> @@ -292,6 +442,12 @@ void __init kvmclock_init(void)
>         x86_platform.get_wallclock = kvm_get_wallclock;
>         x86_platform.set_wallclock = kvm_set_wallclock;
>  #ifdef CONFIG_X86_LOCAL_APIC
> +       if (boot_cpu_has(X86_FEATURE_TSC_DEADLINE_TIMER) &&
> +           !disable_apic && !disable_apic_timer) {

Request that this be a hypervisor-controllable feature. e.g. we could
add a new bit to KVM's CPUID leaf to indicate kvmclock is the
definitive source of TSC rate.

> +               pv_time_ops.local_apic_timer_interrupt = kvmclock_local_apic_timer_interrupt;
> +               x86_init.timers.setup_percpu_clockev = setup_kvmclock_timer;
> +               x86_cpuinit.setup_percpu_clockev = setup_kvmclock_timer;
> +       }
>         x86_cpuinit.early_percpu_clock_init =
>                 kvm_setup_secondary_clock;
>  #endif
> --
> 1.8.3.1
>

[toc] | [prev] | [next] | [standalone]

#1477858 — [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops

FromPaolo Bonzini <pbonzini@redhat.com>
Date2016-09-07 00:40 +0200
Subject[PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops
Message-ID<sexc5-8bZ-9@gated-at.bofh.it>
In reply to#1477856
local_apic_timer_interrupt refers to a static clockevent in
apic.c.  Paravirtualized implementations need to check a different
clockevent, so allow customizing the guts of the APIC timer interrupt.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/include/asm/apic.h           | 1 +
 arch/x86/include/asm/paravirt.h       | 5 +++++
 arch/x86/include/asm/paravirt_types.h | 1 +
 arch/x86/kernel/apic/apic.c           | 2 +-
 arch/x86/kernel/paravirt.c            | 1 +
 5 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 124357773ffa..f6e0bad1cde2 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -109,6 +109,7 @@ extern void native_apic_wait_icr_idle(void);
 extern u32 native_safe_apic_wait_icr_idle(void);
 extern void native_apic_icr_write(u32 low, u32 id);
 extern u64 native_apic_icr_read(void);
+extern void native_local_apic_timer_interrupt(void);
 
 static inline bool apic_is_x2apic_enabled(void)
 {
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 2970d22d7766..71ec3a73745f 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -197,6 +197,11 @@ static inline u64 paravirt_steal_clock(int cpu)
 	return PVOP_CALL1(u64, pv_time_ops.steal_clock, cpu);
 }
 
+static inline void local_apic_timer_interrupt(void)
+{
+	PVOP_VCALL0(pv_time_ops.local_apic_timer_interrupt);
+}
+
 static inline unsigned long long paravirt_read_pmc(int counter)
 {
 	return PVOP_CALL1(u64, pv_cpu_ops.read_pmc, counter);
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 7fa9e7740ba3..ea1f27400098 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -96,6 +96,7 @@ struct pv_lazy_ops {
 struct pv_time_ops {
 	unsigned long long (*sched_clock)(void);
 	unsigned long long (*steal_clock)(int cpu);
+	void (*local_apic_timer_interrupt)(void);
 };
 
 struct pv_cpu_ops {
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index cea4fc19e844..5b63bec7d0af 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -902,7 +902,7 @@ void setup_secondary_APIC_clock(void)
 /*
  * The guts of the apic timer interrupt
  */
-static void local_apic_timer_interrupt(void)
+void native_local_apic_timer_interrupt(void)
 {
 	int cpu = smp_processor_id();
 	struct clock_event_device *evt = &per_cpu(lapic_events, cpu);
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index ad5bc9578a73..0e056271d714 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -310,6 +310,7 @@ struct pv_init_ops pv_init_ops = {
 struct pv_time_ops pv_time_ops = {
 	.sched_clock = native_sched_clock,
 	.steal_clock = native_steal_clock,
+	.local_apic_timer_interrupt = native_local_apic_timer_interrupt,
 };
 
 __visible struct pv_irq_ops pv_irq_ops = {
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1477986 — Re: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops

Fromkbuild test robot <lkp@intel.com>
Date2016-09-07 08:30 +0200
SubjectRe: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops
Message-ID<seEwV-4wu-23@gated-at.bofh.it>
In reply to#1477858

[Multipart message — attachments visible in raw view] — view raw

Hi Paolo,

[auto build test ERROR on tip/x86/core]
[also build test ERROR on v4.8-rc5 next-20160906]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
[Suggest to use git(>=2.9.0) format-patch --base=<commit> (or --base=auto for convenience) to record what (public, well-known) commit your patch series was built on]
[Check https://git-scm.com/docs/git-format-patch for more information]

url:    https://github.com/0day-ci/linux/commits/Paolo-Bonzini/if-running-under-KVM-use-kvmclock-to-compute-TSC-deadline-value/20160907-131553
config: i386-defconfig (attached as .config)
compiler: gcc-6 (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

   arch/x86/kernel/apic/apic.c: In function 'smp_apic_timer_interrupt':
>> arch/x86/kernel/apic/apic.c:931:2: error: implicit declaration of function 'local_apic_timer_interrupt' [-Werror=implicit-function-declaration]
     local_apic_timer_interrupt();
     ^~~~~~~~~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors

vim +/local_apic_timer_interrupt +931 arch/x86/kernel/apic/apic.c

eddc0e92 arch/x86/kernel/apic/apic.c Seiji Aguchi    2013-06-20  925  	 *
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30  926  	 * update_process_times() expects us to have done irq_enter().
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30  927  	 * Besides, if we don't timer interrupts ignore the global
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30  928  	 * interrupt lock, which is the WrongThing (tm) to do.
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30  929  	 */
eddc0e92 arch/x86/kernel/apic/apic.c Seiji Aguchi    2013-06-20  930  	entering_ack_irq();
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30 @931  	local_apic_timer_interrupt();
eddc0e92 arch/x86/kernel/apic/apic.c Seiji Aguchi    2013-06-20  932  	exiting_irq();
274cfe59 arch/x86/kernel/apic_64.c   Cyrill Gorcunov 2008-08-16  933  
0e078e2f arch/x86/kernel/apic_64.c   Thomas Gleixner 2008-01-30  934  	set_irq_regs(old_regs);

:::::: The code at line 931 was first introduced by commit
:::::: 0e078e2f5060e06f9b3f32e55665ea835343440e x86: prepare merging arch/x86/kernel/apic_32/64.c

:::::: TO: Thomas Gleixner <tglx@linutronix.de>
:::::: CC: Ingo Molnar <mingo@elte.hu>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[toc] | [prev] | [next] | [standalone]

#1477988 — Re: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops

Fromkbuild test robot <lkp@intel.com>
Date2016-09-07 08:40 +0200
SubjectRe: [PATCH 1/2] x86: paravirt: add local_apic_timer_interrupt to pv_ops
Message-ID<seEGB-4zn-1@gated-at.bofh.it>
In reply to#1477858

[Multipart message — attachments visible in raw view] — view raw

Hi Paolo,

[auto build test ERROR on tip/x86/core]
[also build test ERROR on v4.8-rc5 next-20160906]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
[Suggest to use git(>=2.9.0) format-patch --base=<commit> (or --base=auto for convenience) to record what (public, well-known) commit your patch series was built on]
[Check https://git-scm.com/docs/git-format-patch for more information]

url:    https://github.com/0day-ci/linux/commits/Paolo-Bonzini/if-running-under-KVM-use-kvmclock-to-compute-TSC-deadline-value/20160907-131553
config: i386-randconfig-s1-201636 (attached as .config)
compiler: gcc-6 (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All errors (new ones prefixed by >>):

>> arch/x86/kernel/paravirt.c:313:32: error: 'native_local_apic_timer_interrupt' undeclared here (not in a function)
     .local_apic_timer_interrupt = native_local_apic_timer_interrupt,
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/native_local_apic_timer_interrupt +313 arch/x86/kernel/paravirt.c

   307		.patch = native_patch,
   308	};
   309	
   310	struct pv_time_ops pv_time_ops = {
   311		.sched_clock = native_sched_clock,
   312		.steal_clock = native_steal_clock,
 > 313		.local_apic_timer_interrupt = native_local_apic_timer_interrupt,
   314	};
   315	
   316	__visible struct pv_irq_ops pv_irq_ops = {

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web