Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1673502 > unrolled thread
| Started by | Wanpeng Li <kernellwp@gmail.com> |
|---|---|
| First post | 2017-06-23 14:10 +0200 |
| Last post | 2017-06-28 00:20 +0200 |
| Articles | 4 — 2 participants |
Back to article view | Back to linux.kernel
[PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Wanpeng Li <kernellwp@gmail.com> - 2017-06-23 14:10 +0200
Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Paolo Bonzini <pbonzini@redhat.com> - 2017-06-27 18:20 +0200
Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Wanpeng Li <kernellwp@gmail.com> - 2017-06-28 00:10 +0200
Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Wanpeng Li <kernellwp@gmail.com> - 2017-06-28 00:20 +0200
| From | Wanpeng Li <kernellwp@gmail.com> |
|---|---|
| Date | 2017-06-23 14:10 +0200 |
| Subject | [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay |
| Message-ID | <tVvzs-1Yc-23@gated-at.bofh.it> |
From: Wanpeng Li <wanpeng.li@hotmail.com>
Linux as a guest will prevent to program the next event to the clock event
device when the event is in the past. However, it is not guaranteed by all
the guests, the guest like kvm-unit-tests/apic.flat will write current tsc
value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated
by vmx preemption timer will program the absolute target tsc value to vmcs
preemption timer field w/ delta == 0, then plays a vmentry and an upcoming
vmx preemption timer fire vmexit dance, the lapic timer injection is delayed
for this duration. Actually the lapic timer which is emulated by hrtimer can
handle this correctly.
This patch fixes it by firing the lapic timer and injecting a timer interrupt
immediately during the next vmentry if guest programs an expired timer to
the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
test of apic.flat.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
arch/x86/kvm/lapic.c | 7 ++++++-
arch/x86/kvm/vmx.c | 3 +++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index d24c874..08cf73d 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1504,12 +1504,17 @@ static void cancel_hv_timer(struct kvm_lapic *apic)
static bool start_hv_timer(struct kvm_lapic *apic)
{
u64 tscdeadline = apic->lapic_timer.tscdeadline;
+ int ret = 0;
if ((atomic_read(&apic->lapic_timer.pending) &&
!apic_lvtt_period(apic)) ||
- kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline)) {
+ (ret = kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline))) {
if (apic->lapic_timer.hv_timer_in_use)
cancel_hv_timer(apic);
+ if (ret == 1) {
+ apic_timer_expired(apic);
+ return true;
+ }
} else {
apic->lapic_timer.hv_timer_in_use = true;
hrtimer_cancel(&apic->lapic_timer.timer);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 2e906cf..2008e9b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -11149,6 +11149,9 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc)
u64 guest_tscl = kvm_read_l1_tsc(vcpu, tscl);
u64 delta_tsc = max(guest_deadline_tsc, guest_tscl) - guest_tscl;
+ if (delta_tsc == 0)
+ return 1;
+
/* Convert to host delta tsc if tsc scaling is enabled */
if (vcpu->arch.tsc_scaling_ratio != kvm_default_tsc_scaling_ratio &&
u64_shl_div_u64(delta_tsc,
--
2.7.4
[toc] | [next] | [standalone]
| From | Paolo Bonzini <pbonzini@redhat.com> |
|---|---|
| Date | 2017-06-27 18:20 +0200 |
| Message-ID | <tX1nA-2Ys-25@gated-at.bofh.it> |
| In reply to | #1673502 |
On 23/06/2017 13:59, Wanpeng Li wrote: > From: Wanpeng Li <wanpeng.li@hotmail.com> > > Linux as a guest will prevent to program the next event to the clock event > device when the event is in the past. However, it is not guaranteed by all > the guests, the guest like kvm-unit-tests/apic.flat will write current tsc > value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated > by vmx preemption timer will program the absolute target tsc value to vmcs > preemption timer field w/ delta == 0, then plays a vmentry and an upcoming > vmx preemption timer fire vmexit dance, the lapic timer injection is delayed > for this duration. Actually the lapic timer which is emulated by hrtimer can > handle this correctly. > > This patch fixes it by firing the lapic timer and injecting a timer interrupt > immediately during the next vmentry if guest programs an expired timer to > the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer > test of apic.flat. If it doesn't matter for actual guests, I'd rather not complicate the logic. Paolo
[toc] | [prev] | [next] | [standalone]
| From | Wanpeng Li <kernellwp@gmail.com> |
|---|---|
| Date | 2017-06-28 00:10 +0200 |
| Message-ID | <tX6Qh-6Lo-5@gated-at.bofh.it> |
| In reply to | #1675960 |
2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>
>
> On 23/06/2017 13:59, Wanpeng Li wrote:
>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>
>> Linux as a guest will prevent to program the next event to the clock event
>> device when the event is in the past. However, it is not guaranteed by all
>> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc
>> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated
>> by vmx preemption timer will program the absolute target tsc value to vmcs
>> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming
>> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed
>> for this duration. Actually the lapic timer which is emulated by hrtimer can
>> handle this correctly.
>>
>> This patch fixes it by firing the lapic timer and injecting a timer interrupt
>> immediately during the next vmentry if guest programs an expired timer to
>> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
>> test of apic.flat.
>
> If it doesn't matter for actual guests, I'd rather not complicate the logic.
Sorry, maybe my description is not that accurate. It can also benefit
if the tsc deadline timer is programmed really close to the deadline.
That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in
TSC deadline calculation"). In addition, I capture there is frequently
programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0):
qemu-system-x86-2352 [003] .... 457.709435: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [004] .... 457.941090: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.381340: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.381565: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.382079: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.382277: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.382576: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.383065: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.565506: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.565718: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.566199: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.566431: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.566680: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [005] .... 458.567154: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [006] .... 458.989358: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [006] .... 458.989554: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [006] .... 458.990075: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.397456: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.397618: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.397877: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.398091: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.398307: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.398672: vmx_set_hv_timer: expired timer
qemu-system-x86-2352 [003] .... 459.502826: vmx_set_hv_timer: expired timer
Regards,
Wanpeng Li
[toc] | [prev] | [next] | [standalone]
| From | Wanpeng Li <kernellwp@gmail.com> |
|---|---|
| Date | 2017-06-28 00:20 +0200 |
| Message-ID | <tX6ZY-6Ou-21@gated-at.bofh.it> |
| In reply to | #1676206 |
2017-06-28 6:09 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>:
> 2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>>
>>
>> On 23/06/2017 13:59, Wanpeng Li wrote:
>>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>>
>>> Linux as a guest will prevent to program the next event to the clock event
>>> device when the event is in the past. However, it is not guaranteed by all
>>> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc
>>> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated
>>> by vmx preemption timer will program the absolute target tsc value to vmcs
>>> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming
>>> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed
>>> for this duration. Actually the lapic timer which is emulated by hrtimer can
>>> handle this correctly.
>>>
>>> This patch fixes it by firing the lapic timer and injecting a timer interrupt
>>> immediately during the next vmentry if guest programs an expired timer to
>>> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
>>> test of apic.flat.
>>
>> If it doesn't matter for actual guests, I'd rather not complicate the logic.
>
> Sorry, maybe my description is not that accurate. It can also benefit
> if the tsc deadline timer is programmed really close to the deadline.
> That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in
> TSC deadline calculation"). In addition, I capture there is frequently
> programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0):
I test a linux guest.
>
> qemu-system-x86-2352 [003] .... 457.709435: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [004] .... 457.941090: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.381340: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.381565: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.382079: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.382277: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.382576: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.383065: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.565506: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.565718: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.566199: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.566431: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.566680: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [005] .... 458.567154: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [006] .... 458.989358: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [006] .... 458.989554: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [006] .... 458.990075: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.397456: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.397618: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.397877: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.398091: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.398307: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.398672: vmx_set_hv_timer: expired timer
> qemu-system-x86-2352 [003] .... 459.502826: vmx_set_hv_timer: expired timer
>
> Regards,
> Wanpeng Li
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web