Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1705249 > unrolled thread

[PATCH 0/3] KVM: optimize the kvm_vcpu_on_spin

Started by"Longpeng(Mike)" <longpeng2@huawei.com>
First post2017-08-07 10:50 +0200
Last post2017-08-07 11:10 +0200
Articles 7 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/3] KVM: optimize the kvm_vcpu_on_spin "Longpeng(Mike)" <longpeng2@huawei.com> - 2017-08-07 10:50 +0200
    [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390 "Longpeng(Mike)" <longpeng2@huawei.com> - 2017-08-07 10:50 +0200
      Re: [PATCH 3/3] KVM: implement spinlock optimization logic for  arm/s390 David Hildenbrand <david@redhat.com> - 2017-08-07 11:00 +0200
        Re: [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390 "Longpeng (Mike)" <longpeng2@huawei.com> - 2017-08-07 11:00 +0200
    [PATCH 1/3] KVM: add spinlock-exiting optimize framework "Longpeng(Mike)" <longpeng2@huawei.com> - 2017-08-07 10:50 +0200
      Re: [PATCH 1/3] KVM: add spinlock-exiting optimize framework David Hildenbrand <david@redhat.com> - 2017-08-07 11:00 +0200
        Re: [PATCH 1/3] KVM: add spinlock-exiting optimize framework "Longpeng (Mike)" <longpeng2@huawei.com> - 2017-08-07 11:10 +0200

#1705249 — [PATCH 0/3] KVM: optimize the kvm_vcpu_on_spin

From"Longpeng(Mike)" <longpeng2@huawei.com>
Date2017-08-07 10:50 +0200
Subject[PATCH 0/3] KVM: optimize the kvm_vcpu_on_spin
Message-ID<ubLTz-5V4-11@gated-at.bofh.it>
This is a simple optimization for kvm_vcpu_on_spin, the
main idea is described in patch-1's commit msg.

I did some tests base on the RFC version, the result shows
that it can improves the performance slightly.

== Geekbench-3.4.1 ==
VM1: 	8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19)
	running Geekbench-3.4.1 *10 truns*
VM2/VM3/VM4: configure is the same as VM1
	stress each vcpu usage(seed by top in guest) to 40%

The comparison of each testcase's score:
(higher is better)
		before		after		improve
Inter
 single		1176.7		1179.0		0.2%
 multi		3459.5		3426.5		-0.9%
Float
 single		1150.5		1150.9		0.0%
 multi		3364.5		3391.9		0.8%
Memory(stream)
 single		1768.7		1773.1		0.2%
 multi		2511.6		2557.2		1.8%
Overall
 single		1284.2		1286.2		0.2%
 multi		3231.4		3238.4		0.2%


== kernbench-0.42 ==
VM1:    8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19)
        running "kernbench -n 10"
VM2/VM3/VM4: configure is the same as VM1
        stress each vcpu usage(seed by top in guest) to 40%

The comparison of 'Elapsed Time':
(sooner is better)
		before		after		improve
load -j4	12.762		12.751		0.1%
load -j32	9.743		8.955		8.1%
load -j		9.688		9.229		4.7%


Physical Machine:
  Architecture:          x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Byte Order:            Little Endian
  CPU(s):                24
  On-line CPU(s) list:   0-23
  Thread(s) per core:    2
  Core(s) per socket:    6
  Socket(s):             2
  NUMA node(s):          2
  Vendor ID:             GenuineIntel
  CPU family:            6
  Model:                 45
  Model name:            Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
  Stepping:              7
  CPU MHz:               2799.902
  BogoMIPS:              5004.67
  Virtualization:        VT-x
  L1d cache:             32K
  L1i cache:             32K
  L2 cache:              256K
  L3 cache:              15360K
  NUMA node0 CPU(s):     0-5,12-17
  NUMA node1 CPU(s):     6-11,18-23

---
Changes since RFC:
 - only cache result for X86. [David & Cornlia & Paolo]
 - add performance numbers. [David]
 - impls arm/s390. [Christoffer & David]
 - refactor the impls. [me]

---
Longpeng(Mike) (3):
  KVM: add spinlock-exiting optimize framework
  KVM: X86: implement the logic for spinlock optimization
  KVM: implement spinlock optimization logic for arm/s390

 arch/mips/kvm/mips.c            | 10 ++++++++++
 arch/powerpc/kvm/powerpc.c      | 10 ++++++++++
 arch/s390/kvm/kvm-s390.c        | 10 ++++++++++
 arch/x86/include/asm/kvm_host.h |  5 +++++
 arch/x86/kvm/svm.c              |  6 ++++++
 arch/x86/kvm/vmx.c              | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c              | 15 +++++++++++++++
 include/linux/kvm_host.h        |  2 ++
 virt/kvm/arm/arm.c              | 10 ++++++++++
 virt/kvm/kvm_main.c             |  4 ++++
 10 files changed, 92 insertions(+)

-- 
1.8.3.1

[toc] | [next] | [standalone]

#1705257 — [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390

From"Longpeng(Mike)" <longpeng2@huawei.com>
Date2017-08-07 10:50 +0200
Subject[PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390
Message-ID<ubLTB-5V4-35@gated-at.bofh.it>
In reply to#1705249
Implements the kvm_arch_vcpu_spin/preempt_in_kernel() for arm/s390,
they needn't cache the result.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
---
 arch/s390/kvm/kvm-s390.c | 4 ++--
 virt/kvm/arm/arm.c       | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index f78cdc2..49b9178 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -2449,12 +2449,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 
 bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
 {
-	return false;
+	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
 }
 
 bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
 {
-	return false;
+	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
 }
 
 void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index e45f780..956f025 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -418,12 +418,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 
 bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
 {
-	return false;
+	return vcpu_mode_priv(vcpu);
 }
 
 bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
 {
-	return false;
+	return vcpu_mode_priv(vcpu);
 }
 
 /* Just ensure a guest exit from a particular CPU */
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1705265 — Re: [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390

FromDavid Hildenbrand <david@redhat.com>
Date2017-08-07 11:00 +0200
SubjectRe: [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390
Message-ID<ubM3f-5YJ-7@gated-at.bofh.it>
In reply to#1705257
On 07.08.2017 10:44, Longpeng(Mike) wrote:
> Implements the kvm_arch_vcpu_spin/preempt_in_kernel() for arm/s390,
> they needn't cache the result.
> 
> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
> ---
>  arch/s390/kvm/kvm-s390.c | 4 ++--
>  virt/kvm/arm/arm.c       | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index f78cdc2..49b9178 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -2449,12 +2449,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>  
>  bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>  {
> -	return false;
> +	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
>  }
>  
>  bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>  {
> -	return false;
> +	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
>  }
>  
>  void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index e45f780..956f025 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -418,12 +418,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>  
>  bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>  {
> -	return false;
> +	return vcpu_mode_priv(vcpu);
>  }
>  
>  bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>  {
> -	return false;
> +	return vcpu_mode_priv(vcpu);
>  }
>  
>  /* Just ensure a guest exit from a particular CPU */
> 

Can you split that into two parts? (arm and s390x?)

-- 

Thanks,

David

[toc] | [prev] | [next] | [standalone]

#1705266 — Re: [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390

From"Longpeng (Mike)" <longpeng2@huawei.com>
Date2017-08-07 11:00 +0200
SubjectRe: [PATCH 3/3] KVM: implement spinlock optimization logic for arm/s390
Message-ID<ubM3f-5YJ-9@gated-at.bofh.it>
In reply to#1705265
On 2017/8/7 16:52, David Hildenbrand wrote:

> On 07.08.2017 10:44, Longpeng(Mike) wrote:
>> Implements the kvm_arch_vcpu_spin/preempt_in_kernel() for arm/s390,
>> they needn't cache the result.
>>
>> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
>> ---
>>  arch/s390/kvm/kvm-s390.c | 4 ++--
>>  virt/kvm/arm/arm.c       | 4 ++--
>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index f78cdc2..49b9178 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -2449,12 +2449,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>>  
>>  bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>>  {
>> -	return false;
>> +	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
>>  }
>>  
>>  bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>>  {
>> -	return false;
>> +	return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE);
>>  }
>>  
>>  void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index e45f780..956f025 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -418,12 +418,12 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>>  
>>  bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>>  {
>> -	return false;
>> +	return vcpu_mode_priv(vcpu);
>>  }
>>  
>>  bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>>  {
>> -	return false;
>> +	return vcpu_mode_priv(vcpu);
>>  }
>>  
>>  /* Just ensure a guest exit from a particular CPU */
>>
> 
> Can you split that into two parts? (arm and s390x?)


OK, I'll split in V2. :)

> 


-- 
Regards,
Longpeng(Mike)

[toc] | [prev] | [next] | [standalone]

#1705260 — [PATCH 1/3] KVM: add spinlock-exiting optimize framework

From"Longpeng(Mike)" <longpeng2@huawei.com>
Date2017-08-07 10:50 +0200
Subject[PATCH 1/3] KVM: add spinlock-exiting optimize framework
Message-ID<ubLTB-5V4-43@gated-at.bofh.it>
In reply to#1705249
If the vcpu(me) exit due to request a usermode spinlock, then
the spinlock-holder may be preempted in usermode or kernmode.

But if the vcpu(me) is in kernmode, then the holder must be
preempted in kernmode, so we should choose a vcpu in kernmode
as the most eligible candidate.

For some architecture(e.g. arm/s390), spin/preempt_in_kernel()
are the same, but they are different for X86.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
---
 arch/mips/kvm/mips.c       | 10 ++++++++++
 arch/powerpc/kvm/powerpc.c | 10 ++++++++++
 arch/s390/kvm/kvm-s390.c   | 10 ++++++++++
 arch/x86/kvm/x86.c         | 10 ++++++++++
 include/linux/kvm_host.h   |  2 ++
 virt/kvm/arm/arm.c         | 10 ++++++++++
 virt/kvm/kvm_main.c        |  4 ++++
 7 files changed, 56 insertions(+)

diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index d4b2ad1..e04e6b3 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -98,6 +98,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 	return !!(vcpu->arch.pending_exceptions);
 }
 
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 {
 	return 1;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 1a75c0b..c573ddd 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -58,6 +58,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 	return !!(v->arch.pending_exceptions) || kvm_request_pending(v);
 }
 
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 {
 	return 1;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index af09d34..f78cdc2 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -2447,6 +2447,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 	return kvm_s390_vcpu_has_irq(vcpu, 0);
 }
 
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
 {
 	atomic_or(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6c97c82..04c6a1f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8435,6 +8435,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 	return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu);
 }
 
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 {
 	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 890b706..9613620 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -798,6 +798,8 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
 void kvm_arch_hardware_unsetup(void);
 void kvm_arch_check_processor_compat(void *rtn);
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu);
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
 
 #ifndef __KVM_HAVE_ARCH_VM_ALLOC
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index a39a1e1..e45f780 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -416,6 +416,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 		&& !v->arch.power_off && !v->arch.pause);
 }
 
+bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 /* Just ensure a guest exit from a particular CPU */
 static void exit_vm_noop(void *info)
 {
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index f3f7427..0d0527b 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2324,12 +2324,14 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
 {
 	struct kvm *kvm = me->kvm;
 	struct kvm_vcpu *vcpu;
+	bool in_kern;
 	int last_boosted_vcpu = me->kvm->last_boosted_vcpu;
 	int yielded = 0;
 	int try = 3;
 	int pass;
 	int i;
 
+	in_kern = kvm_arch_vcpu_spin_in_kernel(me);
 	kvm_vcpu_set_in_spin_loop(me, true);
 	/*
 	 * We boost the priority of a VCPU that is runnable but not
@@ -2351,6 +2353,8 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
 				continue;
 			if (swait_active(&vcpu->wq) && !kvm_arch_vcpu_runnable(vcpu))
 				continue;
+			if (in_kern && !kvm_arch_vcpu_preempt_in_kernel(vcpu))
+				continue;
 			if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
 				continue;
 
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1705267 — Re: [PATCH 1/3] KVM: add spinlock-exiting optimize framework

FromDavid Hildenbrand <david@redhat.com>
Date2017-08-07 11:00 +0200
SubjectRe: [PATCH 1/3] KVM: add spinlock-exiting optimize framework
Message-ID<ubM3g-5YJ-13@gated-at.bofh.it>
In reply to#1705260
On 07.08.2017 10:44, Longpeng(Mike) wrote:
> If the vcpu(me) exit due to request a usermode spinlock, then
> the spinlock-holder may be preempted in usermode or kernmode.
> 
> But if the vcpu(me) is in kernmode, then the holder must be
> preempted in kernmode, so we should choose a vcpu in kernmode
> as the most eligible candidate.
> 
> For some architecture(e.g. arm/s390), spin/preempt_in_kernel()
> are the same, but they are different for X86.
> 
> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
> ---
>  arch/mips/kvm/mips.c       | 10 ++++++++++
>  arch/powerpc/kvm/powerpc.c | 10 ++++++++++
>  arch/s390/kvm/kvm-s390.c   | 10 ++++++++++
>  arch/x86/kvm/x86.c         | 10 ++++++++++
>  include/linux/kvm_host.h   |  2 ++
>  virt/kvm/arm/arm.c         | 10 ++++++++++
>  virt/kvm/kvm_main.c        |  4 ++++
>  7 files changed, 56 insertions(+)
> 
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index d4b2ad1..e04e6b3 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -98,6 +98,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>  	return !!(vcpu->arch.pending_exceptions);
>  }
>  
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>  {
>  	return 1;
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 1a75c0b..c573ddd 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -58,6 +58,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>  	return !!(v->arch.pending_exceptions) || kvm_request_pending(v);
>  }
>  
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>  {
>  	return 1;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index af09d34..f78cdc2 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -2447,6 +2447,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>  	return kvm_s390_vcpu_has_irq(vcpu, 0);
>  }
>  
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
>  void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
>  {
>  	atomic_or(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20);
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 6c97c82..04c6a1f 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8435,6 +8435,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>  	return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu);
>  }
>  
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>  {
>  	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 890b706..9613620 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -798,6 +798,8 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
>  void kvm_arch_hardware_unsetup(void);
>  void kvm_arch_check_processor_compat(void *rtn);
>  int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu);
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu);
>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
>  
>  #ifndef __KVM_HAVE_ARCH_VM_ALLOC
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index a39a1e1..e45f780 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -416,6 +416,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>  		&& !v->arch.power_off && !v->arch.pause);
>  }
>  
> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}
> +
> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
> +{
> +	return false;
> +}

Is the differentiation really necessary?

Can't you cache for x86 in all scenarios and simply introduce
kvm_arch_vcpu_in_kernel() ?

Otherwise, we have complexity that might just be avoided (e.g.
kvm_arch_vcpu_spin_in_kernel must only be called on the loaded VCPU)

> +
>  /* Just ensure a guest exit from a particular CPU */
>  static void exit_vm_noop(void *info)
>  {
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index f3f7427..0d0527b 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2324,12 +2324,14 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
>  {
>  	struct kvm *kvm = me->kvm;
>  	struct kvm_vcpu *vcpu;
> +	bool in_kern;
>  	int last_boosted_vcpu = me->kvm->last_boosted_vcpu;
>  	int yielded = 0;
>  	int try = 3;
>  	int pass;
>  	int i;
>  
> +	in_kern = kvm_arch_vcpu_spin_in_kernel(me);
>  	kvm_vcpu_set_in_spin_loop(me, true);
>  	/*
>  	 * We boost the priority of a VCPU that is runnable but not
> @@ -2351,6 +2353,8 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
>  				continue;
>  			if (swait_active(&vcpu->wq) && !kvm_arch_vcpu_runnable(vcpu))
>  				continue;
> +			if (in_kern && !kvm_arch_vcpu_preempt_in_kernel(vcpu))
> +				continue;
>  			if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
>  				continue;
>  
> 


-- 

Thanks,

David

[toc] | [prev] | [next] | [standalone]

#1705271 — Re: [PATCH 1/3] KVM: add spinlock-exiting optimize framework

From"Longpeng (Mike)" <longpeng2@huawei.com>
Date2017-08-07 11:10 +0200
SubjectRe: [PATCH 1/3] KVM: add spinlock-exiting optimize framework
Message-ID<ubMcV-6gU-7@gated-at.bofh.it>
In reply to#1705267

On 2017/8/7 16:55, David Hildenbrand wrote:

> On 07.08.2017 10:44, Longpeng(Mike) wrote:
>> If the vcpu(me) exit due to request a usermode spinlock, then
>> the spinlock-holder may be preempted in usermode or kernmode.
>>
>> But if the vcpu(me) is in kernmode, then the holder must be
>> preempted in kernmode, so we should choose a vcpu in kernmode
>> as the most eligible candidate.
>>
>> For some architecture(e.g. arm/s390), spin/preempt_in_kernel()
>> are the same, but they are different for X86.
>>
>> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
>> ---
>>  arch/mips/kvm/mips.c       | 10 ++++++++++
>>  arch/powerpc/kvm/powerpc.c | 10 ++++++++++
>>  arch/s390/kvm/kvm-s390.c   | 10 ++++++++++
>>  arch/x86/kvm/x86.c         | 10 ++++++++++
>>  include/linux/kvm_host.h   |  2 ++
>>  virt/kvm/arm/arm.c         | 10 ++++++++++
>>  virt/kvm/kvm_main.c        |  4 ++++
>>  7 files changed, 56 insertions(+)
>>
>> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
>> index d4b2ad1..e04e6b3 100644
>> --- a/arch/mips/kvm/mips.c
>> +++ b/arch/mips/kvm/mips.c
>> @@ -98,6 +98,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>>  	return !!(vcpu->arch.pending_exceptions);
>>  }
>>  
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>>  {
>>  	return 1;
>> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
>> index 1a75c0b..c573ddd 100644
>> --- a/arch/powerpc/kvm/powerpc.c
>> +++ b/arch/powerpc/kvm/powerpc.c
>> @@ -58,6 +58,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>>  	return !!(v->arch.pending_exceptions) || kvm_request_pending(v);
>>  }
>>  
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>>  {
>>  	return 1;
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index af09d34..f78cdc2 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -2447,6 +2447,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>>  	return kvm_s390_vcpu_has_irq(vcpu, 0);
>>  }
>>  
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>>  void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu)
>>  {
>>  	atomic_or(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20);
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 6c97c82..04c6a1f 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -8435,6 +8435,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
>>  	return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu);
>>  }
>>  
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
>>  {
>>  	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
>> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> index 890b706..9613620 100644
>> --- a/include/linux/kvm_host.h
>> +++ b/include/linux/kvm_host.h
>> @@ -798,6 +798,8 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
>>  void kvm_arch_hardware_unsetup(void);
>>  void kvm_arch_check_processor_compat(void *rtn);
>>  int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu);
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu);
>>  int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
>>  
>>  #ifndef __KVM_HAVE_ARCH_VM_ALLOC
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index a39a1e1..e45f780 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -416,6 +416,16 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>>  		&& !v->arch.power_off && !v->arch.pause);
>>  }
>>  
>> +bool kvm_arch_vcpu_spin_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
>> +
>> +bool kvm_arch_vcpu_preempt_in_kernel(struct kvm_vcpu *vcpu)
>> +{
>> +	return false;
>> +}
> 
> Is the differentiation really necessary?
> 
> Can't you cache for x86 in all scenarios and simply introduce
> kvm_arch_vcpu_in_kernel() ?
> 


For X86 this is necessary,  I have no idea how to avoid this, hopes
someone could give me some suggestion. :)

> Otherwise, we have complexity that might just be avoided (e.g.
> kvm_arch_vcpu_spin_in_kernel must only be called on the loaded VCPU)
> 
>> +
>>  /* Just ensure a guest exit from a particular CPU */
>>  static void exit_vm_noop(void *info)
>>  {
>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>> index f3f7427..0d0527b 100644
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -2324,12 +2324,14 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
>>  {
>>  	struct kvm *kvm = me->kvm;
>>  	struct kvm_vcpu *vcpu;
>> +	bool in_kern;
>>  	int last_boosted_vcpu = me->kvm->last_boosted_vcpu;
>>  	int yielded = 0;
>>  	int try = 3;
>>  	int pass;
>>  	int i;
>>  
>> +	in_kern = kvm_arch_vcpu_spin_in_kernel(me);
>>  	kvm_vcpu_set_in_spin_loop(me, true);
>>  	/*
>>  	 * We boost the priority of a VCPU that is runnable but not
>> @@ -2351,6 +2353,8 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)
>>  				continue;
>>  			if (swait_active(&vcpu->wq) && !kvm_arch_vcpu_runnable(vcpu))
>>  				continue;
>> +			if (in_kern && !kvm_arch_vcpu_preempt_in_kernel(vcpu))
>> +				continue;
>>  			if (!kvm_vcpu_eligible_for_directed_yield(vcpu))
>>  				continue;
>>  
>>
> 
> 


-- 
Regards,
Longpeng(Mike)

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web