Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1705996 > unrolled thread
| Started by | "Longpeng(Mike)" <longpeng2@huawei.com> |
|---|---|
| First post | 2017-08-08 06:10 +0200 |
| Last post | 2017-08-08 14:00 +0200 |
| Articles | 13 — 7 participants |
Back to article view | Back to linux.kernel
[PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin "Longpeng(Mike)" <longpeng2@huawei.com> - 2017-08-08 06:10 +0200
[PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() "Longpeng(Mike)" <longpeng2@huawei.com> - 2017-08-08 06:10 +0200
Re: [PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() Christoffer Dall <cdall@linaro.org> - 2017-08-08 09:40 +0200
Re: [PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() Christoffer Dall <christoffer.dall@linaro.org> - 2017-08-08 09:50 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin Cornelia Huck <cohuck@redhat.com> - 2017-08-08 09:50 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin "Longpeng (Mike)" <longpeng2@huawei.com> - 2017-08-08 10:20 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin Eric Farman <farman@linux.vnet.ibm.com> - 2017-08-10 15:20 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin "Longpeng (Mike)" <longpeng2@huawei.com> - 2017-08-11 03:50 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin Cornelia Huck <cohuck@redhat.com> - 2017-08-11 09:30 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin David Hildenbrand <david@redhat.com> - 2017-08-08 13:30 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin Christoffer Dall <christoffer.dall@linaro.org> - 2017-08-08 13:40 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin "Longpeng (Mike)" <longpeng2@huawei.com> - 2017-08-08 13:50 +0200
Re: [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin David Hildenbrand <david@redhat.com> - 2017-08-08 14:00 +0200
| From | "Longpeng(Mike)" <longpeng2@huawei.com> |
|---|---|
| Date | 2017-08-08 06:10 +0200 |
| Subject | [PATCH v2 0/4] KVM: optimize the kvm_vcpu_on_spin |
| Message-ID | <uc409-2u9-11@gated-at.bofh.it> |
This is a simple optimization for kvm_vcpu_on_spin, the
main idea is described in patch-1's commit msg.
I did some tests base on the RFC version, the result shows
that it can improves the performance slightly.
== Geekbench-3.4.1 ==
VM1: 8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19)
running Geekbench-3.4.1 *10 truns*
VM2/VM3/VM4: configure is the same as VM1
stress each vcpu usage(seed by top in guest) to 40%
The comparison of each testcase's score:
(higher is better)
before after improve
Inter
single 1176.7 1179.0 0.2%
multi 3459.5 3426.5 -0.9%
Float
single 1150.5 1150.9 0.0%
multi 3364.5 3391.9 0.8%
Memory(stream)
single 1768.7 1773.1 0.2%
multi 2511.6 2557.2 1.8%
Overall
single 1284.2 1286.2 0.2%
multi 3231.4 3238.4 0.2%
== kernbench-0.42 ==
VM1: 8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19)
running "kernbench -n 10"
VM2/VM3/VM4: configure is the same as VM1
stress each vcpu usage(seed by top in guest) to 40%
The comparison of 'Elapsed Time':
(sooner is better)
before after improve
load -j4 12.762 12.751 0.1%
load -j32 9.743 8.955 8.1%
load -j 9.688 9.229 4.7%
Physical Machine:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 24
On-line CPU(s) list: 0-23
Thread(s) per core: 2
Core(s) per socket: 6
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Model name: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz
Stepping: 7
CPU MHz: 2799.902
BogoMIPS: 5004.67
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 15360K
NUMA node0 CPU(s): 0-5,12-17
NUMA node1 CPU(s): 6-11,18-23
---
Changes since V1:
- split the implementation of s390 & arm. [David]
- refactor the impls according to the suggestion. [Paolo]
Changes since RFC:
- only cache result for X86. [David & Cornlia & Paolo]
- add performance numbers. [David]
- impls arm/s390. [Christoffer & David]
- refactor the impls. [me]
---
Longpeng(Mike) (4):
KVM: add spinlock optimization framework
KVM: X86: implement the logic for spinlock optimization
KVM: s390: implements the kvm_arch_vcpu_in_kernel()
KVM: arm: implements the kvm_arch_vcpu_in_kernel()
arch/arm/kvm/handle_exit.c | 2 +-
arch/arm64/kvm/handle_exit.c | 2 +-
arch/mips/kvm/mips.c | 6 ++++++
arch/powerpc/kvm/powerpc.c | 6 ++++++
arch/s390/kvm/diag.c | 2 +-
arch/s390/kvm/kvm-s390.c | 6 ++++++
arch/x86/include/asm/kvm_host.h | 5 +++++
arch/x86/kvm/hyperv.c | 2 +-
arch/x86/kvm/svm.c | 10 +++++++++-
arch/x86/kvm/vmx.c | 16 +++++++++++++++-
arch/x86/kvm/x86.c | 11 +++++++++++
include/linux/kvm_host.h | 3 ++-
virt/kvm/arm/arm.c | 5 +++++
virt/kvm/kvm_main.c | 4 +++-
14 files changed, 72 insertions(+), 8 deletions(-)
--
1.8.3.1
[toc] | [next] | [standalone]
| From | "Longpeng(Mike)" <longpeng2@huawei.com> |
|---|---|
| Date | 2017-08-08 06:10 +0200 |
| Subject | [PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() |
| Message-ID | <uc40a-2u9-23@gated-at.bofh.it> |
| In reply to | #1705996 |
This implements the kvm_arch_vcpu_in_kernel() for ARM.
Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
---
virt/kvm/arm/arm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 862f820..b9f68e4 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -418,7 +418,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
{
- return false;
+ return vcpu_mode_priv(vcpu);
}
/* Just ensure a guest exit from a particular CPU */
--
1.8.3.1
[toc] | [prev] | [next] | [standalone]
| From | Christoffer Dall <cdall@linaro.org> |
|---|---|
| Date | 2017-08-08 09:40 +0200 |
| Subject | Re: [PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() |
| Message-ID | <uc7hn-4KD-9@gated-at.bofh.it> |
| In reply to | #1705997 |
On Tue, Aug 08, 2017 at 12:05:35PM +0800, Longpeng(Mike) wrote:
> This implements the kvm_arch_vcpu_in_kernel() for ARM.
>
> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
> ---
> virt/kvm/arm/arm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 862f820..b9f68e4 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -418,7 +418,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>
> bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
> {
> - return false;
> + return vcpu_mode_priv(vcpu);
> }
>
> /* Just ensure a guest exit from a particular CPU */
> --
> 1.8.3.1
>
>
I'm not taking any position on whether this series makes sense overall
or not (it's a bit concerning to do this without having measured a
benefit), but for the arm/arm64 change:
Acked-by: Christoffer Dall <cdall@linaro.org>
[toc] | [prev] | [next] | [standalone]
| From | Christoffer Dall <christoffer.dall@linaro.org> |
|---|---|
| Date | 2017-08-08 09:50 +0200 |
| Subject | Re: [PATCH v2 4/4] KVM: arm: implements the kvm_arch_vcpu_in_kernel() |
| Message-ID | <uc7r3-4Ox-13@gated-at.bofh.it> |
| In reply to | #1706108 |
On Tue, Aug 8, 2017 at 9:39 AM, Christoffer Dall <cdall@linaro.org> wrote:
> On Tue, Aug 08, 2017 at 12:05:35PM +0800, Longpeng(Mike) wrote:
>> This implements the kvm_arch_vcpu_in_kernel() for ARM.
>>
>> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
>> ---
>> virt/kvm/arm/arm.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 862f820..b9f68e4 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -418,7 +418,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>>
>> bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
>> {
>> - return false;
>> + return vcpu_mode_priv(vcpu);
>> }
>>
>> /* Just ensure a guest exit from a particular CPU */
>> --
>> 1.8.3.1
>>
>>
> I'm not taking any position on whether this series makes sense overall
> or not (it's a bit concerning to do this without having measured a
> benefit), but for the arm/arm64 change:
oh, sorry, strike that, I didn't notice you had added performance
results in the cover letter.
Thanks,
-Christoffer
>
> Acked-by: Christoffer Dall <cdall@linaro.org>
[toc] | [prev] | [next] | [standalone]
| From | Cornelia Huck <cohuck@redhat.com> |
|---|---|
| Date | 2017-08-08 09:50 +0200 |
| Message-ID | <uc7r3-4Ox-3@gated-at.bofh.it> |
| In reply to | #1705996 |
On Tue, 8 Aug 2017 12:05:31 +0800 "Longpeng(Mike)" <longpeng2@huawei.com> wrote: > This is a simple optimization for kvm_vcpu_on_spin, the > main idea is described in patch-1's commit msg. I think this generally looks good now. > > I did some tests base on the RFC version, the result shows > that it can improves the performance slightly. Did you re-run tests on this version? I would also like to see some s390 numbers; unfortunately I only have a z/VM environment and any performance numbers would be nearly useless there. Maybe somebody within IBM with a better setup can run a quick test?
[toc] | [prev] | [next] | [standalone]
| From | "Longpeng (Mike)" <longpeng2@huawei.com> |
|---|---|
| Date | 2017-08-08 10:20 +0200 |
| Message-ID | <uc7U6-5eH-29@gated-at.bofh.it> |
| In reply to | #1706115 |
On 2017/8/8 15:41, Cornelia Huck wrote: > On Tue, 8 Aug 2017 12:05:31 +0800 > "Longpeng(Mike)" <longpeng2@huawei.com> wrote: > >> This is a simple optimization for kvm_vcpu_on_spin, the >> main idea is described in patch-1's commit msg. > > I think this generally looks good now. > >> >> I did some tests base on the RFC version, the result shows >> that it can improves the performance slightly. > > Did you re-run tests on this version? Hi Cornelia, I didn't re-run tests on V2. But the major difference between RFC and V2 is that V2 only cache result for X86 (s390/arm needn't) and V2 saves a expensive operation ( 440-1400 cycles on my test machine ) for X86/VMX. So I think V2's performance is at least the same as RFC or even slightly better. :) > > I would also like to see some s390 numbers; unfortunately I only have a > z/VM environment and any performance numbers would be nearly useless > there. Maybe somebody within IBM with a better setup can run a quick > test? > > . > -- Regards, Longpeng(Mike)
[toc] | [prev] | [next] | [standalone]
| From | Eric Farman <farman@linux.vnet.ibm.com> |
|---|---|
| Date | 2017-08-10 15:20 +0200 |
| Message-ID | <ucVxw-62g-19@gated-at.bofh.it> |
| In reply to | #1706143 |
On 08/08/2017 04:14 AM, Longpeng (Mike) wrote: > > > On 2017/8/8 15:41, Cornelia Huck wrote: > >> On Tue, 8 Aug 2017 12:05:31 +0800 >> "Longpeng(Mike)" <longpeng2@huawei.com> wrote: >> >>> This is a simple optimization for kvm_vcpu_on_spin, the >>> main idea is described in patch-1's commit msg. >> >> I think this generally looks good now. >> >>> >>> I did some tests base on the RFC version, the result shows >>> that it can improves the performance slightly. >> >> Did you re-run tests on this version? > > > Hi Cornelia, > > I didn't re-run tests on V2. But the major difference between RFC and V2 > is that V2 only cache result for X86 (s390/arm needn't) and V2 saves a > expensive operation ( 440-1400 cycles on my test machine ) for X86/VMX. > > So I think V2's performance is at least the same as RFC or even slightly > better. :) > >> >> I would also like to see some s390 numbers; unfortunately I only have a >> z/VM environment and any performance numbers would be nearly useless >> there. Maybe somebody within IBM with a better setup can run a quick >> test? Won't swear I didn't screw something up, but here's some quick numbers. Host was 4.12.0 with and without this series, running QEMU 2.10.0-rc0. Created 4 guests, each with 4 CPU (unpinned) and 4GB RAM. VM1 did full kernel compiles with kernbench, which took averages of 5 runs of different job sizes (I threw away the "-j 1" numbers). VM2-VM4 ran cpu burners on 2 of their 4 cpus. Numbers from VM1 kernbench output, and the delta between runs: load -j 3 before after delta Elapsed Time 183.178 182.58 -0.598 User Time 534.19 531.52 -2.67 System Time 32.538 33.37 0.832 Percent CPU 308.8 309 0.2 Context Switches 98484.6 99001 516.4 Sleeps 227347 228752 1405 load -j 16 before after delta Elapsed Time 153.352 147.59 -5.762 User Time 545.829 533.41 -12.419 System Time 34.289 34.85 0.561 Percent CPU 347.6 348 0.4 Context Switches 160518 159120 -1398 Sleeps 240740 240536 -204 - Eric
[toc] | [prev] | [next] | [standalone]
| From | "Longpeng (Mike)" <longpeng2@huawei.com> |
|---|---|
| Date | 2017-08-11 03:50 +0200 |
| Message-ID | <ud7fk-5uc-7@gated-at.bofh.it> |
| In reply to | #1708626 |
On 2017/8/10 21:18, Eric Farman wrote: > > > On 08/08/2017 04:14 AM, Longpeng (Mike) wrote: >> >> >> On 2017/8/8 15:41, Cornelia Huck wrote: >> >>> On Tue, 8 Aug 2017 12:05:31 +0800 >>> "Longpeng(Mike)" <longpeng2@huawei.com> wrote: >>> >>>> This is a simple optimization for kvm_vcpu_on_spin, the >>>> main idea is described in patch-1's commit msg. >>> >>> I think this generally looks good now. >>> >>>> >>>> I did some tests base on the RFC version, the result shows >>>> that it can improves the performance slightly. >>> >>> Did you re-run tests on this version? >> >> >> Hi Cornelia, >> >> I didn't re-run tests on V2. But the major difference between RFC and V2 >> is that V2 only cache result for X86 (s390/arm needn't) and V2 saves a >> expensive operation ( 440-1400 cycles on my test machine ) for X86/VMX. >> >> So I think V2's performance is at least the same as RFC or even slightly >> better. :) >> >>> >>> I would also like to see some s390 numbers; unfortunately I only have a >>> z/VM environment and any performance numbers would be nearly useless >>> there. Maybe somebody within IBM with a better setup can run a quick >>> test? > > Won't swear I didn't screw something up, but here's some quick numbers. Host was > 4.12.0 with and without this series, running QEMU 2.10.0-rc0. Created 4 guests, > each with 4 CPU (unpinned) and 4GB RAM. VM1 did full kernel compiles with > kernbench, which took averages of 5 runs of different job sizes (I threw away > the "-j 1" numbers). VM2-VM4 ran cpu burners on 2 of their 4 cpus. > > Numbers from VM1 kernbench output, and the delta between runs: > > load -j 3 before after delta > Elapsed Time 183.178 182.58 -0.598 > User Time 534.19 531.52 -2.67 > System Time 32.538 33.37 0.832 > Percent CPU 308.8 309 0.2 > Context Switches 98484.6 99001 516.4 > Sleeps 227347 228752 1405 > > load -j 16 before after delta > Elapsed Time 153.352 147.59 -5.762 > User Time 545.829 533.41 -12.419 > System Time 34.289 34.85 0.561 > Percent CPU 347.6 348 0.4 > Context Switches 160518 159120 -1398 > Sleeps 240740 240536 -204 > Thanks Eric! The `Elapsed Time` is smaller with this series , the result is the same as my numbers in cover-letter. > > - Eric > > > . > -- Regards, Longpeng(Mike)
[toc] | [prev] | [next] | [standalone]
| From | Cornelia Huck <cohuck@redhat.com> |
|---|---|
| Date | 2017-08-11 09:30 +0200 |
| Message-ID | <udcym-A7-13@gated-at.bofh.it> |
| In reply to | #1708626 |
On Thu, 10 Aug 2017 09:18:09 -0400 Eric Farman <farman@linux.vnet.ibm.com> wrote: > On 08/08/2017 04:14 AM, Longpeng (Mike) wrote: > > > > > > On 2017/8/8 15:41, Cornelia Huck wrote: > > > >> On Tue, 8 Aug 2017 12:05:31 +0800 > >> "Longpeng(Mike)" <longpeng2@huawei.com> wrote: > >> > >>> This is a simple optimization for kvm_vcpu_on_spin, the > >>> main idea is described in patch-1's commit msg. > >> > >> I think this generally looks good now. > >> > >>> > >>> I did some tests base on the RFC version, the result shows > >>> that it can improves the performance slightly. > >> > >> Did you re-run tests on this version? > > > > > > Hi Cornelia, > > > > I didn't re-run tests on V2. But the major difference between RFC and V2 > > is that V2 only cache result for X86 (s390/arm needn't) and V2 saves a > > expensive operation ( 440-1400 cycles on my test machine ) for X86/VMX. > > > > So I think V2's performance is at least the same as RFC or even slightly > > better. :) > > > >> > >> I would also like to see some s390 numbers; unfortunately I only have a > >> z/VM environment and any performance numbers would be nearly useless > >> there. Maybe somebody within IBM with a better setup can run a quick > >> test? > > Won't swear I didn't screw something up, but here's some quick numbers. > Host was 4.12.0 with and without this series, running QEMU 2.10.0-rc0. > Created 4 guests, each with 4 CPU (unpinned) and 4GB RAM. VM1 did full > kernel compiles with kernbench, which took averages of 5 runs of > different job sizes (I threw away the "-j 1" numbers). VM2-VM4 ran cpu > burners on 2 of their 4 cpus. > > Numbers from VM1 kernbench output, and the delta between runs: > > load -j 3 before after delta > Elapsed Time 183.178 182.58 -0.598 > User Time 534.19 531.52 -2.67 > System Time 32.538 33.37 0.832 > Percent CPU 308.8 309 0.2 > Context Switches 98484.6 99001 516.4 > Sleeps 227347 228752 1405 > > load -j 16 before after delta > Elapsed Time 153.352 147.59 -5.762 > User Time 545.829 533.41 -12.419 > System Time 34.289 34.85 0.561 > Percent CPU 347.6 348 0.4 > Context Switches 160518 159120 -1398 > Sleeps 240740 240536 -204 Thanks a lot, Eric! The decreases in elapsed time look nice, and we probably should not care about the increases reported.
[toc] | [prev] | [next] | [standalone]
| From | David Hildenbrand <david@redhat.com> |
|---|---|
| Date | 2017-08-08 13:30 +0200 |
| Message-ID | <ucaRX-7FO-3@gated-at.bofh.it> |
| In reply to | #1705996 |
On 08.08.2017 06:05, Longpeng(Mike) wrote: > This is a simple optimization for kvm_vcpu_on_spin, the > main idea is described in patch-1's commit msg. > > I did some tests base on the RFC version, the result shows > that it can improves the performance slightly. > > == Geekbench-3.4.1 == > VM1: 8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) > running Geekbench-3.4.1 *10 truns* > VM2/VM3/VM4: configure is the same as VM1 > stress each vcpu usage(seed by top in guest) to 40% > > The comparison of each testcase's score: > (higher is better) > before after improve > Inter > single 1176.7 1179.0 0.2% > multi 3459.5 3426.5 -0.9% > Float > single 1150.5 1150.9 0.0% > multi 3364.5 3391.9 0.8% > Memory(stream) > single 1768.7 1773.1 0.2% > multi 2511.6 2557.2 1.8% > Overall > single 1284.2 1286.2 0.2% > multi 3231.4 3238.4 0.2% > > > == kernbench-0.42 == > VM1: 8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) > running "kernbench -n 10" > VM2/VM3/VM4: configure is the same as VM1 > stress each vcpu usage(seed by top in guest) to 40% > > The comparison of 'Elapsed Time': > (sooner is better) > before after improve > load -j4 12.762 12.751 0.1% > load -j32 9.743 8.955 8.1% > load -j 9.688 9.229 4.7% > > > Physical Machine: > Architecture: x86_64 > CPU op-mode(s): 32-bit, 64-bit > Byte Order: Little Endian > CPU(s): 24 > On-line CPU(s) list: 0-23 > Thread(s) per core: 2 > Core(s) per socket: 6 > Socket(s): 2 > NUMA node(s): 2 > Vendor ID: GenuineIntel > CPU family: 6 > Model: 45 > Model name: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz > Stepping: 7 > CPU MHz: 2799.902 > BogoMIPS: 5004.67 > Virtualization: VT-x > L1d cache: 32K > L1i cache: 32K > L2 cache: 256K > L3 cache: 15360K > NUMA node0 CPU(s): 0-5,12-17 > NUMA node1 CPU(s): 6-11,18-23 > > --- > Changes since V1: > - split the implementation of s390 & arm. [David] > - refactor the impls according to the suggestion. [Paolo] > > Changes since RFC: > - only cache result for X86. [David & Cornlia & Paolo] > - add performance numbers. [David] > - impls arm/s390. [Christoffer & David] > - refactor the impls. [me] > > --- > Longpeng(Mike) (4): > KVM: add spinlock optimization framework > KVM: X86: implement the logic for spinlock optimization > KVM: s390: implements the kvm_arch_vcpu_in_kernel() > KVM: arm: implements the kvm_arch_vcpu_in_kernel() > > arch/arm/kvm/handle_exit.c | 2 +- > arch/arm64/kvm/handle_exit.c | 2 +- > arch/mips/kvm/mips.c | 6 ++++++ > arch/powerpc/kvm/powerpc.c | 6 ++++++ > arch/s390/kvm/diag.c | 2 +- > arch/s390/kvm/kvm-s390.c | 6 ++++++ > arch/x86/include/asm/kvm_host.h | 5 +++++ > arch/x86/kvm/hyperv.c | 2 +- > arch/x86/kvm/svm.c | 10 +++++++++- > arch/x86/kvm/vmx.c | 16 +++++++++++++++- > arch/x86/kvm/x86.c | 11 +++++++++++ > include/linux/kvm_host.h | 3 ++- > virt/kvm/arm/arm.c | 5 +++++ > virt/kvm/kvm_main.c | 4 +++- > 14 files changed, 72 insertions(+), 8 deletions(-) > I am curious, is there any architecture that allows to trigger kvm_vcpu_on_spin(vcpu); while _not_ in kernel mode? I would have guessed that user space should never be allowed to make cpu wide decisions (giving up the CPU to the hypervisor). E.g. s390x diag can only be executed from kernel space. VMX PAUSE is only valid from kernel space. I.o.w. do we need a parameter to kvm_vcpu_on_spin(vcpu); at all, or is "me_in_kernel" basically always true? -- Thanks, David
[toc] | [prev] | [next] | [standalone]
| From | Christoffer Dall <christoffer.dall@linaro.org> |
|---|---|
| Date | 2017-08-08 13:40 +0200 |
| Message-ID | <ucb1D-7Kd-13@gated-at.bofh.it> |
| In reply to | #1706369 |
On Tue, Aug 8, 2017 at 1:25 PM, David Hildenbrand <david@redhat.com> wrote: > On 08.08.2017 06:05, Longpeng(Mike) wrote: >> This is a simple optimization for kvm_vcpu_on_spin, the >> main idea is described in patch-1's commit msg. >> >> I did some tests base on the RFC version, the result shows >> that it can improves the performance slightly. >> >> == Geekbench-3.4.1 == >> VM1: 8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >> running Geekbench-3.4.1 *10 truns* >> VM2/VM3/VM4: configure is the same as VM1 >> stress each vcpu usage(seed by top in guest) to 40% >> >> The comparison of each testcase's score: >> (higher is better) >> before after improve >> Inter >> single 1176.7 1179.0 0.2% >> multi 3459.5 3426.5 -0.9% >> Float >> single 1150.5 1150.9 0.0% >> multi 3364.5 3391.9 0.8% >> Memory(stream) >> single 1768.7 1773.1 0.2% >> multi 2511.6 2557.2 1.8% >> Overall >> single 1284.2 1286.2 0.2% >> multi 3231.4 3238.4 0.2% >> >> >> == kernbench-0.42 == >> VM1: 8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >> running "kernbench -n 10" >> VM2/VM3/VM4: configure is the same as VM1 >> stress each vcpu usage(seed by top in guest) to 40% >> >> The comparison of 'Elapsed Time': >> (sooner is better) >> before after improve >> load -j4 12.762 12.751 0.1% >> load -j32 9.743 8.955 8.1% >> load -j 9.688 9.229 4.7% >> >> >> Physical Machine: >> Architecture: x86_64 >> CPU op-mode(s): 32-bit, 64-bit >> Byte Order: Little Endian >> CPU(s): 24 >> On-line CPU(s) list: 0-23 >> Thread(s) per core: 2 >> Core(s) per socket: 6 >> Socket(s): 2 >> NUMA node(s): 2 >> Vendor ID: GenuineIntel >> CPU family: 6 >> Model: 45 >> Model name: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz >> Stepping: 7 >> CPU MHz: 2799.902 >> BogoMIPS: 5004.67 >> Virtualization: VT-x >> L1d cache: 32K >> L1i cache: 32K >> L2 cache: 256K >> L3 cache: 15360K >> NUMA node0 CPU(s): 0-5,12-17 >> NUMA node1 CPU(s): 6-11,18-23 >> >> --- >> Changes since V1: >> - split the implementation of s390 & arm. [David] >> - refactor the impls according to the suggestion. [Paolo] >> >> Changes since RFC: >> - only cache result for X86. [David & Cornlia & Paolo] >> - add performance numbers. [David] >> - impls arm/s390. [Christoffer & David] >> - refactor the impls. [me] >> >> --- >> Longpeng(Mike) (4): >> KVM: add spinlock optimization framework >> KVM: X86: implement the logic for spinlock optimization >> KVM: s390: implements the kvm_arch_vcpu_in_kernel() >> KVM: arm: implements the kvm_arch_vcpu_in_kernel() >> >> arch/arm/kvm/handle_exit.c | 2 +- >> arch/arm64/kvm/handle_exit.c | 2 +- >> arch/mips/kvm/mips.c | 6 ++++++ >> arch/powerpc/kvm/powerpc.c | 6 ++++++ >> arch/s390/kvm/diag.c | 2 +- >> arch/s390/kvm/kvm-s390.c | 6 ++++++ >> arch/x86/include/asm/kvm_host.h | 5 +++++ >> arch/x86/kvm/hyperv.c | 2 +- >> arch/x86/kvm/svm.c | 10 +++++++++- >> arch/x86/kvm/vmx.c | 16 +++++++++++++++- >> arch/x86/kvm/x86.c | 11 +++++++++++ >> include/linux/kvm_host.h | 3 ++- >> virt/kvm/arm/arm.c | 5 +++++ >> virt/kvm/kvm_main.c | 4 +++- >> 14 files changed, 72 insertions(+), 8 deletions(-) >> > > I am curious, is there any architecture that allows to trigger > kvm_vcpu_on_spin(vcpu); while _not_ in kernel mode? > > I would have guessed that user space should never be allowed to make cpu > wide decisions (giving up the CPU to the hypervisor). > > E.g. s390x diag can only be executed from kernel space. VMX PAUSE is > only valid from kernel space. > > I.o.w. do we need a parameter to kvm_vcpu_on_spin(vcpu); at all, or is > "me_in_kernel" basically always true? > ARM can be configured to not trap WFE in userspace. Thanks, -Christoffer
[toc] | [prev] | [next] | [standalone]
| From | "Longpeng (Mike)" <longpeng2@huawei.com> |
|---|---|
| Date | 2017-08-08 13:50 +0200 |
| Message-ID | <ucbbk-7PZ-27@gated-at.bofh.it> |
| In reply to | #1706369 |
On 2017/8/8 19:25, David Hildenbrand wrote: > On 08.08.2017 06:05, Longpeng(Mike) wrote: >> This is a simple optimization for kvm_vcpu_on_spin, the >> main idea is described in patch-1's commit msg. >> >> I did some tests base on the RFC version, the result shows >> that it can improves the performance slightly. >> >> == Geekbench-3.4.1 == >> VM1: 8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >> running Geekbench-3.4.1 *10 truns* >> VM2/VM3/VM4: configure is the same as VM1 >> stress each vcpu usage(seed by top in guest) to 40% >> >> The comparison of each testcase's score: >> (higher is better) >> before after improve >> Inter >> single 1176.7 1179.0 0.2% >> multi 3459.5 3426.5 -0.9% >> Float >> single 1150.5 1150.9 0.0% >> multi 3364.5 3391.9 0.8% >> Memory(stream) >> single 1768.7 1773.1 0.2% >> multi 2511.6 2557.2 1.8% >> Overall >> single 1284.2 1286.2 0.2% >> multi 3231.4 3238.4 0.2% >> >> >> == kernbench-0.42 == >> VM1: 8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >> running "kernbench -n 10" >> VM2/VM3/VM4: configure is the same as VM1 >> stress each vcpu usage(seed by top in guest) to 40% >> >> The comparison of 'Elapsed Time': >> (sooner is better) >> before after improve >> load -j4 12.762 12.751 0.1% >> load -j32 9.743 8.955 8.1% >> load -j 9.688 9.229 4.7% >> >> >> Physical Machine: >> Architecture: x86_64 >> CPU op-mode(s): 32-bit, 64-bit >> Byte Order: Little Endian >> CPU(s): 24 >> On-line CPU(s) list: 0-23 >> Thread(s) per core: 2 >> Core(s) per socket: 6 >> Socket(s): 2 >> NUMA node(s): 2 >> Vendor ID: GenuineIntel >> CPU family: 6 >> Model: 45 >> Model name: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz >> Stepping: 7 >> CPU MHz: 2799.902 >> BogoMIPS: 5004.67 >> Virtualization: VT-x >> L1d cache: 32K >> L1i cache: 32K >> L2 cache: 256K >> L3 cache: 15360K >> NUMA node0 CPU(s): 0-5,12-17 >> NUMA node1 CPU(s): 6-11,18-23 >> >> --- >> Changes since V1: >> - split the implementation of s390 & arm. [David] >> - refactor the impls according to the suggestion. [Paolo] >> >> Changes since RFC: >> - only cache result for X86. [David & Cornlia & Paolo] >> - add performance numbers. [David] >> - impls arm/s390. [Christoffer & David] >> - refactor the impls. [me] >> >> --- >> Longpeng(Mike) (4): >> KVM: add spinlock optimization framework >> KVM: X86: implement the logic for spinlock optimization >> KVM: s390: implements the kvm_arch_vcpu_in_kernel() >> KVM: arm: implements the kvm_arch_vcpu_in_kernel() >> >> arch/arm/kvm/handle_exit.c | 2 +- >> arch/arm64/kvm/handle_exit.c | 2 +- >> arch/mips/kvm/mips.c | 6 ++++++ >> arch/powerpc/kvm/powerpc.c | 6 ++++++ >> arch/s390/kvm/diag.c | 2 +- >> arch/s390/kvm/kvm-s390.c | 6 ++++++ >> arch/x86/include/asm/kvm_host.h | 5 +++++ >> arch/x86/kvm/hyperv.c | 2 +- >> arch/x86/kvm/svm.c | 10 +++++++++- >> arch/x86/kvm/vmx.c | 16 +++++++++++++++- >> arch/x86/kvm/x86.c | 11 +++++++++++ >> include/linux/kvm_host.h | 3 ++- >> virt/kvm/arm/arm.c | 5 +++++ >> virt/kvm/kvm_main.c | 4 +++- >> 14 files changed, 72 insertions(+), 8 deletions(-) >> > > I am curious, is there any architecture that allows to trigger > kvm_vcpu_on_spin(vcpu); while _not_ in kernel mode? IIUC, X86/SVM will trap to host due to PAUSE insn no matter the vcpu is in kernel-mode or user-mode. > > I would have guessed that user space should never be allowed to make cpu > wide decisions (giving up the CPU to the hypervisor). > > E.g. s390x diag can only be executed from kernel space. VMX PAUSE is > only valid from kernel space. X86/VMX has "PAUSE exiting" and "PAUSE-loop exiting"(PLE). KVM only uses PLE, this is as you said "only valid from kernel space" However, the "PAUSE exiting" can cause user-mode vcpu exit too. > > I.o.w. do we need a parameter to kvm_vcpu_on_spin(vcpu); at all, or is > "me_in_kernel" basically always true? > -- Regards, Longpeng(Mike)
[toc] | [prev] | [next] | [standalone]
| From | David Hildenbrand <david@redhat.com> |
|---|---|
| Date | 2017-08-08 14:00 +0200 |
| Message-ID | <ucbl0-7TS-35@gated-at.bofh.it> |
| In reply to | #1706415 |
On 08.08.2017 13:49, Longpeng (Mike) wrote: > > > On 2017/8/8 19:25, David Hildenbrand wrote: > >> On 08.08.2017 06:05, Longpeng(Mike) wrote: >>> This is a simple optimization for kvm_vcpu_on_spin, the >>> main idea is described in patch-1's commit msg. >>> >>> I did some tests base on the RFC version, the result shows >>> that it can improves the performance slightly. >>> >>> == Geekbench-3.4.1 == >>> VM1: 8U,4G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >>> running Geekbench-3.4.1 *10 truns* >>> VM2/VM3/VM4: configure is the same as VM1 >>> stress each vcpu usage(seed by top in guest) to 40% >>> >>> The comparison of each testcase's score: >>> (higher is better) >>> before after improve >>> Inter >>> single 1176.7 1179.0 0.2% >>> multi 3459.5 3426.5 -0.9% >>> Float >>> single 1150.5 1150.9 0.0% >>> multi 3364.5 3391.9 0.8% >>> Memory(stream) >>> single 1768.7 1773.1 0.2% >>> multi 2511.6 2557.2 1.8% >>> Overall >>> single 1284.2 1286.2 0.2% >>> multi 3231.4 3238.4 0.2% >>> >>> >>> == kernbench-0.42 == >>> VM1: 8U,12G, vcpu(0...7) is 1:1 pinned to pcpu(6...11,18,19) >>> running "kernbench -n 10" >>> VM2/VM3/VM4: configure is the same as VM1 >>> stress each vcpu usage(seed by top in guest) to 40% >>> >>> The comparison of 'Elapsed Time': >>> (sooner is better) >>> before after improve >>> load -j4 12.762 12.751 0.1% >>> load -j32 9.743 8.955 8.1% >>> load -j 9.688 9.229 4.7% >>> >>> >>> Physical Machine: >>> Architecture: x86_64 >>> CPU op-mode(s): 32-bit, 64-bit >>> Byte Order: Little Endian >>> CPU(s): 24 >>> On-line CPU(s) list: 0-23 >>> Thread(s) per core: 2 >>> Core(s) per socket: 6 >>> Socket(s): 2 >>> NUMA node(s): 2 >>> Vendor ID: GenuineIntel >>> CPU family: 6 >>> Model: 45 >>> Model name: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz >>> Stepping: 7 >>> CPU MHz: 2799.902 >>> BogoMIPS: 5004.67 >>> Virtualization: VT-x >>> L1d cache: 32K >>> L1i cache: 32K >>> L2 cache: 256K >>> L3 cache: 15360K >>> NUMA node0 CPU(s): 0-5,12-17 >>> NUMA node1 CPU(s): 6-11,18-23 >>> >>> --- >>> Changes since V1: >>> - split the implementation of s390 & arm. [David] >>> - refactor the impls according to the suggestion. [Paolo] >>> >>> Changes since RFC: >>> - only cache result for X86. [David & Cornlia & Paolo] >>> - add performance numbers. [David] >>> - impls arm/s390. [Christoffer & David] >>> - refactor the impls. [me] >>> >>> --- >>> Longpeng(Mike) (4): >>> KVM: add spinlock optimization framework >>> KVM: X86: implement the logic for spinlock optimization >>> KVM: s390: implements the kvm_arch_vcpu_in_kernel() >>> KVM: arm: implements the kvm_arch_vcpu_in_kernel() >>> >>> arch/arm/kvm/handle_exit.c | 2 +- >>> arch/arm64/kvm/handle_exit.c | 2 +- >>> arch/mips/kvm/mips.c | 6 ++++++ >>> arch/powerpc/kvm/powerpc.c | 6 ++++++ >>> arch/s390/kvm/diag.c | 2 +- >>> arch/s390/kvm/kvm-s390.c | 6 ++++++ >>> arch/x86/include/asm/kvm_host.h | 5 +++++ >>> arch/x86/kvm/hyperv.c | 2 +- >>> arch/x86/kvm/svm.c | 10 +++++++++- >>> arch/x86/kvm/vmx.c | 16 +++++++++++++++- >>> arch/x86/kvm/x86.c | 11 +++++++++++ >>> include/linux/kvm_host.h | 3 ++- >>> virt/kvm/arm/arm.c | 5 +++++ >>> virt/kvm/kvm_main.c | 4 +++- >>> 14 files changed, 72 insertions(+), 8 deletions(-) >>> >> >> I am curious, is there any architecture that allows to trigger >> kvm_vcpu_on_spin(vcpu); while _not_ in kernel mode? > > > IIUC, X86/SVM will trap to host due to PAUSE insn no matter the vcpu is in > kernel-mode or user-mode. > >> >> I would have guessed that user space should never be allowed to make cpu >> wide decisions (giving up the CPU to the hypervisor). >> >> E.g. s390x diag can only be executed from kernel space. VMX PAUSE is >> only valid from kernel space. > > > X86/VMX has "PAUSE exiting" and "PAUSE-loop exiting"(PLE). KVM only uses PLE, > this is as you said "only valid from kernel space" > > However, the "PAUSE exiting" can cause user-mode vcpu exit too. Thanks Longpeng and Christoffer! -- Thanks, David
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web