Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1708393 > unrolled thread

[PATCH] KVM/x86: Increase max vcpu number to 352

Back to article view | Back to linux.kernel

Contents

  [PATCH] KVM/x86: Increase max vcpu number to 352 Lan Tianyu <tianyu.lan@intel.com> - 2017-08-10 12:10 +0200
    Re: [PATCH] KVM/x86: Increase max vcpu number to 352 Lan Tianyu <tianyu.lan@intel.com> - 2017-08-11 10:00 +0200
      Re: [PATCH] KVM/x86: Increase max vcpu number to 352 David Hildenbrand <david@redhat.com> - 2017-08-11 10:20 +0200
        Re: [PATCH] KVM/x86: Increase max vcpu number to 352 Radim Krčmář <rkrcmar@redhat.com> - 2017-08-11 15:10 +0200
          Re: [PATCH] KVM/x86: Increase max vcpu number to 352 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2017-08-11 21:40 +0200

#1708393 — [PATCH] KVM/x86: Increase max vcpu number to 352

Intel Xeon phi chip will support 352 logical threads. For HPC usage
case, it will create a huge VM with vcpu number as same as host cpus. This
patch is to increase max vcpu number to 352.

Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 arch/x86/include/asm/kvm_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 87ac4fb..2cdc043 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -35,7 +35,7 @@
 #include <asm/asm.h>
 #include <asm/kvm_page_track.h>
 
-#define KVM_MAX_VCPUS 288
+#define KVM_MAX_VCPUS 352
 #define KVM_SOFT_MAX_VCPUS 240
 #define KVM_MAX_VCPU_ID 1023
 #define KVM_USER_MEM_SLOTS 509
-- 
1.8.3.1

[toc] | [next] | [standalone]

#1709346

Hi Konrad:
	Thanks for your review.

On 2017年08月11日 01:50, Konrad Rzeszutek Wilk wrote:
> On Thu, Aug 10, 2017 at 06:00:59PM +0800, Lan Tianyu wrote:
>> Intel Xeon phi chip will support 352 logical threads. For HPC usage
>> case, it will create a huge VM with vcpu number as same as host cpus. This
>> patch is to increase max vcpu number to 352.
> 
> Why not 1024 or 4096?

This is on demand. We can set a higher number since KVM already has
x2apic and vIOMMU interrupt remapping support.

> 
> Are there any issues with increasing the value from 288 to 352 right now?

No found.

> 
> Also perhaps this should be made in an Kconfig entry?

That will be anther option but I find different platforms will define
different MAX_VCPU. If we introduce a generic Kconfig entry, different
platforms should have different range.

Radim & Paolo, Could you give some input? In qemu thread, we will set
max vcpu to 8192 for x86 VM. In KVM, The length of vcpu pointer array in
struct kvm and dest_vcpu_bitmap in kvm_irq_delivery_to_apic() are
specified by KVM_MAX_VCPUS. Should we keep align with Qemu?

-- 
Best regards
Tianyu Lan

[toc] | [prev] | [next] | [standalone]

#1709364

On 11.08.2017 09:49, Lan Tianyu wrote:
> Hi Konrad:
> 	Thanks for your review.
> 
> On 2017年08月11日 01:50, Konrad Rzeszutek Wilk wrote:
>> On Thu, Aug 10, 2017 at 06:00:59PM +0800, Lan Tianyu wrote:
>>> Intel Xeon phi chip will support 352 logical threads. For HPC usage
>>> case, it will create a huge VM with vcpu number as same as host cpus. This
>>> patch is to increase max vcpu number to 352.
>>
>> Why not 1024 or 4096?
> 
> This is on demand. We can set a higher number since KVM already has
> x2apic and vIOMMU interrupt remapping support.
> 
>>
>> Are there any issues with increasing the value from 288 to 352 right now?
> 
> No found.
> 
>>
>> Also perhaps this should be made in an Kconfig entry?
> 
> That will be anther option but I find different platforms will define
> different MAX_VCPU. If we introduce a generic Kconfig entry, different
> platforms should have different range.
> 
> Radim & Paolo, Could you give some input? In qemu thread, we will set
> max vcpu to 8192 for x86 VM. In KVM, The length of vcpu pointer array in
> struct kvm and dest_vcpu_bitmap in kvm_irq_delivery_to_apic() are
> specified by KVM_MAX_VCPUS. Should we keep align with Qemu?
> 

commit 682f732ecf7396e9d6fe24d44738966699fae6c0
Author: Radim Krčmář <rkrcmar@redhat.com>
Date:   Tue Jul 12 22:09:29 2016 +0200

    KVM: x86: bump MAX_VCPUS to 288

    288 is in high demand because of Knights Landing CPU.
    We cannot set the limit to 640k, because that would be wasting space.

I think we want to keep it small as long as possible. I remember a patch
series from Radim which would dynamically allocate memory for these
arrays (using a new VM creation ioctl, specifying the max # of vcpus).
Wonder what happened to that (I remember requesting a simply remalloc
instead of a new VM creation ioctl :] ).

-- 

Thanks,

David

[toc] | [prev] | [next] | [standalone]

#1709603

2017-08-11 10:11+0200, David Hildenbrand:
> On 11.08.2017 09:49, Lan Tianyu wrote:
>> Hi Konrad:
>> 	Thanks for your review.
>> 
>> On 2017年08月11日 01:50, Konrad Rzeszutek Wilk wrote:
>>> On Thu, Aug 10, 2017 at 06:00:59PM +0800, Lan Tianyu wrote:
>>>> Intel Xeon phi chip will support 352 logical threads. For HPC usage
>>>> case, it will create a huge VM with vcpu number as same as host cpus. This
>>>> patch is to increase max vcpu number to 352.
>>>
>>> Why not 1024 or 4096?
>> 
>> This is on demand. We can set a higher number since KVM already has
>> x2apic and vIOMMU interrupt remapping support.
>> 
>>>
>>> Are there any issues with increasing the value from 288 to 352 right now?
>> 
>> No found.

Yeah, the only issue until around 2^20 (when we reach the maximum of
logical x2APIC addressing) should be the size of per-VM arrays when only
few VCPUs are going to be used.

>>> Also perhaps this should be made in an Kconfig entry?
>> 
>> That will be anther option but I find different platforms will define
>> different MAX_VCPU. If we introduce a generic Kconfig entry, different
>> platforms should have different range.
>> 
>> Radim & Paolo, Could you give some input? In qemu thread, we will set
>> max vcpu to 8192 for x86 VM. In KVM, The length of vcpu pointer array in
>> struct kvm and dest_vcpu_bitmap in kvm_irq_delivery_to_apic() are
>> specified by KVM_MAX_VCPUS. Should we keep align with Qemu?

That would be great.

> commit 682f732ecf7396e9d6fe24d44738966699fae6c0
> Author: Radim Krčmář <rkrcmar@redhat.com>
> Date:   Tue Jul 12 22:09:29 2016 +0200
> 
>     KVM: x86: bump MAX_VCPUS to 288
> 
>     288 is in high demand because of Knights Landing CPU.
>     We cannot set the limit to 640k, because that would be wasting space.
> 
> I think we want to keep it small as long as possible. I remember a patch
> series from Radim which would dynamically allocate memory for these
> arrays (using a new VM creation ioctl, specifying the max # of vcpus).
> Wonder what happened to that (I remember requesting a simply remalloc
> instead of a new VM creation ioctl :] ).

Eh, I forgot about them ...  I didn't like the dynamic allocation as we
would need to protect the memory, which would result in a much bigger
changeset, or fragile macros.

I can't recall the disgust now, so I'll send a RFC with the dynamic
version to see how it turned out.

Thanks.

[toc] | [prev] | [next] | [standalone]

#1709942

On Fri, Aug 11, 2017 at 03:00:20PM +0200, Radim Krčmář wrote:
> 2017-08-11 10:11+0200, David Hildenbrand:
> > On 11.08.2017 09:49, Lan Tianyu wrote:
> >> Hi Konrad:
> >> 	Thanks for your review.
> >> 
> >> On 2017年08月11日 01:50, Konrad Rzeszutek Wilk wrote:
> >>> On Thu, Aug 10, 2017 at 06:00:59PM +0800, Lan Tianyu wrote:
> >>>> Intel Xeon phi chip will support 352 logical threads. For HPC usage
> >>>> case, it will create a huge VM with vcpu number as same as host cpus. This
> >>>> patch is to increase max vcpu number to 352.
> >>>
> >>> Why not 1024 or 4096?
> >> 
> >> This is on demand. We can set a higher number since KVM already has
> >> x2apic and vIOMMU interrupt remapping support.
> >> 
> >>>
> >>> Are there any issues with increasing the value from 288 to 352 right now?
> >> 
> >> No found.
> 
> Yeah, the only issue until around 2^20 (when we reach the maximum of
> logical x2APIC addressing) should be the size of per-VM arrays when only
> few VCPUs are going to be used.

Migration with 352 CPUs all being busy dirtying memory and also poking
at various I/O ports (say all of them dirtying the VGA) is no problem?


> 
> >>> Also perhaps this should be made in an Kconfig entry?
> >> 
> >> That will be anther option but I find different platforms will define
> >> different MAX_VCPU. If we introduce a generic Kconfig entry, different
> >> platforms should have different range.


By different platforms you mean q35 vs the older one, and such?
Not whether the underlaying accelerator is tcg, Xen, KVM, or bHyve?

What I was trying to understand whether it makes even sense for
the platforms to have such limits in the first place - and instead the
accelerators should be the ones setting it?


> >> 
> >> Radim & Paolo, Could you give some input? In qemu thread, we will set
> >> max vcpu to 8192 for x86 VM. In KVM, The length of vcpu pointer array in
> >> struct kvm and dest_vcpu_bitmap in kvm_irq_delivery_to_apic() are
> >> specified by KVM_MAX_VCPUS. Should we keep align with Qemu?
> 
> That would be great.
> 
> > commit 682f732ecf7396e9d6fe24d44738966699fae6c0
> > Author: Radim Krčmář <rkrcmar@redhat.com>
> > Date:   Tue Jul 12 22:09:29 2016 +0200
> > 
> >     KVM: x86: bump MAX_VCPUS to 288
> > 
> >     288 is in high demand because of Knights Landing CPU.
> >     We cannot set the limit to 640k, because that would be wasting space.
> > 
> > I think we want to keep it small as long as possible. I remember a patch
> > series from Radim which would dynamically allocate memory for these
> > arrays (using a new VM creation ioctl, specifying the max # of vcpus).
> > Wonder what happened to that (I remember requesting a simply remalloc
> > instead of a new VM creation ioctl :] ).
> 
> Eh, I forgot about them ...  I didn't like the dynamic allocation as we
> would need to protect the memory, which would result in a much bigger
> changeset, or fragile macros.
> 
> I can't recall the disgust now, so I'll send a RFC with the dynamic
> version to see how it turned out.
> 
> Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web