Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1532915 > unrolled thread

[PATCH v3 0/5] VMX Capability MSRs

Started byDavid Matlack <dmatlack@google.com>
First post2016-11-30 03:20 +0100
Last post2016-11-30 19:10 +0100
Articles 5 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v3 0/5] VMX Capability MSRs David Matlack <dmatlack@google.com> - 2016-11-30 03:20 +0100
    [PATCH v3 4/5] KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID David Matlack <dmatlack@google.com> - 2016-11-30 03:20 +0100
    [PATCH v3 5/5] KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry David Matlack <dmatlack@google.com> - 2016-11-30 03:20 +0100
    Re: [PATCH v3 0/5] VMX Capability MSRs Paolo Bonzini <pbonzini@redhat.com> - 2016-11-30 12:30 +0100
      Re: [PATCH v3 0/5] VMX Capability MSRs David Matlack <dmatlack@google.com> - 2016-11-30 19:10 +0100

#1532915 — [PATCH v3 0/5] VMX Capability MSRs

FromDavid Matlack <dmatlack@google.com>
Date2016-11-30 03:20 +0100
Subject[PATCH v3 0/5] VMX Capability MSRs
Message-ID<sJ2F3-8jd-3@gated-at.bofh.it>
This patchset adds support setting the VMX capability MSRs from userspace.
This is required for migration of nested-capable VMs to different CPUs and
KVM versions.

Patch 1 generates the non-true VMX MSRs using the true MSRs, which allows
userspace to skip restoring them.

Patch 2 adds support for restoring the VMX capability MSRs.

Patches 3 and 4 make KVM's emulation of MSR_IA32_VMX_CR{0,4}_FIXED1 more
accurate.

Patch 5 fixes a bug in emulated VM-entry that came up when testing patches
3 and 4.

Changes since v2:
  * Generate CR0_FIXED1 in addition to CR4_FIXED1
  * Generate "non-true" capability MSRs from the "true" versions and remove
    "non-true" MSRs from struct nested_vmx.
  * Disallow restore of CR{0,4}_FIXED1 and "non-true" MSRs since they are
    generated.

Changes since v1:
  * Support restoring less-capable versions of MSR_IA32_VMX_BASIC,
    MSR_IA32_VMX_CR{0,4}_FIXED{0,1}.
  * Include VMX_INS_OUTS in MSR_IA32_VMX_BASIC initial value.

David Matlack (5):
  KVM: nVMX: generate non-true VMX MSRs based on true versions
  KVM: nVMX: support restore of VMX capability MSRs
  KVM: nVMX: fix checks on CR{0,4} during virtual VMX operation
  KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID
  KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry

 arch/x86/include/asm/vmx.h |  31 +++
 arch/x86/kvm/vmx.c         | 479 +++++++++++++++++++++++++++++++++++++--------
 2 files changed, 427 insertions(+), 83 deletions(-)

-- 
2.8.0.rc3.226.g39d4020

[toc] | [next] | [standalone]

#1532916 — [PATCH v3 4/5] KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID

FromDavid Matlack <dmatlack@google.com>
Date2016-11-30 03:20 +0100
Subject[PATCH v3 4/5] KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID
Message-ID<sJ2F4-8jd-15@gated-at.bofh.it>
In reply to#1532915
MSR_IA32_CR{0,4}_FIXED1 define which bits in CR0 and CR4 are allowed to
be 1 during VMX operation. Since the set of allowed-1 bits is the same
in and out of VMX operation, we can generate these MSRs entirely from
the guest's CPUID. This lets userspace avoiding having to save/restore
these MSRs.

This patch also initializes MSR_IA32_CR{0,4}_FIXED1 from the CPU's MSRs
by default. This is a saner than the current default of -1ull, which
includes bits that the host CPU does not support.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 arch/x86/kvm/vmx.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 52 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index b414513..49270c4 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2849,16 +2849,18 @@ static void nested_vmx_setup_ctls_msrs(struct vcpu_vmx *vmx)
 		vmx->nested.nested_vmx_basic |= VMX_BASIC_INOUT;
 
 	/*
-	 * These MSRs specify bits which the guest must keep fixed (on or off)
+	 * These MSRs specify bits which the guest must keep fixed on
 	 * while L1 is in VMXON mode (in L1's root mode, or running an L2).
 	 * We picked the standard core2 setting.
 	 */
 #define VMXON_CR0_ALWAYSON     (X86_CR0_PE | X86_CR0_PG | X86_CR0_NE)
 #define VMXON_CR4_ALWAYSON     X86_CR4_VMXE
 	vmx->nested.nested_vmx_cr0_fixed0 = VMXON_CR0_ALWAYSON;
-	vmx->nested.nested_vmx_cr0_fixed1 = -1ULL;
 	vmx->nested.nested_vmx_cr4_fixed0 = VMXON_CR4_ALWAYSON;
-	vmx->nested.nested_vmx_cr4_fixed1 = -1ULL;
+
+	/* These MSRs specify bits which the guest must keep fixed off. */
+	rdmsrl(MSR_IA32_VMX_CR0_FIXED1, vmx->nested.nested_vmx_cr0_fixed1);
+	rdmsrl(MSR_IA32_VMX_CR4_FIXED1, vmx->nested.nested_vmx_cr4_fixed1);
 
 	/* highest index: VMX_PREEMPTION_TIMER_VALUE */
 	vmx->nested.nested_vmx_vmcs_enum = 0x2e;
@@ -9547,6 +9549,50 @@ static void vmcs_set_secondary_exec_control(u32 new_ctl)
 		     (new_ctl & ~mask) | (cur_ctl & mask));
 }
 
+/*
+ * Generate MSR_IA32_VMX_CR{0,4}_FIXED1 according to CPUID. Only set bits
+ * (indicating "allowed-1") if they are supported in the guest's CPUID.
+ */
+static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct kvm_cpuid_entry2 *entry;
+
+	vmx->nested.nested_vmx_cr0_fixed1 = 0xffffffff;
+	vmx->nested.nested_vmx_cr4_fixed1 = X86_CR4_PCE;
+
+#define cr4_fixed1_update(_cr4_mask, _reg, _cpuid_mask) do {		\
+	if (entry && (entry->_reg & (_cpuid_mask)))			\
+		vmx->nested.nested_vmx_cr4_fixed1 |= (_cr4_mask);	\
+} while (0)
+
+	entry = kvm_find_cpuid_entry(vcpu, 0x1, 0);
+	cr4_fixed1_update(X86_CR4_VME,        edx, bit(X86_FEATURE_VME));
+	cr4_fixed1_update(X86_CR4_PVI,        edx, bit(X86_FEATURE_VME));
+	cr4_fixed1_update(X86_CR4_TSD,        edx, bit(X86_FEATURE_TSC));
+	cr4_fixed1_update(X86_CR4_DE,         edx, bit(X86_FEATURE_DE));
+	cr4_fixed1_update(X86_CR4_PSE,        edx, bit(X86_FEATURE_PSE));
+	cr4_fixed1_update(X86_CR4_PAE,        edx, bit(X86_FEATURE_PAE));
+	cr4_fixed1_update(X86_CR4_MCE,        edx, bit(X86_FEATURE_MCE));
+	cr4_fixed1_update(X86_CR4_PGE,        edx, bit(X86_FEATURE_PGE));
+	cr4_fixed1_update(X86_CR4_OSFXSR,     edx, bit(X86_FEATURE_FXSR));
+	cr4_fixed1_update(X86_CR4_OSXMMEXCPT, edx, bit(X86_FEATURE_XMM));
+	cr4_fixed1_update(X86_CR4_VMXE,       ecx, bit(X86_FEATURE_VMX));
+	cr4_fixed1_update(X86_CR4_SMXE,       ecx, bit(X86_FEATURE_SMX));
+	cr4_fixed1_update(X86_CR4_PCIDE,      ecx, bit(X86_FEATURE_PCID));
+	cr4_fixed1_update(X86_CR4_OSXSAVE,    ecx, bit(X86_FEATURE_XSAVE));
+
+	entry = kvm_find_cpuid_entry(vcpu, 0x7, 0);
+	cr4_fixed1_update(X86_CR4_FSGSBASE,   ebx, bit(X86_FEATURE_FSGSBASE));
+	cr4_fixed1_update(X86_CR4_SMEP,       ebx, bit(X86_FEATURE_SMEP));
+	cr4_fixed1_update(X86_CR4_SMAP,       ebx, bit(X86_FEATURE_SMAP));
+	cr4_fixed1_update(X86_CR4_PKE,        ecx, bit(X86_FEATURE_PKU));
+	/* TODO: Use X86_CR4_UMIP and X86_FEATURE_UMIP macros */
+	cr4_fixed1_update(bit(11),            ecx, bit(2));
+
+#undef cr4_fixed1_update
+}
+
 static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpuid_entry2 *best;
@@ -9588,6 +9634,9 @@ static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 	else
 		to_vmx(vcpu)->msr_ia32_feature_control_valid_bits &=
 			~FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX;
+
+	if (nested_vmx_allowed(vcpu))
+		nested_vmx_cr_fixed1_bits_update(vcpu);
 }
 
 static void vmx_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)
-- 
2.8.0.rc3.226.g39d4020

[toc] | [prev] | [next] | [standalone]

#1532917 — [PATCH v3 5/5] KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry

FromDavid Matlack <dmatlack@google.com>
Date2016-11-30 03:20 +0100
Subject[PATCH v3 5/5] KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry
Message-ID<sJ2F4-8jd-13@gated-at.bofh.it>
In reply to#1532915
vmx_set_cr0() modifies GUEST_EFER and "IA-32e mode guest" in the current
VMCS. Call vmx_set_efer() after vmx_set_cr0() so that emulated VM-entry
is more faithful to VMCS12.

This patch correctly causes VM-entry to fail when "IA-32e mode guest" is
1 and GUEST_CR0.PG is 0. Previously this configuration would succeed and
"IA-32e mode guest" would silently be disabled by KVM.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 arch/x86/kvm/vmx.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 49270c4..776dc67 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -10386,15 +10386,6 @@ static void prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
 		nested_ept_init_mmu_context(vcpu);
 	}
 
-	if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_EFER)
-		vcpu->arch.efer = vmcs12->guest_ia32_efer;
-	else if (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE)
-		vcpu->arch.efer |= (EFER_LMA | EFER_LME);
-	else
-		vcpu->arch.efer &= ~(EFER_LMA | EFER_LME);
-	/* Note: modifies VM_ENTRY/EXIT_CONTROLS and GUEST/HOST_IA32_EFER */
-	vmx_set_efer(vcpu, vcpu->arch.efer);
-
 	/*
 	 * This sets GUEST_CR0 to vmcs12->guest_cr0, with possibly a modified
 	 * TS bit (for lazy fpu) and bits which we consider mandatory enabled.
@@ -10409,6 +10400,15 @@ static void prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
 	vmx_set_cr4(vcpu, vmcs12->guest_cr4);
 	vmcs_writel(CR4_READ_SHADOW, nested_read_cr4(vmcs12));
 
+	if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_EFER)
+		vcpu->arch.efer = vmcs12->guest_ia32_efer;
+	else if (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE)
+		vcpu->arch.efer |= (EFER_LMA | EFER_LME);
+	else
+		vcpu->arch.efer &= ~(EFER_LMA | EFER_LME);
+	/* Note: modifies VM_ENTRY/EXIT_CONTROLS and GUEST/HOST_IA32_EFER */
+	vmx_set_efer(vcpu, vcpu->arch.efer);
+
 	/* shadow page tables on either EPT or shadow page tables */
 	kvm_set_cr3(vcpu, vmcs12->guest_cr3);
 	kvm_mmu_reset_context(vcpu);
-- 
2.8.0.rc3.226.g39d4020

[toc] | [prev] | [next] | [standalone]

#1533236

FromPaolo Bonzini <pbonzini@redhat.com>
Date2016-11-30 12:30 +0100
Message-ID<sJbfj-5rV-9@gated-at.bofh.it>
In reply to#1532915

On 30/11/2016 03:14, David Matlack wrote:
> This patchset adds support setting the VMX capability MSRs from userspace.
> This is required for migration of nested-capable VMs to different CPUs and
> KVM versions.
> 
> Patch 1 generates the non-true VMX MSRs using the true MSRs, which allows
> userspace to skip restoring them.
> 
> Patch 2 adds support for restoring the VMX capability MSRs.
> 
> Patches 3 and 4 make KVM's emulation of MSR_IA32_VMX_CR{0,4}_FIXED1 more
> accurate.
> 
> Patch 5 fixes a bug in emulated VM-entry that came up when testing patches
> 3 and 4.
> 
> Changes since v2:
>   * Generate CR0_FIXED1 in addition to CR4_FIXED1
>   * Generate "non-true" capability MSRs from the "true" versions and remove
>     "non-true" MSRs from struct nested_vmx.
>   * Disallow restore of CR{0,4}_FIXED1 and "non-true" MSRs since they are
>     generated.
> 
> Changes since v1:
>   * Support restoring less-capable versions of MSR_IA32_VMX_BASIC,
>     MSR_IA32_VMX_CR{0,4}_FIXED{0,1}.
>   * Include VMX_INS_OUTS in MSR_IA32_VMX_BASIC initial value.
> 
> David Matlack (5):
>   KVM: nVMX: generate non-true VMX MSRs based on true versions
>   KVM: nVMX: support restore of VMX capability MSRs
>   KVM: nVMX: fix checks on CR{0,4} during virtual VMX operation
>   KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID
>   KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry
> 
>  arch/x86/include/asm/vmx.h |  31 +++
>  arch/x86/kvm/vmx.c         | 479 +++++++++++++++++++++++++++++++++++++--------
>  2 files changed, 427 insertions(+), 83 deletions(-)

Just a small nit that can be fixed on applying.  Thanks!

Paolo

[toc] | [prev] | [next] | [standalone]

#1533478

FromDavid Matlack <dmatlack@google.com>
Date2016-11-30 19:10 +0100
Message-ID<sJhup-10J-7@gated-at.bofh.it>
In reply to#1533236
On Wed, Nov 30, 2016 at 3:22 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 30/11/2016 03:14, David Matlack wrote:
>> This patchset adds support setting the VMX capability MSRs from userspace.
>> This is required for migration of nested-capable VMs to different CPUs and
>> KVM versions.
>>
>> Patch 1 generates the non-true VMX MSRs using the true MSRs, which allows
>> userspace to skip restoring them.
>>
>> Patch 2 adds support for restoring the VMX capability MSRs.
>>
>> Patches 3 and 4 make KVM's emulation of MSR_IA32_VMX_CR{0,4}_FIXED1 more
>> accurate.
>>
>> Patch 5 fixes a bug in emulated VM-entry that came up when testing patches
>> 3 and 4.
>>
>> Changes since v2:
>>   * Generate CR0_FIXED1 in addition to CR4_FIXED1
>>   * Generate "non-true" capability MSRs from the "true" versions and remove
>>     "non-true" MSRs from struct nested_vmx.
>>   * Disallow restore of CR{0,4}_FIXED1 and "non-true" MSRs since they are
>>     generated.
>>
>> Changes since v1:
>>   * Support restoring less-capable versions of MSR_IA32_VMX_BASIC,
>>     MSR_IA32_VMX_CR{0,4}_FIXED{0,1}.
>>   * Include VMX_INS_OUTS in MSR_IA32_VMX_BASIC initial value.
>>
>> David Matlack (5):
>>   KVM: nVMX: generate non-true VMX MSRs based on true versions
>>   KVM: nVMX: support restore of VMX capability MSRs
>>   KVM: nVMX: fix checks on CR{0,4} during virtual VMX operation
>>   KVM: nVMX: generate MSR_IA32_CR{0,4}_FIXED1 from guest CPUID
>>   KVM: nVMX: load GUEST_EFER after GUEST_CR0 during emulated VM-entry
>>
>>  arch/x86/include/asm/vmx.h |  31 +++
>>  arch/x86/kvm/vmx.c         | 479 +++++++++++++++++++++++++++++++++++++--------
>>  2 files changed, 427 insertions(+), 83 deletions(-)
>
> Just a small nit that can be fixed on applying.  Thanks!

Thanks for the thorough review!

>
> Paolo

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web