Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1315292 > unrolled thread

[PATCH v1 00/12] HVMlite domU support

Back to article view | Back to linux.kernel

Contents

  [PATCH v1 00/12] HVMlite domU support Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 11/12] xen/hvmlite: Boot secondary CPUs Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 08/12] xen/hvmlite: Initialize context for secondary VCPUs Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 12/12] xen/hvmlite: Enable CPU on-/offlining Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 03/12] xen/hvmlite: Import hvmlite-related Xen public interfaces Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 05/12] xen/hvmlite: Allow HVMlite guests delay initializing grant table Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    [PATCH v1 07/12] xen/hvmlite: Prepare cpu_initialize_context() routine for HVMlite SMP Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    Re: [PATCH v1 00/12] HVMlite domU support David Vrabel <david.vrabel@citrix.com> - 2016-01-25 12:00 +0100
      Re: [PATCH v1 00/12] HVMlite domU support Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-25 16:30 +0100

#1315292 — [PATCH v1 00/12] HVMlite domU support

This series introduces HVMlite support for unprivileged guests.

It has been tested on Intel/AMD, both 32- and 64-bit, including CPU on- and
offlining and save/restore. (Restore will result in APIC write warnings
which exist now for 32-bit PV guests as well so I didn't address this in
this series)

Compile-tested on ARM

Boris Ostrovsky (12):
  x86/smp: Make start_secondary() and initial_pg_pmd visible globally
  xen/hvmlite: Factor out common kernel init code
  xen/hvmlite: Import hvmlite-related Xen public interfaces
  xen/hvmlite: Bootstrap HVMlite guest
  xen/hvmlite: Allow HVMlite guests delay initializing grant table
  xen/hvmlite: Initialize PCI
  xen/hvmlite: Prepare cpu_initialize_context() routine for HVMlite SMP
  xen/hvmlite: Initialize context for secondary VCPUs
  xen/hvmlite: Extend APIC operations for HVMlite guests
  xen/hvmlite: Use x86's default timer init for HVMlite guests
  xen/hvmlite: Boot secondary CPUs
  xen/hvmlite: Enable CPU on-/offlining

 arch/x86/include/asm/smp.h           |    1 +
 arch/x86/kernel/head_32.S            |    2 +-
 arch/x86/kernel/smpboot.c            |    2 +-
 arch/x86/pci/xen.c                   |    2 +-
 arch/x86/xen/Makefile                |    1 +
 arch/x86/xen/apic.c                  |   39 ++++-
 arch/x86/xen/enlighten.c             |  318 ++++++++++++++++++++++------------
 arch/x86/xen/grant-table.c           |    4 +-
 arch/x86/xen/platform-pci-unplug.c   |    4 +-
 arch/x86/xen/pmu.c                   |    4 +-
 arch/x86/xen/smp.c                   |  248 ++++++++++++++++++--------
 arch/x86/xen/smp.h                   |    4 +
 arch/x86/xen/time.c                  |    5 +-
 arch/x86/xen/xen-hvmlite.S           |  173 ++++++++++++++++++
 drivers/xen/grant-table.c            |    8 +-
 include/xen/interface/elfnote.h      |   12 ++-
 include/xen/interface/hvm/hvm_vcpu.h |  143 +++++++++++++++
 include/xen/interface/xen.h          |   24 +++
 include/xen/xen.h                    |    6 +
 19 files changed, 801 insertions(+), 199 deletions(-)
 create mode 100644 arch/x86/xen/xen-hvmlite.S
 create mode 100644 include/xen/interface/hvm/hvm_vcpu.h

[toc] | [next] | [standalone]

#1315293 — [PATCH v1 11/12] xen/hvmlite: Boot secondary CPUs

HVMlite secondary VCPUs use baremetal bringup path (i.e. native_*
smp_ops) but need to do some preparation in PV code.

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/enlighten.c |    2 +
 arch/x86/xen/pmu.c       |    4 +-
 arch/x86/xen/smp.c       |   60 +++++++++++++++++++++++++++++++++-------------
 3 files changed, 47 insertions(+), 19 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 2ed8b2b..850ce66 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1937,6 +1937,8 @@ static void __init xen_hvm_guest_init(void)
 		xen_have_vector_callback = 1;
 	xen_hvm_smp_init();
 	register_cpu_notifier(&xen_hvm_cpu_notifier);
+	if (xen_hvmlite)
+		smp_found_config = 1;
 	xen_unplug_emulated_devices();
 	x86_init.irqs.intr_init = xen_init_IRQ;
 	xen_hvm_init_time_ops();
diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c
index 724a087..7bc209b 100644
--- a/arch/x86/xen/pmu.c
+++ b/arch/x86/xen/pmu.c
@@ -518,7 +518,7 @@ void xen_pmu_init(int cpu)
 
 	BUILD_BUG_ON(sizeof(struct xen_pmu_data) > PAGE_SIZE);
 
-	if (xen_hvm_domain())
+	if (xen_hvm_domain() && !xen_hvmlite)
 		return;
 
 	xenpmu_data = (struct xen_pmu_data *)get_zeroed_page(GFP_KERNEL);
@@ -556,7 +556,7 @@ void xen_pmu_finish(int cpu)
 {
 	struct xen_pmu_params xp;
 
-	if (xen_hvm_domain())
+	if (xen_hvm_domain() && !xen_hvmlite)
 		return;
 
 	xp.vcpu = cpu;
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index fb085ef..fbad829 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -348,26 +348,31 @@ static void __init xen_smp_prepare_cpus(unsigned int max_cpus)
 	}
 	xen_init_lock_cpu(0);
 
-	smp_store_boot_cpu_info();
-	cpu_data(0).x86_max_cores = 1;
+	if (!xen_hvmlite) {
+		smp_store_boot_cpu_info();
+		cpu_data(0).x86_max_cores = 1;
+
+		for_each_possible_cpu(i) {
+			zalloc_cpumask_var(&per_cpu(cpu_sibling_map, i),
+					   GFP_KERNEL);
+			zalloc_cpumask_var(&per_cpu(cpu_core_map, i),
+					   GFP_KERNEL);
+			zalloc_cpumask_var(&per_cpu(cpu_llc_shared_map, i),
+					   GFP_KERNEL);
+		}
+		set_cpu_sibling_map(0);
 
-	for_each_possible_cpu(i) {
-		zalloc_cpumask_var(&per_cpu(cpu_sibling_map, i), GFP_KERNEL);
-		zalloc_cpumask_var(&per_cpu(cpu_core_map, i), GFP_KERNEL);
-		zalloc_cpumask_var(&per_cpu(cpu_llc_shared_map, i), GFP_KERNEL);
+		if (!alloc_cpumask_var(&xen_cpu_initialized_map, GFP_KERNEL))
+			panic("could not allocate xen_cpu_initialized_map\n");
+
+		cpumask_copy(xen_cpu_initialized_map, cpumask_of(0));
 	}
-	set_cpu_sibling_map(0);
 
 	xen_pmu_init(0);
 
 	if (xen_smp_intr_init(0))
 		BUG();
 
-	if (!alloc_cpumask_var(&xen_cpu_initialized_map, GFP_KERNEL))
-		panic("could not allocate xen_cpu_initialized_map\n");
-
-	cpumask_copy(xen_cpu_initialized_map, cpumask_of(0));
-
 	/* Restrict the possible_map according to max_cpus. */
 	while ((num_possible_cpus() > 1) && (num_possible_cpus() > max_cpus)) {
 		for (cpu = nr_cpu_ids - 1; !cpu_possible(cpu); cpu--)
@@ -375,8 +380,11 @@ static void __init xen_smp_prepare_cpus(unsigned int max_cpus)
 		set_cpu_possible(cpu, false);
 	}
 
-	for_each_possible_cpu(cpu)
+	for_each_possible_cpu(cpu) {
 		set_cpu_present(cpu, true);
+		if (xen_hvmlite)
+			physid_set(cpu, phys_cpu_present_map);
+	}
 }
 
 static int
@@ -810,10 +818,15 @@ void __init xen_smp_init(void)
 
 static void __init xen_hvm_smp_prepare_cpus(unsigned int max_cpus)
 {
+	if (xen_hvmlite)
+		xen_smp_prepare_cpus(max_cpus);
+
 	native_smp_prepare_cpus(max_cpus);
-	WARN_ON(xen_smp_intr_init(0));
 
-	xen_init_lock_cpu(0);
+	if (!xen_hvmlite) {
+		WARN_ON(xen_smp_intr_init(0));
+		xen_init_lock_cpu(0);
+	}
 }
 
 static int xen_hvm_cpu_up(unsigned int cpu, struct task_struct *tidle)
@@ -836,8 +849,21 @@ static int xen_hvm_cpu_up(unsigned int cpu, struct task_struct *tidle)
 	*/
 	rc = xen_smp_intr_init(cpu);
 	WARN_ON(rc);
-	if (!rc)
-		rc =  native_cpu_up(cpu, tidle);
+
+	if (xen_hvmlite) {
+		rc = cpu_initialize_context(cpu, tidle);
+		if (rc) {
+			xen_smp_intr_free(cpu);
+			return rc;
+		}
+		xen_pmu_init(cpu);
+	}
+
+	if (!rc) {
+		rc = native_cpu_up(cpu, tidle);
+		if (rc && xen_hvmlite)
+			xen_pmu_finish(cpu);
+	}
 
 	/*
 	 * We must initialize the slowpath CPU kicker _after_ the native
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1315294 — [PATCH v1 08/12] xen/hvmlite: Initialize context for secondary VCPUs

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/smp.c         |   57 ++++++++++++++++++++++++++++++++++++++++----
 arch/x86/xen/smp.h         |    4 +++
 arch/x86/xen/xen-hvmlite.S |    7 +++++
 3 files changed, 63 insertions(+), 5 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 5fc4afb..b265c4f 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -27,6 +27,7 @@
 #include <xen/interface/xen.h>
 #include <xen/interface/vcpu.h>
 #include <xen/interface/xenpmu.h>
+#include <xen/interface/hvm/hvm_vcpu.h>
 
 #include <asm/xen/interface.h>
 #include <asm/xen/hypercall.h>
@@ -384,6 +385,7 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)
 	struct vcpu_guest_context *ctxt;
 	struct desc_struct *gdt;
 	unsigned long gdt_mfn;
+	void *ctxt_arg;
 
 	/* used to tell cpu_init() that it can proceed with initialization */
 	cpumask_set_cpu(cpu, cpu_callout_mask);
@@ -392,7 +394,7 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)
 
 	if (!xen_hvmlite) {
 
-		ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
+		ctxt_arg = ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
 		if (ctxt == NULL)
 			return -ENOMEM;
 
@@ -460,14 +462,59 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)
 		ctxt->user_regs.esp = idle->thread.sp0 - sizeof(struct pt_regs);
 		ctxt->ctrlreg[3] = xen_pfn_to_cr3(virt_to_gfn(swapper_pg_dir));
 	} else {
-		ctxt = NULL; /* To quiet down compiler */
-		BUG();
+#ifdef CONFIG_XEN_PVHVM
+		struct vcpu_hvm_context *hctxt;
+
+		ctxt_arg = hctxt = kzalloc(sizeof(*hctxt), GFP_KERNEL);
+		if (hctxt == NULL)
+			return -ENOMEM;
+
+#ifdef CONFIG_X86_64
+		hctxt->mode = VCPU_HVM_MODE_64B;
+		hctxt->cpu_regs.x86_64.rip =
+			(unsigned long)secondary_startup_64;
+		hctxt->cpu_regs.x86_64.rsp = stack_start;
+
+		hctxt->cpu_regs.x86_64.cr0 =
+			X86_CR0_PG | X86_CR0_WP | X86_CR0_PE;
+		hctxt->cpu_regs.x86_64.cr4 = X86_CR4_PAE;
+		hctxt->cpu_regs.x86_64.cr3 =
+			xen_pfn_to_cr3(virt_to_mfn(init_level4_pgt));
+		hctxt->cpu_regs.x86_64.efer = EFER_LME | EFER_NX;
+#else
+		hctxt->mode = VCPU_HVM_MODE_32B;
+		/*
+		 * startup_32_smp expects GDT loaded so we can't jump
+		 * there directly.
+		 */
+		hctxt->cpu_regs.x86_32.eip =
+			(unsigned long)hvmlite_smp_32 - __START_KERNEL_map;
+
+		hctxt->cpu_regs.x86_32.cr0 = X86_CR0_PE;
+
+		hctxt->cpu_regs.x86_32.cs_base = 0;
+		hctxt->cpu_regs.x86_32.cs_limit = ~0u;
+		hctxt->cpu_regs.x86_32.cs_ar = 0xc9b;
+		hctxt->cpu_regs.x86_32.ds_base = 0;
+		hctxt->cpu_regs.x86_32.ds_limit = ~0u;
+		hctxt->cpu_regs.x86_32.ds_ar = 0xc93;
+		hctxt->cpu_regs.x86_32.es_base = 0;
+		hctxt->cpu_regs.x86_32.es_limit = ~0u;
+		hctxt->cpu_regs.x86_32.es_ar = 0xc93;
+		hctxt->cpu_regs.x86_32.ss_base = 0;
+		hctxt->cpu_regs.x86_32.ss_limit = ~0u;
+		hctxt->cpu_regs.x86_32.ss_ar = 0xc93;
+		hctxt->cpu_regs.x86_32.tr_base = 0;
+		hctxt->cpu_regs.x86_32.tr_limit = 0xff;
+		hctxt->cpu_regs.x86_32.tr_ar = 0x8b;
+#endif
+#endif
 	}
 
-	if (HYPERVISOR_vcpu_op(VCPUOP_initialise, cpu, ctxt))
+	if (HYPERVISOR_vcpu_op(VCPUOP_initialise, cpu, ctxt_arg))
 		BUG();
 
-	kfree(ctxt);
+	kfree(ctxt_arg);
 	return 0;
 }
 
diff --git a/arch/x86/xen/smp.h b/arch/x86/xen/smp.h
index 963d62a..b4a833c 100644
--- a/arch/x86/xen/smp.h
+++ b/arch/x86/xen/smp.h
@@ -8,6 +8,10 @@ extern void xen_send_IPI_allbutself(int vector);
 extern void xen_send_IPI_all(int vector);
 extern void xen_send_IPI_self(int vector);
 
+#ifdef CONFIG_X86_32
+extern void hvmlite_smp_32(void);
+#endif
+
 #ifdef CONFIG_XEN_PVH
 extern void xen_pvh_early_cpu_init(int cpu, bool entry);
 #else
diff --git a/arch/x86/xen/xen-hvmlite.S b/arch/x86/xen/xen-hvmlite.S
index 90f03d0..8d6a642 100644
--- a/arch/x86/xen/xen-hvmlite.S
+++ b/arch/x86/xen/xen-hvmlite.S
@@ -134,6 +134,13 @@ ENTRY(hvmlite_start_xen)
 	ljmp    $0x10, $_pa(startup_32)
 #endif
 
+#ifdef CONFIG_X86_32
+ENTRY(hvmlite_smp_32)
+        mov $_pa(boot_gdt_descr), %eax
+        lgdt (%eax)
+        jmp startup_32_smp
+#endif
+
 	.data
 gdt:
 	.word	gdt_end - gdt
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1315296 — [PATCH v1 12/12] xen/hvmlite: Enable CPU on-/offlining

When offlining, we should properly clean up interrupts and wait
until hypervisor declares VCPU as down before cleaning up.

After VCPU that was previously offlined is brought back to life
we want to jump back to bare-metal entry points. It's a simple
jump on 64-bit but requires minor tweaking for 32-bit case.

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/smp.c         |   35 +++++++++++++++++++++++++----------
 arch/x86/xen/xen-hvmlite.S |    8 ++++++++
 2 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index fbad829..7e96a23 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -143,7 +143,7 @@ static void xen_smp_intr_free(unsigned int cpu)
 		kfree(per_cpu(xen_callfuncsingle_irq, cpu).name);
 		per_cpu(xen_callfuncsingle_irq, cpu).name = NULL;
 	}
-	if (xen_hvm_domain())
+	if (xen_hvm_domain() && !xen_hvmlite)
 		return;
 
 	if (per_cpu(xen_irq_work, cpu).irq >= 0) {
@@ -585,7 +585,8 @@ static int xen_cpu_disable(void)
 
 static void xen_cpu_die(unsigned int cpu)
 {
-	while (xen_pv_domain() && HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL)) {
+	while ((xen_pv_domain() || xen_hvmlite) &&
+	       HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL)) {
 		__set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule_timeout(HZ/10);
 	}
@@ -602,14 +603,28 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
 {
 	play_dead_common();
 	HYPERVISOR_vcpu_op(VCPUOP_down, smp_processor_id(), NULL);
-	cpu_bringup();
-	/*
-	 * commit 4b0c0f294 (tick: Cleanup NOHZ per cpu data on cpu down)
-	 * clears certain data that the cpu_idle loop (which called us
-	 * and that we return from) expects. The only way to get that
-	 * data back is to call:
-	 */
-	tick_nohz_idle_enter();
+
+	if (!xen_hvm_domain()) {
+		cpu_bringup();
+		/*
+		 * commit 4b0c0f294 (tick: Cleanup NOHZ per cpu data on cpu
+		 * down) clears certain data that the cpu_idle loop (which
+		 * called us and that we return from) expects. The only way to
+		 * get that data back is to call:
+		 */
+		tick_nohz_idle_enter();
+	} else {
+		/*
+		 * For 64-bit we can jump directly to SMP entry point but for
+		 * 32-bit we need to disable paging and load boot GDT (just
+		 * like in cpu_initialize_context()).
+		 */
+#ifdef CONFIG_X86_64
+		asm("jmp secondary_startup_64");
+#else
+		asm("jmp hvmlite_smp_32_hp");
+#endif
+	}
 }
 
 #else /* !CONFIG_HOTPLUG_CPU */
diff --git a/arch/x86/xen/xen-hvmlite.S b/arch/x86/xen/xen-hvmlite.S
index 8d6a642..4edd6ef 100644
--- a/arch/x86/xen/xen-hvmlite.S
+++ b/arch/x86/xen/xen-hvmlite.S
@@ -135,6 +135,14 @@ ENTRY(hvmlite_start_xen)
 #endif
 
 #ifdef CONFIG_X86_32
+ENTRY(hvmlite_smp_32_hp)
+	movl $_pa(initial_page_table), %eax
+	movl %eax, %cr3
+	ljmp $__KERNEL_CS,$_pa(5f)
+5:
+	movl $X86_CR0_PE, %eax
+	movl %eax, %cr0
+
 ENTRY(hvmlite_smp_32)
         mov $_pa(boot_gdt_descr), %eax
         lgdt (%eax)
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1315298 — [PATCH v1 03/12] xen/hvmlite: Import hvmlite-related Xen public interfaces

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 include/xen/interface/elfnote.h      |   12 +++-
 include/xen/interface/hvm/hvm_vcpu.h |  143 ++++++++++++++++++++++++++++++++++
 include/xen/interface/xen.h          |   24 ++++++
 3 files changed, 178 insertions(+), 1 deletions(-)
 create mode 100644 include/xen/interface/hvm/hvm_vcpu.h

diff --git a/include/xen/interface/elfnote.h b/include/xen/interface/elfnote.h
index f90b034..9e9f9bf 100644
--- a/include/xen/interface/elfnote.h
+++ b/include/xen/interface/elfnote.h
@@ -193,9 +193,19 @@
 #define XEN_ELFNOTE_SUPPORTED_FEATURES 17
 
 /*
+ * Physical entry point into the kernel.
+ *
+ * 32bit entry point into the kernel. When requested to launch the
+ * guest kernel in a HVM container, Xen will use this entry point to
+ * launch the guest in 32bit protected mode with paging disabled.
+ * Ignored otherwise.
+ */
+#define XEN_ELFNOTE_PHYS32_ENTRY 18
+
+/*
  * The number of the highest elfnote defined.
  */
-#define XEN_ELFNOTE_MAX XEN_ELFNOTE_SUPPORTED_FEATURES
+#define XEN_ELFNOTE_MAX XEN_ELFNOTE_PHYS32_ENTRY
 
 #endif /* __XEN_PUBLIC_ELFNOTE_H__ */
 
diff --git a/include/xen/interface/hvm/hvm_vcpu.h b/include/xen/interface/hvm/hvm_vcpu.h
new file mode 100644
index 0000000..32ca83e
--- /dev/null
+++ b/include/xen/interface/hvm/hvm_vcpu.h
@@ -0,0 +1,143 @@
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (c) 2015, Roger Pau Monne <roger.pau@citrix.com>
+ */
+
+#ifndef __XEN_PUBLIC_HVM_HVM_VCPU_H__
+#define __XEN_PUBLIC_HVM_HVM_VCPU_H__
+
+#include "../xen.h"
+
+struct vcpu_hvm_x86_32 {
+    uint32_t eax;
+    uint32_t ecx;
+    uint32_t edx;
+    uint32_t ebx;
+    uint32_t esp;
+    uint32_t ebp;
+    uint32_t esi;
+    uint32_t edi;
+    uint32_t eip;
+    uint32_t eflags;
+
+    uint32_t cr0;
+    uint32_t cr3;
+    uint32_t cr4;
+
+    uint32_t pad1;
+
+    /*
+     * EFER should only be used to set the NXE bit (if required)
+     * when starting a vCPU in 32bit mode with paging enabled or
+     * to set the LME/LMA bits in order to start the vCPU in
+     * compatibility mode.
+     */
+    uint64_t efer;
+
+    uint32_t cs_base;
+    uint32_t ds_base;
+    uint32_t ss_base;
+    uint32_t es_base;
+    uint32_t tr_base;
+    uint32_t cs_limit;
+    uint32_t ds_limit;
+    uint32_t ss_limit;
+    uint32_t es_limit;
+    uint32_t tr_limit;
+    uint16_t cs_ar;
+    uint16_t ds_ar;
+    uint16_t ss_ar;
+    uint16_t es_ar;
+    uint16_t tr_ar;
+
+    uint16_t pad2[3];
+};
+
+/*
+ * The layout of the _ar fields of the segment registers is the
+ * following:
+ *
+ * Bits   [0,3]: type (bits 40-43).
+ * Bit        4: s    (descriptor type, bit 44).
+ * Bit    [5,6]: dpl  (descriptor privilege level, bits 45-46).
+ * Bit        7: p    (segment-present, bit 47).
+ * Bit        8: avl  (available for system software, bit 52).
+ * Bit        9: l    (64-bit code segment, bit 53).
+ * Bit       10: db   (meaning depends on the segment, bit 54).
+ * Bit       11: g    (granularity, bit 55)
+ * Bits [12,15]: unused, must be blank.
+ *
+ * A more complete description of the meaning of this fields can be
+ * obtained from the Intel SDM, Volume 3, section 3.4.5.
+ */
+
+struct vcpu_hvm_x86_64 {
+    uint64_t rax;
+    uint64_t rcx;
+    uint64_t rdx;
+    uint64_t rbx;
+    uint64_t rsp;
+    uint64_t rbp;
+    uint64_t rsi;
+    uint64_t rdi;
+    uint64_t rip;
+    uint64_t rflags;
+
+    uint64_t cr0;
+    uint64_t cr3;
+    uint64_t cr4;
+    uint64_t efer;
+
+    /*
+     * Using VCPU_HVM_MODE_64B implies that the vCPU is launched
+     * directly in long mode, so the cached parts of the segment
+     * registers get set to match that environment.
+     *
+     * If the user wants to launch the vCPU in compatibility mode
+     * the 32-bit structure should be used instead.
+     */
+};
+
+struct vcpu_hvm_context {
+#define VCPU_HVM_MODE_32B 0  /* 32bit fields of the structure will be used. */
+#define VCPU_HVM_MODE_64B 1  /* 64bit fields of the structure will be used. */
+    uint32_t mode;
+
+    uint32_t pad;
+
+    /* CPU registers. */
+    union {
+        struct vcpu_hvm_x86_32 x86_32;
+        struct vcpu_hvm_x86_64 x86_64;
+    } cpu_regs;
+};
+typedef struct vcpu_hvm_context vcpu_hvm_context_t;
+
+#endif /* __XEN_PUBLIC_HVM_HVM_VCPU_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/include/xen/interface/xen.h b/include/xen/interface/xen.h
index d133112..61a25c4 100644
--- a/include/xen/interface/xen.h
+++ b/include/xen/interface/xen.h
@@ -645,6 +645,30 @@ struct start_info {
 	unsigned long nr_p2m_frames;/* # of pfns forming initial P->M table.  */
 };
 
+/*
+ * Start of day structure passed to PVH guests in %ebx.
+ *
+ * NOTE: nothing will be loaded at physical address 0, so
+ * a 0 value in any of the address fields should be treated
+ * as not present.
+ */
+struct hvm_start_info {
+#define HVM_START_MAGIC_VALUE 0x336ec578
+    uint32_t magic;             /* Contains the magic value 0x336ec578       */
+                                /* ("xEn3" with the 0x80 bit of the "E" set).*/
+    uint32_t flags;             /* SIF_xxx flags.                            */
+    uint32_t cmdline_paddr;     /* Physical address of the command line.     */
+    uint32_t nr_modules;        /* Number of modules passed to the kernel.   */
+    uint32_t modlist_paddr;     /* Physical address of an array of           */
+                                /* hvm_modlist_entry.                        */
+};
+
+struct hvm_modlist_entry {
+    uint32_t paddr;             /* Physical address of the module.           */
+    uint32_t size;              /* Size of the module in bytes.              */
+};
+
+
 /* These flags are passed in the 'flags' field of start_info_t. */
 #define SIF_PRIVILEGED      (1<<0)  /* Is the domain privileged? */
 #define SIF_INITDOMAIN      (1<<1)  /* Is this the initial control domain? */
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1315299 — [PATCH v1 05/12] xen/hvmlite: Allow HVMlite guests delay initializing grant table

.. just like we currently do for PVH guests

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/grant-table.c |    4 ++--
 drivers/xen/grant-table.c  |    8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/x86/xen/grant-table.c b/arch/x86/xen/grant-table.c
index e079500..40ad9c2 100644
--- a/arch/x86/xen/grant-table.c
+++ b/arch/x86/xen/grant-table.c
@@ -110,7 +110,7 @@ int arch_gnttab_init(unsigned long nr_shared)
 	return arch_gnttab_valloc(&gnttab_shared_vm_area, nr_shared);
 }
 
-#ifdef CONFIG_XEN_PVH
+#ifdef CONFIG_XEN_PVHVM
 #include <xen/balloon.h>
 #include <xen/events.h>
 #include <linux/slab.h>
@@ -164,7 +164,7 @@ static int __init xlated_setup_gnttab_pages(void)
 
 static int __init xen_pvh_gnttab_setup(void)
 {
-	if (!xen_pvh_domain())
+	if (!xen_pvh_domain() && !xen_hvmlite)
 		return -ENODEV;
 
 	return xlated_setup_gnttab_pages();
diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
index effbaf9..8c6c0cf 100644
--- a/drivers/xen/grant-table.c
+++ b/drivers/xen/grant-table.c
@@ -1147,13 +1147,13 @@ EXPORT_SYMBOL_GPL(gnttab_init);
 
 static int __gnttab_init(void)
 {
+	if (!xen_domain())
+		return -ENODEV;
+
 	/* Delay grant-table initialization in the PV on HVM case */
-	if (xen_hvm_domain())
+	if (xen_hvm_domain() && !xen_hvmlite)
 		return 0;
 
-	if (!xen_pv_domain())
-		return -ENODEV;
-
 	return gnttab_init();
 }
 /* Starts after core_initcall so that xen_pvh_gnttab_setup can be called
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1315300 — [PATCH v1 07/12] xen/hvmlite: Prepare cpu_initialize_context() routine for HVMlite SMP

Subsequent patch will add support for starting secondary VCPUs in
HVMlite guest. This patch exists to make review easier.

No functional changes (except for introduction of 'if (!xen_hvmlite)').

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/smp.c |  104 ++++++++++++++++++++++++++++-----------------------
 1 files changed, 57 insertions(+), 47 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 3f4ebf0..5fc4afb 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -390,70 +390,80 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle)
 	if (cpumask_test_and_set_cpu(cpu, xen_cpu_initialized_map))
 		return 0;
 
-	ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
-	if (ctxt == NULL)
-		return -ENOMEM;
+	if (!xen_hvmlite) {
 
-	gdt = get_cpu_gdt_table(cpu);
+		ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
+		if (ctxt == NULL)
+			return -ENOMEM;
+
+		gdt = get_cpu_gdt_table(cpu);
 
 #ifdef CONFIG_X86_32
-	/* Note: PVH is not yet supported on x86_32. */
-	ctxt->user_regs.fs = __KERNEL_PERCPU;
-	ctxt->user_regs.gs = __KERNEL_STACK_CANARY;
+		/* Note: PVH is not yet supported on x86_32. */
+		ctxt->user_regs.fs = __KERNEL_PERCPU;
+		ctxt->user_regs.gs = __KERNEL_STACK_CANARY;
 #endif
-	memset(&ctxt->fpu_ctxt, 0, sizeof(ctxt->fpu_ctxt));
+		memset(&ctxt->fpu_ctxt, 0, sizeof(ctxt->fpu_ctxt));
 
-	if (!xen_feature(XENFEAT_auto_translated_physmap)) {
-		ctxt->user_regs.eip = (unsigned long)cpu_bringup_and_idle;
-		ctxt->flags = VGCF_IN_KERNEL;
-		ctxt->user_regs.eflags = 0x1000; /* IOPL_RING1 */
-		ctxt->user_regs.ds = __USER_DS;
-		ctxt->user_regs.es = __USER_DS;
-		ctxt->user_regs.ss = __KERNEL_DS;
+		if (!xen_feature(XENFEAT_auto_translated_physmap)) {
+			ctxt->user_regs.eip =
+				(unsigned long)cpu_bringup_and_idle;
+			ctxt->flags = VGCF_IN_KERNEL;
+			ctxt->user_regs.eflags = 0x1000; /* IOPL_RING1 */
+			ctxt->user_regs.ds = __USER_DS;
+			ctxt->user_regs.es = __USER_DS;
+			ctxt->user_regs.ss = __KERNEL_DS;
 
-		xen_copy_trap_info(ctxt->trap_ctxt);
+			xen_copy_trap_info(ctxt->trap_ctxt);
 
-		ctxt->ldt_ents = 0;
+			ctxt->ldt_ents = 0;
 
-		BUG_ON((unsigned long)gdt & ~PAGE_MASK);
+			BUG_ON((unsigned long)gdt & ~PAGE_MASK);
 
-		gdt_mfn = arbitrary_virt_to_mfn(gdt);
-		make_lowmem_page_readonly(gdt);
-		make_lowmem_page_readonly(mfn_to_virt(gdt_mfn));
+			gdt_mfn = arbitrary_virt_to_mfn(gdt);
+			make_lowmem_page_readonly(gdt);
+			make_lowmem_page_readonly(mfn_to_virt(gdt_mfn));
 
-		ctxt->gdt_frames[0] = gdt_mfn;
-		ctxt->gdt_ents      = GDT_ENTRIES;
+			ctxt->gdt_frames[0] = gdt_mfn;
+			ctxt->gdt_ents      = GDT_ENTRIES;
 
-		ctxt->kernel_ss = __KERNEL_DS;
-		ctxt->kernel_sp = idle->thread.sp0;
+			ctxt->kernel_ss = __KERNEL_DS;
+			ctxt->kernel_sp = idle->thread.sp0;
 
 #ifdef CONFIG_X86_32
-		ctxt->event_callback_cs     = __KERNEL_CS;
-		ctxt->failsafe_callback_cs  = __KERNEL_CS;
+			ctxt->event_callback_cs     = __KERNEL_CS;
+			ctxt->failsafe_callback_cs  = __KERNEL_CS;
 #else
-		ctxt->gs_base_kernel = per_cpu_offset(cpu);
+			ctxt->gs_base_kernel = per_cpu_offset(cpu);
 #endif
-		ctxt->event_callback_eip    =
-					(unsigned long)xen_hypervisor_callback;
-		ctxt->failsafe_callback_eip =
-					(unsigned long)xen_failsafe_callback;
-		ctxt->user_regs.cs = __KERNEL_CS;
-		per_cpu(xen_cr3, cpu) = __pa(swapper_pg_dir);
-	}
+			ctxt->event_callback_eip    =
+				(unsigned long)xen_hypervisor_callback;
+			ctxt->failsafe_callback_eip =
+				(unsigned long)xen_failsafe_callback;
+			ctxt->user_regs.cs = __KERNEL_CS;
+			per_cpu(xen_cr3, cpu) = __pa(swapper_pg_dir);
+		}
 #ifdef CONFIG_XEN_PVH
-	else {
-		/*
-		 * The vcpu comes on kernel page tables which have the NX pte
-		 * bit set. This means before DS/SS is touched, NX in
-		 * EFER must be set. Hence the following assembly glue code.
-		 */
-		ctxt->user_regs.eip = (unsigned long)xen_pvh_early_cpu_init;
-		ctxt->user_regs.rdi = cpu;
-		ctxt->user_regs.rsi = true;  /* entry == true */
-	}
+		else {
+			/*
+			 * The vcpu comes on kernel page tables which have the
+			 * NX pte bit set. This means before DS/SS is touched,
+			 * NX in EFER must be set. Hence the following assembly
+			 * glue code.
+			 */
+			ctxt->user_regs.eip =
+				(unsigned long)xen_pvh_early_cpu_init;
+			ctxt->user_regs.rdi = cpu;
+			ctxt->user_regs.rsi = true;  /* entry == true */
+		}
 #endif
-	ctxt->user_regs.esp = idle->thread.sp0 - sizeof(struct pt_regs);
-	ctxt->ctrlreg[3] = xen_pfn_to_cr3(virt_to_gfn(swapper_pg_dir));
+		ctxt->user_regs.esp = idle->thread.sp0 - sizeof(struct pt_regs);
+		ctxt->ctrlreg[3] = xen_pfn_to_cr3(virt_to_gfn(swapper_pg_dir));
+	} else {
+		ctxt = NULL; /* To quiet down compiler */
+		BUG();
+	}
+
 	if (HYPERVISOR_vcpu_op(VCPUOP_initialise, cpu, ctxt))
 		BUG();
 
-- 
1.7.1

[toc] | [prev] | [next] | [standalone]

#1316478

On 22/01/16 21:35, Boris Ostrovsky wrote:
> This series introduces HVMlite support for unprivileged guests.
> 
> It has been tested on Intel/AMD, both 32- and 64-bit, including CPU on- and
> offlining and save/restore. (Restore will result in APIC write warnings
> which exist now for 32-bit PV guests as well so I didn't address this in
> this series)

Can you remove PVH support in this series as well?  We won't necessarily
remove PVH support immediately but I'd like to see the ultimate end result.

David

[toc] | [prev] | [next] | [standalone]

#1316813

On 01/25/2016 05:51 AM, David Vrabel wrote:
> On 22/01/16 21:35, Boris Ostrovsky wrote:
>> This series introduces HVMlite support for unprivileged guests.
>>
>> It has been tested on Intel/AMD, both 32- and 64-bit, including CPU on- and
>> offlining and save/restore. (Restore will result in APIC write warnings
>> which exist now for 32-bit PV guests as well so I didn't address this in
>> this series)
> Can you remove PVH support in this series as well?  We won't necessarily
> remove PVH support immediately but I'd like to see the ultimate end result.

I'd rather wait until we have HVMlite dom0 before dropping PVH. If 
nothing else it may help debugging.

BTW, I assume we are going to rename HVMlite to PVH once the original 
implementation is removed.


-boris

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web