Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1316892 > unrolled thread

[PATCH v2 00/21] arm64: Virtualization Host Extension support

Started byMarc Zyngier <marc.zyngier@arm.com>
First post2016-01-25 17:00 +0100
Last post2016-01-25 20:20 +0100
Articles 9 on this page of 29 — 4 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 00/21] arm64: Virtualization Host Extension support Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 16/21] arm64: KVM: VHE: Add fpsimd enabling on guest access Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 09/21] arm64: KVM: VHE: Differenciate host/guest sysreg save/restore Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 06/21] arm64: KVM: VHE: Patch out use of HVC Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 15/21] arm64: KVM: VHE: Use unified sysreg accessors for timer Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 21/21] arm64: Panic when VHE and non VHE CPUs coexist Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
      Re: [PATCH v2 21/21] arm64: Panic when VHE and non VHE CPUs coexist "Suzuki K. Poulose" <Suzuki.Poulose@arm.com> - 2016-01-26 15:30 +0100
        Re: [PATCH v2 21/21] arm64: Panic when VHE and non VHE CPUs coexist Marc Zyngier <marc.zyngier@arm.com> - 2016-01-26 15:40 +0100
    [PATCH v2 10/21] arm64: KVM: VHE: Split save/restore of sysregs shared between EL1 and EL2 Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 20/21] arm64: VHE: Add support for running Linux in EL2 mode Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
      Re: [PATCH v2 20/21] arm64: VHE: Add support for running Linux in EL2  mode "Suzuki K. Poulose" <Suzuki.Poulose@arm.com> - 2016-01-26 15:10 +0100
        Re: [PATCH v2 20/21] arm64: VHE: Add support for running Linux in EL2  mode "Suzuki K. Poulose" <Suzuki.Poulose@arm.com> - 2016-01-26 15:40 +0100
    [PATCH v2 17/21] arm64: KVM: VHE: Add alternative panic handling Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 05/21] arm64: KVM: VHE: Turn VTCR_EL2 setup into a reusable macro Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:00 +0100
    [PATCH v2 11/21] arm64: KVM: VHE: Use unified system register accessors Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 03/21] arm64: Add ARM64_HAS_VIRT_HOST_EXTN feature Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 08/21] arm64: KVM: VHE: Introduce unified system register accessors Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 04/21] arm64: KVM: Skip HYP setup when already running in HYP Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 02/21] arm64: Allow the arch timer to use the HYP timer Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 13/21] arm64: KVM: VHE: Make __fpsimd_enabled VHE aware Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 14/21] arm64: KVM: VHE: Implement VHE activate/deactivate_traps Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    [PATCH v2 07/21] arm64: KVM: VHE: Patch out kern_hyp_va Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:10 +0100
    Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Arnd Bergmann <arnd@arndb.de> - 2016-01-25 17:20 +0100
      Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Arnd Bergmann <arnd@arndb.de> - 2016-01-25 17:30 +0100
      Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:30 +0100
    Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Will Deacon <will.deacon@arm.com> - 2016-01-25 17:30 +0100
      Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 17:40 +0100
        Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Will Deacon <will.deacon@arm.com> - 2016-01-25 17:50 +0100
          Re: [PATCH v2 00/21] arm64: Virtualization Host Extension support Marc Zyngier <marc.zyngier@arm.com> - 2016-01-25 20:20 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1316932 — [PATCH v2 14/21] arm64: KVM: VHE: Implement VHE activate/deactivate_traps

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-01-25 17:10 +0100
Subject[PATCH v2 14/21] arm64: KVM: VHE: Implement VHE activate/deactivate_traps
Message-ID<qURSl-6uy-71@gated-at.bofh.it>
In reply to#1316892
Running the kernel in HYP mode requires the HCR_E2H bit to be set
at all times, and the HCR_TGE bit to be set when running as a host
(and cleared when running as a guest). At the same time, the vector
 must be set to the current role of the kernel (either host or
hypervisor), and a couple of system registers differ between VHE
and non-VHE.

We implement these by using another set of alternate functions
that get dynamically patched.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h     |  1 +
 arch/arm64/include/asm/kvm_emulate.h |  3 +++
 arch/arm64/kvm/hyp/switch.c          | 52 +++++++++++++++++++++++++++++++++---
 3 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 738a95f..73d3826 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -23,6 +23,7 @@
 #include <asm/types.h>
 
 /* Hyp Configuration Register (HCR) bits */
+#define HCR_E2H		(UL(1) << 34)
 #define HCR_ID		(UL(1) << 33)
 #define HCR_CD		(UL(1) << 32)
 #define HCR_RW_SHIFT	31
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 3066328..5ae0c69 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -29,6 +29,7 @@
 #include <asm/kvm_mmio.h>
 #include <asm/ptrace.h>
 #include <asm/cputype.h>
+#include <asm/virt.h>
 
 unsigned long *vcpu_reg32(const struct kvm_vcpu *vcpu, u8 reg_num);
 unsigned long *vcpu_spsr32(const struct kvm_vcpu *vcpu);
@@ -43,6 +44,8 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);
 static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
 {
 	vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS;
+	if (is_kernel_in_hyp_mode())
+		vcpu->arch.hcr_el2 |= HCR_E2H;
 	if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features))
 		vcpu->arch.hcr_el2 &= ~HCR_RW;
 }
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 6f264dc..77f7c94 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -15,6 +15,8 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <asm/kvm_asm.h>
+
 #include "hyp.h"
 
 static bool __hyp_text __fpsimd_enabled_nvhe(void)
@@ -36,6 +38,27 @@ bool __hyp_text __fpsimd_enabled(void)
 	return __fpsimd_is_enabled()();
 }
 
+static void __hyp_text __activate_traps_vhe(void)
+{
+	u64 val;
+
+	val = read_sysreg(cpacr_el1);
+	val |= 1 << 28;
+	val &= ~(3 << 20);
+	write_sysreg(val, cpacr_el1);
+
+	write_sysreg(__kvm_hyp_vector, vbar_el1);
+}
+
+static void __hyp_text __activate_traps_nvhe(void)
+{
+	write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2);
+}
+
+static hyp_alternate_select(__activate_traps_arch,
+			    __activate_traps_nvhe, __activate_traps_vhe,
+			    ARM64_HAS_VIRT_HOST_EXTN);
+
 static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
 {
 	u64 val;
@@ -55,16 +78,39 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
 	write_sysreg(val, hcr_el2);
 	/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
 	write_sysreg(1 << 15, hstr_el2);
-	write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2);
 	write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+	__activate_traps_arch()();
 }
 
-static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
+static void __hyp_text __deactivate_traps_vhe(void)
+{
+	extern char vectors[];	/* kernel exception vectors */
+	u64 val;
+
+	write_sysreg(HCR_RW | HCR_TGE | HCR_E2H, hcr_el2);
+
+	val = read_sysreg(cpacr_el1);
+	val |= 3 << 20;
+	write_sysreg(val, cpacr_el1);
+
+	write_sysreg(vectors, vbar_el1);
+}
+
+static void __hyp_text __deactivate_traps_nvhe(void)
 {
 	write_sysreg(HCR_RW, hcr_el2);
+	write_sysreg(0, cptr_el2);
+}
+
+static hyp_alternate_select(__deactivate_traps_arch,
+			    __deactivate_traps_nvhe, __deactivate_traps_vhe,
+			    ARM64_HAS_VIRT_HOST_EXTN);
+
+static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
+{
+	__deactivate_traps_arch()();
 	write_sysreg(0, hstr_el2);
 	write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2);
-	write_sysreg(0, cptr_el2);
 }
 
 static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1316934 — [PATCH v2 07/21] arm64: KVM: VHE: Patch out kern_hyp_va

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-01-25 17:10 +0100
Subject[PATCH v2 07/21] arm64: KVM: VHE: Patch out kern_hyp_va
Message-ID<qURSl-6uy-73@gated-at.bofh.it>
In reply to#1316892
The kern_hyp_va macro is pretty meaninless with VHE, as there is
only one mapping - the kernel one.

In order to keep the code readable and efficient, use runtime
patching to replace the 'and' instruction used to compute the VA
with a 'nop'.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/kvm_mmu.h | 11 ++++++++++-
 arch/arm64/kvm/hyp/hyp.h         | 25 ++++++++++++++++++++++---
 2 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index d3e6d7b..62f0d14 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -23,13 +23,16 @@
 #include <asm/cpufeature.h>
 
 /*
- * As we only have the TTBR0_EL2 register, we cannot express
+ * As ARMv8.0 only has the TTBR0_EL2 register, we cannot express
  * "negative" addresses. This makes it impossible to directly share
  * mappings with the kernel.
  *
  * Instead, give the HYP mode its own VA region at a fixed offset from
  * the kernel by just masking the top bits (which are all ones for a
  * kernel address).
+ *
+ * ARMv8.1 (using VHE) does have a TTBR1_EL2, and doesn't use these
+ * macros (the entire kernel runs at EL2).
  */
 #define HYP_PAGE_OFFSET_SHIFT	VA_BITS
 #define HYP_PAGE_OFFSET_MASK	((UL(1) << HYP_PAGE_OFFSET_SHIFT) - 1)
@@ -56,6 +59,8 @@
 
 #ifdef __ASSEMBLY__
 
+#include <asm/alternative.h>
+#include <asm/cpufeature.h>
 #include <asm/kvm_arm.h>
 
 .macro setup_vtcr tmp1, tmp2
@@ -84,7 +89,11 @@
  * reg: VA to be converted.
  */
 .macro kern_hyp_va	reg
+alternative_if_not ARM64_HAS_VIRT_HOST_EXTN	
 	and	\reg, \reg, #HYP_PAGE_OFFSET_MASK
+alternative_else
+	nop
+alternative_endif
 .endm
 
 #else
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index fb27517..fc502f3 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -25,9 +25,28 @@
 
 #define __hyp_text __section(.hyp.text) notrace
 
-#define kern_hyp_va(v) (typeof(v))((unsigned long)(v) & HYP_PAGE_OFFSET_MASK)
-#define hyp_kern_va(v) (typeof(v))((unsigned long)(v) - HYP_PAGE_OFFSET \
-						      + PAGE_OFFSET)
+static inline unsigned long __kern_hyp_va(unsigned long v)
+{
+	asm volatile(ALTERNATIVE("and %0, %0, %1",
+				 "nop",
+				 ARM64_HAS_VIRT_HOST_EXTN)
+		     : "+r" (v) : "i" (HYP_PAGE_OFFSET_MASK));
+	return v;
+}
+
+#define kern_hyp_va(v) (typeof(v))(__kern_hyp_va((unsigned long)(v)))
+
+static inline unsigned long __hyp_kern_va(unsigned long v)
+{
+	u64 offset = PAGE_OFFSET - HYP_PAGE_OFFSET;
+	asm volatile(ALTERNATIVE("add %0, %0, %1",
+				 "nop",
+				 ARM64_HAS_VIRT_HOST_EXTN)
+		     : "+r" (v) : "r" (offset));
+	return v;
+}
+
+#define hyp_kern_va(v) (typeof(v))(__hyp_kern_va((unsigned long)(v)))
 
 /**
  * hyp_alternate_select - Generates patchable code sequences that are
-- 
2.1.4

[toc] | [prev] | [next] | [standalone]

#1316947

FromArnd Bergmann <arnd@arndb.de>
Date2016-01-25 17:20 +0100
Message-ID<qUS1Z-6yn-37@gated-at.bofh.it>
In reply to#1316892
On Monday 25 January 2016 15:53:34 Marc Zyngier wrote:
> host and guest, reducing the overhead of virtualization.
> 
> In order to have the same kernel binary running on all versions of the
> architecture, this series makes heavy use of runtime code patching.
> 
> The first 20 patches massage the KVM code to deal with VHE and enable
> Linux to run at EL2. The last patch catches an ugly case when VHE
> capable CPUs are paired with some of their less capable siblings. This
> should never happen, but hey...
> 
> I have deliberately left out some of the more "advanced"
> optimizations, as they are likely to distract the reviewer from the
> core infrastructure, which is what I care about at the moment.

One question: as you mention that you use a lot of runtime code patching
to make this work transparently, how does this compare to runtime patching
the existing kernel to run in EL2 mode without VHE? Is that even possible?

My interpretation so far as always been "that's too complicated to
do because it would require a lot of runtime patching", but now we seem
to get that anyway because we want to run a hypervisor-enabled kernel in
either EL1 or EL2 depending on the presence of another feature.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1316957

FromArnd Bergmann <arnd@arndb.de>
Date2016-01-25 17:30 +0100
Message-ID<qUSbD-6CP-5@gated-at.bofh.it>
In reply to#1316947
On Monday 25 January 2016 16:23:37 Marc Zyngier wrote:
> On 25/01/16 16:15, Arnd Bergmann wrote:
> > On Monday 25 January 2016 15:53:34 Marc Zyngier wrote:
> >> host and guest, reducing the overhead of virtualization.
> >>
> >> In order to have the same kernel binary running on all versions of the
> >> architecture, this series makes heavy use of runtime code patching.
> >>
> >> The first 20 patches massage the KVM code to deal with VHE and enable
> >> Linux to run at EL2. The last patch catches an ugly case when VHE
> >> capable CPUs are paired with some of their less capable siblings. This
> >> should never happen, but hey...
> >>
> >> I have deliberately left out some of the more "advanced"
> >> optimizations, as they are likely to distract the reviewer from the
> >> core infrastructure, which is what I care about at the moment.
> > 
> > One question: as you mention that you use a lot of runtime code patching
> > to make this work transparently, how does this compare to runtime patching
> > the existing kernel to run in EL2 mode without VHE? Is that even possible?
> 
> I haven't explored that particular avenue - by the look of it, this
> would require a lot more work, as v8.0 EL2 lacks a number of features
> that Linux currently requires (like having two TTBRs, for example).

Ok, I see.

> > My interpretation so far as always been "that's too complicated to
> > do because it would require a lot of runtime patching", but now we seem
> > to get that anyway because we want to run a hypervisor-enabled kernel in
> > either EL1 or EL2 depending on the presence of another feature.
> 
> The kernel itself is mostly untouched (what runs at EL1 also runs at EL2
> without any patching, because the new EL2 is now a superset of EL1). It
> is the hypervisor code that gets a beating with the code-patching stick.

Thanks for the explanation, makes sense.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1316958

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-01-25 17:30 +0100
Message-ID<qUSbD-6CP-7@gated-at.bofh.it>
In reply to#1316947
On 25/01/16 16:15, Arnd Bergmann wrote:
> On Monday 25 January 2016 15:53:34 Marc Zyngier wrote:
>> host and guest, reducing the overhead of virtualization.
>>
>> In order to have the same kernel binary running on all versions of the
>> architecture, this series makes heavy use of runtime code patching.
>>
>> The first 20 patches massage the KVM code to deal with VHE and enable
>> Linux to run at EL2. The last patch catches an ugly case when VHE
>> capable CPUs are paired with some of their less capable siblings. This
>> should never happen, but hey...
>>
>> I have deliberately left out some of the more "advanced"
>> optimizations, as they are likely to distract the reviewer from the
>> core infrastructure, which is what I care about at the moment.
> 
> One question: as you mention that you use a lot of runtime code patching
> to make this work transparently, how does this compare to runtime patching
> the existing kernel to run in EL2 mode without VHE? Is that even possible?

I haven't explored that particular avenue - by the look of it, this
would require a lot more work, as v8.0 EL2 lacks a number of features
that Linux currently requires (like having two TTBRs, for example).

> My interpretation so far as always been "that's too complicated to
> do because it would require a lot of runtime patching", but now we seem
> to get that anyway because we want to run a hypervisor-enabled kernel in
> either EL1 or EL2 depending on the presence of another feature.

The kernel itself is mostly untouched (what runs at EL1 also runs at EL2
without any patching, because the new EL2 is now a superset of EL1). It
is the hypervisor code that gets a beating with the code-patching stick.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[toc] | [prev] | [next] | [standalone]

#1316963

FromWill Deacon <will.deacon@arm.com>
Date2016-01-25 17:30 +0100
Message-ID<qUSbE-6CP-39@gated-at.bofh.it>
In reply to#1316892
On Mon, Jan 25, 2016 at 03:53:34PM +0000, Marc Zyngier wrote:
> ARMv8.1 comes with the "Virtualization Host Extension" (VHE for
> short), which enables simpler support of Type-2 hypervisors.
> 
> This extension allows the kernel to directly run at EL2, and
> significantly reduces the number of system registers shared between
> host and guest, reducing the overhead of virtualization.
> 
> In order to have the same kernel binary running on all versions of the
> architecture, this series makes heavy use of runtime code patching.
> 
> The first 20 patches massage the KVM code to deal with VHE and enable
> Linux to run at EL2. The last patch catches an ugly case when VHE
> capable CPUs are paired with some of their less capable siblings. This
> should never happen, but hey...
> 
> I have deliberately left out some of the more "advanced"
> optimizations, as they are likely to distract the reviewer from the
> core infrastructure, which is what I care about at the moment.
> 
> A few things to note:
> 
> - Given that the code has been almost entierely rewritten, I've
>   dropped all Acks from the new patches
> 
> - GDB is currently busted on VHE systems, as it checks for version 6
>   on the debug architecture, while VHE is version 7. The binutils
>   people are on the case.

[...]

>  arch/arm/include/asm/virt.h          |   5 ++
>  arch/arm/kvm/arm.c                   | 151 +++++++++++++++++++------------
>  arch/arm/kvm/mmu.c                   |   7 ++
>  arch/arm64/Kconfig                   |  13 +++
>  arch/arm64/include/asm/cpufeature.h  |   3 +-
>  arch/arm64/include/asm/kvm_arm.h     |   1 +
>  arch/arm64/include/asm/kvm_emulate.h |   3 +
>  arch/arm64/include/asm/kvm_mmu.h     |  34 ++++++-
>  arch/arm64/include/asm/virt.h        |  27 ++++++
>  arch/arm64/kernel/asm-offsets.c      |   3 -
>  arch/arm64/kernel/cpufeature.c       |  15 +++-
>  arch/arm64/kernel/head.S             |  51 ++++++++++-
>  arch/arm64/kernel/smp.c              |   3 +
>  arch/arm64/kvm/hyp-init.S            |  18 +---
>  arch/arm64/kvm/hyp.S                 |   7 ++
>  arch/arm64/kvm/hyp/entry.S           |   6 ++
>  arch/arm64/kvm/hyp/hyp-entry.S       | 107 +++++++---------------
>  arch/arm64/kvm/hyp/hyp.h             | 119 ++++++++++++++++++++++--
>  arch/arm64/kvm/hyp/switch.c          | 170 +++++++++++++++++++++++++++++++----
>  arch/arm64/kvm/hyp/sysreg-sr.c       | 147 ++++++++++++++++++++----------
>  arch/arm64/kvm/hyp/timer-sr.c        |  10 +--
>  drivers/clocksource/arm_arch_timer.c |  96 ++++++++++++--------
>  22 files changed, 724 insertions(+), 272 deletions(-)

Have you tried hw_breakpoint/perf/ptrace with these changes? I was under
the impression that the debug architecture was aware of E2H and did need
some changes made. I know you say that GDB is broken anyway, but we should
check that the kernel does the right thing if userspace pokes it the
right way.

Will

[toc] | [prev] | [next] | [standalone]

#1316976

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-01-25 17:40 +0100
Message-ID<qUSlk-6Ia-33@gated-at.bofh.it>
In reply to#1316963
On 25/01/16 16:26, Will Deacon wrote:
> On Mon, Jan 25, 2016 at 03:53:34PM +0000, Marc Zyngier wrote:
>> ARMv8.1 comes with the "Virtualization Host Extension" (VHE for
>> short), which enables simpler support of Type-2 hypervisors.
>>
>> This extension allows the kernel to directly run at EL2, and
>> significantly reduces the number of system registers shared between
>> host and guest, reducing the overhead of virtualization.
>>
>> In order to have the same kernel binary running on all versions of the
>> architecture, this series makes heavy use of runtime code patching.
>>
>> The first 20 patches massage the KVM code to deal with VHE and enable
>> Linux to run at EL2. The last patch catches an ugly case when VHE
>> capable CPUs are paired with some of their less capable siblings. This
>> should never happen, but hey...
>>
>> I have deliberately left out some of the more "advanced"
>> optimizations, as they are likely to distract the reviewer from the
>> core infrastructure, which is what I care about at the moment.
>>
>> A few things to note:
>>
>> - Given that the code has been almost entierely rewritten, I've
>>   dropped all Acks from the new patches
>>
>> - GDB is currently busted on VHE systems, as it checks for version 6
>>   on the debug architecture, while VHE is version 7. The binutils
>>   people are on the case.
> 
> [...]
> 
>>  arch/arm/include/asm/virt.h          |   5 ++
>>  arch/arm/kvm/arm.c                   | 151 +++++++++++++++++++------------
>>  arch/arm/kvm/mmu.c                   |   7 ++
>>  arch/arm64/Kconfig                   |  13 +++
>>  arch/arm64/include/asm/cpufeature.h  |   3 +-
>>  arch/arm64/include/asm/kvm_arm.h     |   1 +
>>  arch/arm64/include/asm/kvm_emulate.h |   3 +
>>  arch/arm64/include/asm/kvm_mmu.h     |  34 ++++++-
>>  arch/arm64/include/asm/virt.h        |  27 ++++++
>>  arch/arm64/kernel/asm-offsets.c      |   3 -
>>  arch/arm64/kernel/cpufeature.c       |  15 +++-
>>  arch/arm64/kernel/head.S             |  51 ++++++++++-
>>  arch/arm64/kernel/smp.c              |   3 +
>>  arch/arm64/kvm/hyp-init.S            |  18 +---
>>  arch/arm64/kvm/hyp.S                 |   7 ++
>>  arch/arm64/kvm/hyp/entry.S           |   6 ++
>>  arch/arm64/kvm/hyp/hyp-entry.S       | 107 +++++++---------------
>>  arch/arm64/kvm/hyp/hyp.h             | 119 ++++++++++++++++++++++--
>>  arch/arm64/kvm/hyp/switch.c          | 170 +++++++++++++++++++++++++++++++----
>>  arch/arm64/kvm/hyp/sysreg-sr.c       | 147 ++++++++++++++++++++----------
>>  arch/arm64/kvm/hyp/timer-sr.c        |  10 +--
>>  drivers/clocksource/arm_arch_timer.c |  96 ++++++++++++--------
>>  22 files changed, 724 insertions(+), 272 deletions(-)
> 
> Have you tried hw_breakpoint/perf/ptrace with these changes? I was under
> the impression that the debug architecture was aware of E2H and did need
> some changes made. I know you say that GDB is broken anyway, but we should
> check that the kernel does the right thing if userspace pokes it the
> right way.

I did use HW breakpoints on the model by hacking the host kernel to
return Debug Version 6 instead of 7, and things seem to work as
expected. strace also works out of the box.

As for perf, did you have something precise in mind?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[toc] | [prev] | [next] | [standalone]

#1317000

FromWill Deacon <will.deacon@arm.com>
Date2016-01-25 17:50 +0100
Message-ID<qUSv1-6M7-51@gated-at.bofh.it>
In reply to#1316976
On Mon, Jan 25, 2016 at 04:37:39PM +0000, Marc Zyngier wrote:
> On 25/01/16 16:26, Will Deacon wrote:
> > On Mon, Jan 25, 2016 at 03:53:34PM +0000, Marc Zyngier wrote:
> >> ARMv8.1 comes with the "Virtualization Host Extension" (VHE for
> >> short), which enables simpler support of Type-2 hypervisors.
> >>
> >> This extension allows the kernel to directly run at EL2, and
> >> significantly reduces the number of system registers shared between
> >> host and guest, reducing the overhead of virtualization.
> >>
> >> In order to have the same kernel binary running on all versions of the
> >> architecture, this series makes heavy use of runtime code patching.
> >>
> >> The first 20 patches massage the KVM code to deal with VHE and enable
> >> Linux to run at EL2. The last patch catches an ugly case when VHE
> >> capable CPUs are paired with some of their less capable siblings. This
> >> should never happen, but hey...
> >>
> >> I have deliberately left out some of the more "advanced"
> >> optimizations, as they are likely to distract the reviewer from the
> >> core infrastructure, which is what I care about at the moment.
> >>
> >> A few things to note:
> >>
> >> - Given that the code has been almost entierely rewritten, I've
> >>   dropped all Acks from the new patches
> >>
> >> - GDB is currently busted on VHE systems, as it checks for version 6
> >>   on the debug architecture, while VHE is version 7. The binutils
> >>   people are on the case.
> > 
> > [...]
> > 
> >>  arch/arm/include/asm/virt.h          |   5 ++
> >>  arch/arm/kvm/arm.c                   | 151 +++++++++++++++++++------------
> >>  arch/arm/kvm/mmu.c                   |   7 ++
> >>  arch/arm64/Kconfig                   |  13 +++
> >>  arch/arm64/include/asm/cpufeature.h  |   3 +-
> >>  arch/arm64/include/asm/kvm_arm.h     |   1 +
> >>  arch/arm64/include/asm/kvm_emulate.h |   3 +
> >>  arch/arm64/include/asm/kvm_mmu.h     |  34 ++++++-
> >>  arch/arm64/include/asm/virt.h        |  27 ++++++
> >>  arch/arm64/kernel/asm-offsets.c      |   3 -
> >>  arch/arm64/kernel/cpufeature.c       |  15 +++-
> >>  arch/arm64/kernel/head.S             |  51 ++++++++++-
> >>  arch/arm64/kernel/smp.c              |   3 +
> >>  arch/arm64/kvm/hyp-init.S            |  18 +---
> >>  arch/arm64/kvm/hyp.S                 |   7 ++
> >>  arch/arm64/kvm/hyp/entry.S           |   6 ++
> >>  arch/arm64/kvm/hyp/hyp-entry.S       | 107 +++++++---------------
> >>  arch/arm64/kvm/hyp/hyp.h             | 119 ++++++++++++++++++++++--
> >>  arch/arm64/kvm/hyp/switch.c          | 170 +++++++++++++++++++++++++++++++----
> >>  arch/arm64/kvm/hyp/sysreg-sr.c       | 147 ++++++++++++++++++++----------
> >>  arch/arm64/kvm/hyp/timer-sr.c        |  10 +--
> >>  drivers/clocksource/arm_arch_timer.c |  96 ++++++++++++--------
> >>  22 files changed, 724 insertions(+), 272 deletions(-)
> > 
> > Have you tried hw_breakpoint/perf/ptrace with these changes? I was under
> > the impression that the debug architecture was aware of E2H and did need
> > some changes made. I know you say that GDB is broken anyway, but we should
> > check that the kernel does the right thing if userspace pokes it the
> > right way.
> 
> I did use HW breakpoints on the model by hacking the host kernel to
> return Debug Version 6 instead of 7, and things seem to work as
> expected. strace also works out of the box.
> 
> As for perf, did you have something precise in mind?

It would be worth trying things like the filter options on perf events
(perf stat -e cycles:k to count cycles in kernel space) and also
breakpoints (perf stat -e mem:<addr>:rwx on kernel addresses).

Will

[toc] | [prev] | [next] | [standalone]

#1317221

FromMarc Zyngier <marc.zyngier@arm.com>
Date2016-01-25 20:20 +0100
Message-ID<qUUQ9-cl-1@gated-at.bofh.it>
In reply to#1317000
On 25/01/16 16:44, Will Deacon wrote:
> On Mon, Jan 25, 2016 at 04:37:39PM +0000, Marc Zyngier wrote:
>> On 25/01/16 16:26, Will Deacon wrote:
>>> On Mon, Jan 25, 2016 at 03:53:34PM +0000, Marc Zyngier wrote:
>>>> ARMv8.1 comes with the "Virtualization Host Extension" (VHE for
>>>> short), which enables simpler support of Type-2 hypervisors.
>>>>
>>>> This extension allows the kernel to directly run at EL2, and
>>>> significantly reduces the number of system registers shared between
>>>> host and guest, reducing the overhead of virtualization.
>>>>
>>>> In order to have the same kernel binary running on all versions of the
>>>> architecture, this series makes heavy use of runtime code patching.
>>>>
>>>> The first 20 patches massage the KVM code to deal with VHE and enable
>>>> Linux to run at EL2. The last patch catches an ugly case when VHE
>>>> capable CPUs are paired with some of their less capable siblings. This
>>>> should never happen, but hey...
>>>>
>>>> I have deliberately left out some of the more "advanced"
>>>> optimizations, as they are likely to distract the reviewer from the
>>>> core infrastructure, which is what I care about at the moment.
>>>>
>>>> A few things to note:
>>>>
>>>> - Given that the code has been almost entierely rewritten, I've
>>>>   dropped all Acks from the new patches
>>>>
>>>> - GDB is currently busted on VHE systems, as it checks for version 6
>>>>   on the debug architecture, while VHE is version 7. The binutils
>>>>   people are on the case.
>>>
>>> [...]
>>>
>>>>  arch/arm/include/asm/virt.h          |   5 ++
>>>>  arch/arm/kvm/arm.c                   | 151 +++++++++++++++++++------------
>>>>  arch/arm/kvm/mmu.c                   |   7 ++
>>>>  arch/arm64/Kconfig                   |  13 +++
>>>>  arch/arm64/include/asm/cpufeature.h  |   3 +-
>>>>  arch/arm64/include/asm/kvm_arm.h     |   1 +
>>>>  arch/arm64/include/asm/kvm_emulate.h |   3 +
>>>>  arch/arm64/include/asm/kvm_mmu.h     |  34 ++++++-
>>>>  arch/arm64/include/asm/virt.h        |  27 ++++++
>>>>  arch/arm64/kernel/asm-offsets.c      |   3 -
>>>>  arch/arm64/kernel/cpufeature.c       |  15 +++-
>>>>  arch/arm64/kernel/head.S             |  51 ++++++++++-
>>>>  arch/arm64/kernel/smp.c              |   3 +
>>>>  arch/arm64/kvm/hyp-init.S            |  18 +---
>>>>  arch/arm64/kvm/hyp.S                 |   7 ++
>>>>  arch/arm64/kvm/hyp/entry.S           |   6 ++
>>>>  arch/arm64/kvm/hyp/hyp-entry.S       | 107 +++++++---------------
>>>>  arch/arm64/kvm/hyp/hyp.h             | 119 ++++++++++++++++++++++--
>>>>  arch/arm64/kvm/hyp/switch.c          | 170 +++++++++++++++++++++++++++++++----
>>>>  arch/arm64/kvm/hyp/sysreg-sr.c       | 147 ++++++++++++++++++++----------
>>>>  arch/arm64/kvm/hyp/timer-sr.c        |  10 +--
>>>>  drivers/clocksource/arm_arch_timer.c |  96 ++++++++++++--------
>>>>  22 files changed, 724 insertions(+), 272 deletions(-)
>>>
>>> Have you tried hw_breakpoint/perf/ptrace with these changes? I was under
>>> the impression that the debug architecture was aware of E2H and did need
>>> some changes made. I know you say that GDB is broken anyway, but we should
>>> check that the kernel does the right thing if userspace pokes it the
>>> right way.
>>
>> I did use HW breakpoints on the model by hacking the host kernel to
>> return Debug Version 6 instead of 7, and things seem to work as
>> expected. strace also works out of the box.
>>
>> As for perf, did you have something precise in mind?
> 
> It would be worth trying things like the filter options on perf events
> (perf stat -e cycles:k to count cycles in kernel space) and also
> breakpoints (perf stat -e mem:<addr>:rwx on kernel addresses).

So indeed these didn't work (perf reported 0 for kernel accesses). The
fixes are pretty trivial, and I've put them on top of my kvm-arm64/vhe
branch, for those who want to have a look.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web