Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1653150 > unrolled thread

[PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-05-30 13:40 +0200
    Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush Andy Shevchenko <andy.shevchenko@gmail.com> - 2017-05-30 19:00 +0200
      RE: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush Jork Loeser <Jork.Loeser@microsoft.com> - 2017-05-30 21:20 +0200
        Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush Andy Shevchenko <andy.shevchenko@gmail.com> - 2017-05-30 21:30 +0200
        Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB  flush Stephen Hemminger <stephen@networkplumber.org> - 2017-05-31 01:10 +0200

#1653150 — [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush

Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
this is supposed to work faster than IPIs.

Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
we need to put the input somewhere in memory and we don't really want to
have memory allocation on each call so we pre-allocate per cpu memory areas
on boot. These areas are of fixes size, limit them with an arbitrary number
of 16 (16 gvas are able to specify 16 * 4096 pages).

pv_ops patching is happening very early so we need to separate
hyperv_setup_mmu_ops() and hyper_alloc_mmu().

It is possible and easy to implement local TLB flushing too and there is
even a hint for that. However, I don't see a room for optimization on the
host side as both hypercall and native tlb flush will result in vmexit. The
hint is also not set on modern Hyper-V versions.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Acked-by: K. Y. Srinivasan <kys@microsoft.com>
---
Changes since v4:
- Define HV_TLB_FLUSH_UNIT, use __set_bit(), minor code style changes
  [Andy Shevchenko]
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   2 +
 arch/x86/hyperv/mmu.c              | 121 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    |   3 +
 arch/x86/include/uapi/asm/hyperv.h |   7 +++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 6 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/hyperv/mmu.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 171ae09..367a820 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1 +1 @@
-obj-y		:= hv_init.o
+obj-y		:= hv_init.o mmu.o
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 7fd9cd3..df3252f 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -140,6 +140,8 @@ void hyperv_init(void)
 	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
 	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 
+	hyper_alloc_mmu();
+
 	/*
 	 * Register Hyper-V specific clocksource.
 	 */
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
new file mode 100644
index 0000000..8ccd680
--- /dev/null
+++ b/arch/x86/hyperv/mmu.c
@@ -0,0 +1,121 @@
+#include <linux/types.h>
+#include <linux/hyperv.h>
+#include <linux/slab.h>
+#include <linux/log2.h>
+#include <asm/mshyperv.h>
+#include <asm/tlbflush.h>
+#include <asm/msr.h>
+#include <asm/fpu/api.h>
+
+/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
+struct hv_flush_pcpu {
+	__u64 address_space;
+	__u64 flags;
+	__u64 processor_mask;
+	__u64 gva_list[];
+};
+
+/* Each gva in gva_list encodes up to 4096 pages to flush */
+#define HV_TLB_FLUSH_UNIT (PAGE_SIZE * PAGE_SIZE)
+
+static struct hv_flush_pcpu __percpu *pcpu_flush;
+
+static void hyperv_flush_tlb_others(const struct cpumask *cpus,
+				    struct mm_struct *mm, unsigned long start,
+				    unsigned long end)
+{
+	struct hv_flush_pcpu *flush;
+	unsigned long cur, flags;
+	u64 status = U64_MAX;
+	int cpu, vcpu, gva_n, max_gvas;
+
+	if (!pcpu_flush || !hv_hypercall_pg)
+		goto do_native;
+
+	if (cpumask_empty(cpus))
+		return;
+
+	local_irq_save(flags);
+
+	flush = this_cpu_ptr(pcpu_flush);
+
+	if (mm) {
+		flush->address_space = virt_to_phys(mm->pgd);
+		flush->flags = 0;
+	} else {
+		flush->address_space = 0;
+		flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES;
+	}
+
+	flush->processor_mask = 0;
+	if (cpumask_equal(cpus, cpu_present_mask)) {
+		flush->flags |= HV_FLUSH_ALL_PROCESSORS;
+	} else {
+		for_each_cpu(cpu, cpus) {
+			vcpu = hv_cpu_number_to_vp_number(cpu);
+			if (vcpu != -1 && vcpu < 64)
+				__set_bit(vcpu, (unsigned long *)
+					  &flush->processor_mask);
+			else
+				goto do_native;
+		}
+	}
+
+	/*
+	 * We can flush not more than max_gvas with one hypercall. Flush the
+	 * whole address space if we were asked to do more.
+	 */
+	max_gvas = (PAGE_SIZE - sizeof(*flush)) / 8;
+
+	if (end == TLB_FLUSH_ALL) {
+		flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else if (end && ((end - start)/HV_TLB_FLUSH_UNIT) > max_gvas) {
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else {
+		cur = start;
+		gva_n = 0;
+		do {
+			flush->gva_list[gva_n] = cur & PAGE_MASK;
+			/*
+			 * Lower 12 bits encode the number of additional
+			 * pages to flush (in addition to the 'cur' page).
+			 */
+			if (end >= cur + HV_TLB_FLUSH_UNIT)
+				flush->gva_list[gva_n] |= ~PAGE_MASK;
+			else if (end > cur)
+				flush->gva_list[gva_n] |=
+					(end - cur - 1) >> PAGE_SHIFT;
+
+			cur += HV_TLB_FLUSH_UNIT;
+			++gva_n;
+
+		} while (cur < end);
+
+		status = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
+					     gva_n, 0, flush, NULL);
+	}
+
+	local_irq_restore(flags);
+
+	if (!(status & 0xffff))
+		return;
+do_native:
+	native_flush_tlb_others(cpus, mm, start, end);
+}
+
+void hyperv_setup_mmu_ops(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pr_info("Hyper-V: Using hypercall for remote TLB flush\n");
+		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
+	}
+}
+
+void hyper_alloc_mmu(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED)
+		pcpu_flush = __alloc_percpu(PAGE_SIZE, PAGE_SIZE);
+}
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 702abaf..e30c89f 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -307,6 +307,8 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)
 }
 
 void hyperv_init(void);
+void hyperv_setup_mmu_ops(void);
+void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
@@ -317,6 +319,7 @@ static inline bool hv_is_hypercall_page_setup(void)
 	return false;
 }
 static inline hyperv_cleanup(void) {}
+static inline void hyperv_setup_mmu_ops(void) {}
 #endif /* CONFIG_HYPERV */
 
 #ifdef CONFIG_HYPERV_TSCPAGE
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 432df4b..38808f1 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -239,6 +239,8 @@
 		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
 
 /* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
 #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
 #define HVCALL_POST_MESSAGE			0x005c
 #define HVCALL_SIGNAL_EVENT			0x005d
@@ -256,6 +258,11 @@
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
+#define HV_FLUSH_ALL_PROCESSORS			0x00000001
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	0x00000002
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	0x00000004
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	0x00000008
+
 /* hypercall status code */
 #define HV_STATUS_SUCCESS			0
 #define HV_STATUS_INVALID_HYPERCALL_CODE	2
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index bdcc433..da3635f 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -239,6 +239,7 @@ static void __init ms_hyperv_init_platform(void)
 	 * Setup the hook to get control post apic initialization.
 	 */
 	x86_platform.apic_post_init = hyperv_init;
+	hyperv_setup_mmu_ops();
 #endif
 }
 
-- 
2.9.4

[toc] | [next] | [standalone]

#1653396

On Tue, May 30, 2017 at 2:34 PM, Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
> Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
> this is supposed to work faster than IPIs.
>
> Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
> we need to put the input somewhere in memory and we don't really want to
> have memory allocation on each call so we pre-allocate per cpu memory areas
> on boot. These areas are of fixes size, limit them with an arbitrary number
> of 16 (16 gvas are able to specify 16 * 4096 pages).
>
> pv_ops patching is happening very early so we need to separate
> hyperv_setup_mmu_ops() and hyper_alloc_mmu().
>
> It is possible and easy to implement local TLB flushing too and there is
> even a hint for that. However, I don't see a room for optimization on the
> host side as both hypercall and native tlb flush will result in vmexit. The
> hint is also not set on modern Hyper-V versions.

> @@ -0,0 +1,121 @@
> +#include <linux/types.h>
> +#include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <linux/log2.h>

Alphabetical order, please.

+ empty line

> +#include <asm/mshyperv.h>
> +#include <asm/tlbflush.h>
> +#include <asm/msr.h>
> +#include <asm/fpu/api.h>

Can be alphabetically ordered?

> +/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
> +struct hv_flush_pcpu {
> +       __u64 address_space;
> +       __u64 flags;
> +       __u64 processor_mask;
> +       __u64 gva_list[];
> +};

I dunno what is the style there, but usually in Linux __uXX types are
used exclusively for User API.
Is it a case here? Can we use plain uXX types instead?

> +/* Each gva in gva_list encodes up to 4096 pages to flush */
> +#define HV_TLB_FLUSH_UNIT (PAGE_SIZE * PAGE_SIZE)

Regarding to the comment it would be rather
(4096 * PAGE_SIZE)

Yes, theoretically PAGE_SIZE can be not 4096.

> +static void hyperv_flush_tlb_others(const struct cpumask *cpus,
> +                                   struct mm_struct *mm, unsigned long start,
> +                                   unsigned long end)
> +{

> +       if (cpumask_equal(cpus, cpu_present_mask)) {
> +               flush->flags |= HV_FLUSH_ALL_PROCESSORS;
> +       } else {
> +               for_each_cpu(cpu, cpus) {
> +                       vcpu = hv_cpu_number_to_vp_number(cpu);

> +                       if (vcpu != -1 && vcpu < 64)

Just
if (vcpu < 64)
?

> +                               __set_bit(vcpu, (unsigned long *)
> +                                         &flush->processor_mask);
> +                       else
> +                               goto do_native;
> +               }
> +       }

> +       if (end == TLB_FLUSH_ALL) {
> +               flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
> +               status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
> +                                        flush, NULL);
> +       } else if (end && ((end - start)/HV_TLB_FLUSH_UNIT) > max_gvas) {
> +               status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
> +                                        flush, NULL);

Yes! Looks much more cleaner.

> +       } else {
> +               cur = start;
> +               gva_n = 0;
> +               do {
> +                       flush->gva_list[gva_n] = cur & PAGE_MASK;

> +                       /*
> +                        * Lower 12 bits encode the number of additional
> +                        * pages to flush (in addition to the 'cur' page).
> +                        */
> +                       if (end >= cur + HV_TLB_FLUSH_UNIT)
> +                               flush->gva_list[gva_n] |= ~PAGE_MASK;
> +                       else if (end > cur)
> +                               flush->gva_list[gva_n] |=
> +                                       (end - cur - 1) >> PAGE_SHIFT;

You can also simplify this slightly by introducing

unsigned long diff = end > cur ? end - cur : 0;

if (diff >= HV_TLB_FLUSH_UNIT)
    flush->gva_list[gva_n] |= ~PAGE_MASK;
else if (diff)
    flush->gva_list[gva_n] |= (diff - 1) >> PAGE_SHIFT;

> +
> +                       cur += HV_TLB_FLUSH_UNIT;

> +                       ++gva_n;

Make it post-increment. Better for reader (No need to pay an
additional attention why it's a pre-increment)

> +
> +               } while (cur < end);

> +       if (!(status & 0xffff))

Not first time I see this magic.

Perhaps

#define STATUS_BLA_BLA_MASK GENMASK(15,0)

if (!(status & STATUS_BLA_BLA_MASK))

in all appropriate places?

> +#define HV_FLUSH_ALL_PROCESSORS                        0x00000001
> +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES    0x00000002
> +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY      0x00000004
> +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT     0x00000008

BIT() ?

-- 
With Best Regards,
Andy Shevchenko

[toc] | [prev] | [next] | [standalone]

#1653509

> -----Original Message-----
> From: Andy Shevchenko [mailto:andy.shevchenko@gmail.com]
> Sent: Tuesday, May 30, 2017 09:53
> To: Vitaly Kuznetsov <vkuznets@redhat.com>
> Cc: x86@kernel.org; devel@linuxdriverproject.org; linux-
> kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>; Haiyang Zhang
> <haiyangz@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> Thomas Gleixner <tglx@linutronix.de>; Ingo Molnar <mingo@redhat.com>; H.
> Peter Anvin <hpa@zytor.com>; Steven Rostedt <rostedt@goodmis.org>; Jork
> Loeser <Jork.Loeser@microsoft.com>; Simon Xiao <sixiao@microsoft.com>;
> Andy Lutomirski <luto@kernel.org>
> Subject: Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush
> 
> On Tue, May 30, 2017 at 2:34 PM, Vitaly Kuznetsov <vkuznets@redhat.com>
> wrote:
> > +#define HV_FLUSH_ALL_PROCESSORS                        0x00000001
> > +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES    0x00000002
> > +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY      0x00000004
> > +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT     0x00000008
> 
> BIT() ?

Certainly a matter of taste. Given that the Hyper-V spec lists these as hex numbers, I find the explicit numbers appropriate.

Regards,
Jork

[toc] | [prev] | [next] | [standalone]

#1653514

On Tue, May 30, 2017 at 10:17 PM, Jork Loeser <Jork.Loeser@microsoft.com> wrote:

>> > +#define HV_FLUSH_ALL_PROCESSORS                        0x00000001
>> > +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES    0x00000002
>> > +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY      0x00000004
>> > +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT     0x00000008
>>
>> BIT() ?
>
> Certainly a matter of taste.

That's why ? is used, though slightly better to parse the BIT macros
which is also less error prone.

> Given that the Hyper-V spec lists these as hex numbers, I find the explicit numbers appropriate.

Yes, but since it introduces a full set of the flags I would not see
disadvantages by style.

-- 
With Best Regards,
Andy Shevchenko

[toc] | [prev] | [next] | [standalone]

#1653709 — Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush

On Tue, 30 May 2017 19:17:46 +0000
Jork Loeser <Jork.Loeser@microsoft.com> wrote:

> > -----Original Message-----
> > From: Andy Shevchenko [mailto:andy.shevchenko@gmail.com]
> > Sent: Tuesday, May 30, 2017 09:53
> > To: Vitaly Kuznetsov <vkuznets@redhat.com>
> > Cc: x86@kernel.org; devel@linuxdriverproject.org; linux-
> > kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>; Haiyang Zhang
> > <haiyangz@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> > Thomas Gleixner <tglx@linutronix.de>; Ingo Molnar <mingo@redhat.com>; H.
> > Peter Anvin <hpa@zytor.com>; Steven Rostedt <rostedt@goodmis.org>; Jork
> > Loeser <Jork.Loeser@microsoft.com>; Simon Xiao <sixiao@microsoft.com>;
> > Andy Lutomirski <luto@kernel.org>
> > Subject: Re: [PATCH v5 08/10] x86/hyper-v: use hypercall for remote TLB flush
> > 
> > On Tue, May 30, 2017 at 2:34 PM, Vitaly Kuznetsov <vkuznets@redhat.com>
> > wrote:  
> > > +#define HV_FLUSH_ALL_PROCESSORS                        0x00000001
> > > +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES    0x00000002
> > > +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY      0x00000004
> > > +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT     0x00000008  
> > 
> > BIT() ?  
> 
> Certainly a matter of taste. Given that the Hyper-V spec lists these as hex numbers, I find the explicit numbers appropriate.
> 
> Regards,
> Jork

Keep the hex numbers, it makes more sense not to change it since rest of arch/x86/hyperv
uses hex values.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web