Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1618690 > unrolled thread

[PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and hypercall improvements

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and hypercall improvements Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-07 13:30 +0200
    [PATCH 3/7] hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-07 13:30 +0200
    [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-07 13:30 +0200
      RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush Jork Loeser <Jork.Loeser@microsoft.com> - 2017-04-07 22:50 +0200
        Re: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-10 19:30 +0200
      RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush KY Srinivasan <kys@microsoft.com> - 2017-04-08 18:50 +0200
        Re: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-10 16:50 +0200
          RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush Jork Loeser <Jork.Loeser@microsoft.com> - 2017-04-10 19:40 +0200
          RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush KY Srinivasan <kys@microsoft.com> - 2017-04-11 00:10 +0200
    [PATCH 4/7] x86/hyperv: implement rep hypercalls Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-07 13:30 +0200
      RE: [PATCH 4/7] x86/hyperv: implement rep hypercalls Jork Loeser <Jork.Loeser@microsoft.com> - 2017-04-07 21:50 +0200
        Re: [PATCH 4/7] x86/hyperv: implement rep hypercalls Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-10 11:10 +0200
    [PATCH 5/7] hyper-v: globalize vp_index Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-04-07 13:30 +0200
      RE: [PATCH 5/7] hyper-v: globalize vp_index KY Srinivasan <kys@microsoft.com> - 2017-04-08 17:50 +0200
    RE: [PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and  hypercall improvements KY Srinivasan <kys@microsoft.com> - 2017-04-08 17:00 +0200

#1618690 — [PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and hypercall improvements

Hi,

Hyper-V supports hypercalls for doing local and remote TLB flushing and
gives its guests hints when using hypercall is preferred. While doing
hypercalls for local TLB flushes is probably not practical (and is not
being suggested by modern Hyper-V versions) remote TLB flush with a
hypercall brings significant improvement.

To test the series I wrote a special 'TLB trasher': on a 16 vCPU guest I
was creating 32 threads which were doing 100000 mmap/munmaps each on some
big file. Here are the results:

Before:
# time ./pthread_mmap ./randfile 
real	3m44.994s
user	0m3.829s
sys	3m36.323s

After:
# time ./pthread_mmap ./randfile 
real	2m57.145s
user	0m3.797s
sys	2m34.812s

This series brings a number of small improvements along the way: fast
hypercall implementation and using it for event signaling, rep hypercalls
implementation, hyperv tracing subsystem (which only traces the newly added
remote TLB flush for now).

Vitaly Kuznetsov (7):
  x86/hyperv: make hv_do_hypercall() inline
  x86/hyper-v: fast hypercall implementation
  hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT
  x86/hyperv: implement rep hypercalls
  hyper-v: globalize vp_index
  x86/hyper-v: use hypercall for remove TLB flush
  tracing/hyper-v: trace hyperv_mmu_flush_tlb_others()

 MAINTAINERS                        |   1 +
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |  90 +++++++++++--------------
 arch/x86/hyperv/mmu.c              | 134 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    | 131 ++++++++++++++++++++++++++++++++++++
 arch/x86/include/uapi/asm/hyperv.h |  26 +++++++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 drivers/hv/channel_mgmt.c          |  22 +++---
 drivers/hv/connection.c            |   8 ++-
 drivers/hv/hv.c                    |   9 ---
 drivers/hv/hyperv_vmbus.h          |  11 ---
 drivers/hv/vmbus_drv.c             |  17 -----
 include/linux/hyperv.h             |  21 +++---
 include/trace/events/hyperv.h      |  30 +++++++++
 14 files changed, 386 insertions(+), 117 deletions(-)
 create mode 100644 arch/x86/hyperv/mmu.c
 create mode 100644 include/trace/events/hyperv.h

-- 
2.9.3

[toc] | [next] | [standalone]

#1618691 — [PATCH 3/7] hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT

We need to pass only 8 bytes of input for HvSignalEvent which makes it a
perfect fit for fast hypercall. hv_input_signal_event_buffer is not needed
any more and hv_input_signal_event is converted to union for convenience.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 drivers/hv/channel_mgmt.c | 15 +++++----------
 drivers/hv/connection.c   |  3 ++-
 include/linux/hyperv.h    | 19 ++++++++-----------
 3 files changed, 15 insertions(+), 22 deletions(-)

diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 735f936..6cfa297 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -804,20 +804,15 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
 	/*
 	 * Setup state for signalling the host.
 	 */
-	newchannel->sig_event = (struct hv_input_signal_event *)
-				(ALIGN((unsigned long)
-				&newchannel->sig_buf,
-				HV_HYPERCALL_PARAM_ALIGN));
-
-	newchannel->sig_event->connectionid.asu32 = 0;
-	newchannel->sig_event->connectionid.u.id = VMBUS_EVENT_CONNECTION_ID;
-	newchannel->sig_event->flag_number = 0;
-	newchannel->sig_event->rsvdz = 0;
+	newchannel->sig_event.connectionid.asu32 = 0;
+	newchannel->sig_event.connectionid.u.id = VMBUS_EVENT_CONNECTION_ID;
+	newchannel->sig_event.flag_number = 0;
+	newchannel->sig_event.rsvdz = 0;
 
 	if (vmbus_proto_version != VERSION_WS2008) {
 		newchannel->is_dedicated_interrupt =
 				(offer->is_dedicated_interrupt != 0);
-		newchannel->sig_event->connectionid.u.id =
+		newchannel->sig_event.connectionid.u.id =
 				offer->connection_id;
 	}
 
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index cf77c7f..545f2a4 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -405,6 +405,7 @@ void vmbus_set_event(struct vmbus_channel *channel)
 	if (!channel->is_dedicated_interrupt)
 		vmbus_send_interrupt(child_relid);
 
-	hv_do_hypercall(HVCALL_SIGNAL_EVENT, channel->sig_event, NULL);
+	hv_do_fast_hypercall8(HVCALL_SIGNAL_EVENT,
+			      channel->sig_event.as_uint64);
 }
 EXPORT_SYMBOL_GPL(vmbus_set_event);
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 677f084..5d6777c 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -678,15 +678,13 @@ union hv_connection_id {
 };
 
 /* Definition of the hv_signal_event hypercall input structure. */
-struct hv_input_signal_event {
-	union hv_connection_id connectionid;
-	u16 flag_number;
-	u16 rsvdz;
-};
-
-struct hv_input_signal_event_buffer {
-	u64 align8;
-	struct hv_input_signal_event event;
+union hv_input_signal_event {
+	u64 as_uint64;
+	struct {
+		union hv_connection_id connectionid;
+		u16 flag_number;
+		u16 rsvdz;
+	};
 };
 
 enum hv_numa_policy {
@@ -771,8 +769,7 @@ struct vmbus_channel {
 	} callback_mode;
 
 	bool is_dedicated_interrupt;
-	struct hv_input_signal_event_buffer sig_buf;
-	struct hv_input_signal_event *sig_event;
+	union hv_input_signal_event sig_event;
 
 	/*
 	 * Starting with win8, this field will be used to specify
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1618692 — [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
this is supposed to work faster than IPIs.

Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
we need to put the input somewhere in memory and we don't really want to
have memory allocation on each call so we pre-allocate per cpu memory areas
on boot. These areas are of fixes size, limit them with an arbitrary number
of 16 (16 gvas are able to specify 16 * 4096 pages).

pv_ops patching is happening very early so we need to separate
hyperv_setup_mmu_ops() and hyper_alloc_mmu().

It is possible and easy to implement local TLB flushing too and there is
even a hint for that. However, I don't see a room for optimization on the
host side as both hypercall and native tlb flush will result in vmexit. The
hint is also not set on modern Hyper-V versions.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   2 +
 arch/x86/hyperv/mmu.c              | 128 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    |   2 +
 arch/x86/include/uapi/asm/hyperv.h |   7 ++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 6 files changed, 141 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/hyperv/mmu.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 171ae09..367a820 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1 +1 @@
-obj-y		:= hv_init.o
+obj-y		:= hv_init.o mmu.o
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 1c14088..2cf8a98 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -163,6 +163,8 @@ void hyperv_init(void)
 	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
 	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 
+	hyper_alloc_mmu();
+
 	/*
 	 * Register Hyper-V specific clocksource.
 	 */
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
new file mode 100644
index 0000000..fb487cb
--- /dev/null
+++ b/arch/x86/hyperv/mmu.c
@@ -0,0 +1,128 @@
+#include <linux/types.h>
+#include <linux/hyperv.h>
+#include <linux/slab.h>
+#include <asm/mshyperv.h>
+#include <asm/tlbflush.h>
+#include <asm/msr.h>
+#include <asm/fpu/api.h>
+
+/*
+ * Arbitrary number; we need to pre-allocate per-cpu struct for doing TLB
+ * flush hypercalls and we need to pick a size. '16' means we'll be able
+ * to flush 16 * 4096 pages (256MB) with one hypercall.
+ */
+#define HV_MMU_MAX_GVAS 16
+
+/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls */
+struct hv_flush_pcpu {
+	struct {
+		__u64 address_space;
+		__u64 flags;
+		__u64 processor_mask;
+		__u64 gva_list[HV_MMU_MAX_GVAS];
+	} flush;
+
+	spinlock_t lock;
+};
+
+static struct hv_flush_pcpu __percpu *pcpu_flush;
+
+static void hyperv_flush_tlb_others(const struct cpumask *cpus,
+				    struct mm_struct *mm, unsigned long start,
+				    unsigned long end)
+{
+	struct hv_flush_pcpu *flush;
+	unsigned long cur, flags;
+	u64 status = -1ULL;
+	int cpu, vcpu, gva_n;
+
+	if (!pcpu_flush || !hv_hypercall_pg)
+		goto do_native;
+
+	if (cpumask_empty(cpus))
+		return;
+
+	flush = this_cpu_ptr(pcpu_flush);
+	spin_lock_irqsave(&flush->lock, flags);
+
+	flush->flush.address_space = virt_to_phys(mm->pgd);
+	flush->flush.processor_mask = 0;
+	if (cpumask_equal(cpus, cpu_present_mask)) {
+		flush->flush.flags = HV_FLUSH_ALL_PROCESSORS;
+	} else {
+		flush->flush.flags = 0;
+		for_each_cpu(cpu, cpus) {
+			vcpu = vmbus_cpu_number_to_vp_number(cpu);
+			if (vcpu != -1 && vcpu < 64)
+				flush->flush.processor_mask |= 1 << vcpu;
+			else
+				goto unlock_do_native;
+		}
+	}
+
+	if (end == TLB_FLUSH_ALL) {
+		flush->flush.flags = HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 &flush->flush, NULL);
+	} else {
+		cur = start;
+more_gvas:
+		gva_n = 0;
+
+		do {
+			flush->flush.gva_list[gva_n] = cur & PAGE_MASK;
+			/*
+			 * Lower 12 bits encode the number of additional
+			 * pages to flush (in addition to the 'cur' page).
+			 */
+			if (end >= cur + PAGE_SIZE * PAGE_SIZE)
+				flush->flush.gva_list[gva_n] |= ~PAGE_MASK;
+			else if (end > cur)
+				flush->flush.gva_list[gva_n] |=
+					(end - cur - 1) >> PAGE_SHIFT;
+
+			cur += PAGE_SIZE * PAGE_SIZE;
+			++gva_n;
+
+		} while (cur < end && gva_n < HV_MMU_MAX_GVAS);
+
+		status = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
+					     gva_n, &flush->flush, NULL);
+
+		if (!(status & 0xffff) && cur < end)
+			goto more_gvas;
+	}
+
+unlock_do_native:
+	spin_unlock_irqrestore(&flush->lock, flags);
+
+	if (!(status & 0xffff))
+		return;
+do_native:
+	native_flush_tlb_others(cpus, mm, start, end);
+}
+
+void hyperv_setup_mmu_ops(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pr_info("Hyper-V: Using hypercall for remote TLB flush\n");
+		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
+	}
+}
+
+void hyper_alloc_mmu(void)
+{
+	int cpu;
+	struct hv_flush_pcpu *flush;
+
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pcpu_flush = alloc_percpu(struct hv_flush_pcpu);
+		if (!pcpu_flush)
+			return;
+
+		for_each_possible_cpu(cpu) {
+			flush = per_cpu_ptr(pcpu_flush, cpu);
+			spin_lock_init(&flush->lock);
+		}
+	}
+}
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 1293c84..a5041c3 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -301,6 +301,8 @@ static inline int vmbus_cpu_number_to_vp_number(int cpu_number)
 }
 
 void hyperv_init(void);
+void hyperv_setup_mmu_ops(void);
+void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index c87e900..3d44036 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -239,6 +239,8 @@
 		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
 
 /* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
 #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
 #define HVCALL_POST_MESSAGE			0x005c
 #define HVCALL_SIGNAL_EVENT			0x005d
@@ -256,6 +258,11 @@
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
+#define HV_FLUSH_ALL_PROCESSORS			0x00000001
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	0x00000002
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	0x00000004
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	0x00000008
+
 /* Hypercall interface */
 union hv_hypercall_input {
 	u64 as_uint64;
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 04cb8d3..fc228d8 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -233,6 +233,7 @@ static void __init ms_hyperv_init_platform(void)
 	 * Setup the hook to get control post apic initialization.
 	 */
 	x86_platform.apic_post_init = hyperv_init;
+	hyperv_setup_mmu_ops();
 #endif
 }
 
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1619093 — RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Friday, April 7, 2017 04:27
> To: devel@linuxdriverproject.org; x86@kernel.org
> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
> Subject: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

> diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c new file
> mode 100644 index 0000000..fb487cb
> --- /dev/null
> +++ b/arch/x86/hyperv/mmu.c
> @@ -0,0 +1,128 @@
> +#include <linux/types.h>
> +#include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <asm/mshyperv.h>
> +#include <asm/tlbflush.h>
> +#include <asm/msr.h>
> +#include <asm/fpu/api.h>
> +
> +/*
> + * Arbitrary number; we need to pre-allocate per-cpu struct for doing
> +TLB
> + * flush hypercalls and we need to pick a size. '16' means we'll be
> +able
> + * to flush 16 * 4096 pages (256MB) with one hypercall.
> + */
> +#define HV_MMU_MAX_GVAS 16
> +
> +/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls */
> +struct hv_flush_pcpu {
> +	struct {
> +		__u64 address_space;
> +		__u64 flags;
> +		__u64 processor_mask;
> +		__u64 gva_list[HV_MMU_MAX_GVAS];
> +	} flush;
> +
> +	spinlock_t lock;
> +};
Does this need an alignment declaration, so that the flush portion never crosses a page boundary when allocated with alloc_percpu()?

> +
> +static struct hv_flush_pcpu __percpu *pcpu_flush;
> +
> +static void hyperv_flush_tlb_others(const struct cpumask *cpus,
> +				    struct mm_struct *mm, unsigned long
> start,
> +				    unsigned long end)
> +{
> +	struct hv_flush_pcpu *flush;
> +	unsigned long cur, flags;
> +	u64 status = -1ULL;
> +	int cpu, vcpu, gva_n;
> +
> +	if (!pcpu_flush || !hv_hypercall_pg)
> +		goto do_native;
> +
> +	if (cpumask_empty(cpus))
> +		return;
> +
> +	flush = this_cpu_ptr(pcpu_flush);
> +	spin_lock_irqsave(&flush->lock, flags);

What purpose does the spinlock on the CPU-local struct serve? Would a local_irq_save() do?

Could this be called from NMI context, such as from the debugger?

Could this be a long-running loop, e.g. due to a large start/end range? If so, consider disabling interrupts only in the inner loop / flush the entire space?

Regards,
Jork

[toc] | [prev] | [next] | [standalone]

#1620463 — Re: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

Jork Loeser <Jork.Loeser@microsoft.com> writes:

>> -----Original Message-----
>> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
>> Sent: Friday, April 7, 2017 04:27
>> To: devel@linuxdriverproject.org; x86@kernel.org
>> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
>> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
>> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
>> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
>> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
>> Subject: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush
>
>> diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c new file
>> mode 100644 index 0000000..fb487cb
>> --- /dev/null
>> +++ b/arch/x86/hyperv/mmu.c
>> @@ -0,0 +1,128 @@
>> +#include <linux/types.h>
>> +#include <linux/hyperv.h>
>> +#include <linux/slab.h>
>> +#include <asm/mshyperv.h>
>> +#include <asm/tlbflush.h>
>> +#include <asm/msr.h>
>> +#include <asm/fpu/api.h>
>> +
>> +/*
>> + * Arbitrary number; we need to pre-allocate per-cpu struct for doing
>> +TLB
>> + * flush hypercalls and we need to pick a size. '16' means we'll be
>> +able
>> + * to flush 16 * 4096 pages (256MB) with one hypercall.
>> + */
>> +#define HV_MMU_MAX_GVAS 16
>> +
>> +/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls */
>> +struct hv_flush_pcpu {
>> +	struct {
>> +		__u64 address_space;
>> +		__u64 flags;
>> +		__u64 processor_mask;
>> +		__u64 gva_list[HV_MMU_MAX_GVAS];
>> +	} flush;
>> +
>> +	spinlock_t lock;
>> +};
> Does this need an alignment declaration, so that the flush portion never crosses a page boundary when allocated with alloc_percpu()?
>

Thanks for pointing this out! I would slightly prefer we use
__alloc_percpu() and specify something like roundup_pow_of_two()
alignment.

>> +
>> +static struct hv_flush_pcpu __percpu *pcpu_flush;
>> +
>> +static void hyperv_flush_tlb_others(const struct cpumask *cpus,
>> +				    struct mm_struct *mm, unsigned long
>> start,
>> +				    unsigned long end)
>> +{
>> +	struct hv_flush_pcpu *flush;
>> +	unsigned long cur, flags;
>> +	u64 status = -1ULL;
>> +	int cpu, vcpu, gva_n;
>> +
>> +	if (!pcpu_flush || !hv_hypercall_pg)
>> +		goto do_native;
>> +
>> +	if (cpumask_empty(cpus))
>> +		return;
>> +
>> +	flush = this_cpu_ptr(pcpu_flush);
>> +	spin_lock_irqsave(&flush->lock, flags);
>
> What purpose does the spinlock on the CPU-local struct serve? Would a
> local_irq_save() do?

Now I'm not sure why I put it here in the first place :-) Yes, it would
probably do.

> Could this be called from NMI context, such as from the debugger?
>

NMI - I don't think so, native function does smp_call_function_many()
which WARNs even if it's called with interrupts disabled.

> Could this be a long-running loop, e.g. due to a large start/end
> range? If so, consider disabling interrupts only in the inner loop /
> flush the entire space?

The decision for flushing the entire space should probably be done
elsewhere as it is not implementation-specific (and I think it's done
somewhere as I never see requests to flush more than 4096 pages in my
testing).

I can disable interrupts in the inner loop but we'll have to stash flags
and calculated cpu_mask to some local variables. This is not supposed to
be expensive.

-- 
  Vitaly

[toc] | [prev] | [next] | [standalone]

#1619330 — RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Friday, April 7, 2017 4:27 AM
> To: devel@linuxdriverproject.org; x86@kernel.org
> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
> Subject: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush
> 
> Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
> this is supposed to work faster than IPIs.
> 
> Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
> we need to put the input somewhere in memory and we don't really want to
> have memory allocation on each call so we pre-allocate per cpu memory
> areas
> on boot. These areas are of fixes size, limit them with an arbitrary number
> of 16 (16 gvas are able to specify 16 * 4096 pages).
> 
> pv_ops patching is happening very early so we need to separate
> hyperv_setup_mmu_ops() and hyper_alloc_mmu().
> 
> It is possible and easy to implement local TLB flushing too and there is
> even a hint for that. However, I don't see a room for optimization on the
> host side as both hypercall and native tlb flush will result in vmexit. The
> hint is also not set on modern Hyper-V versions.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  arch/x86/hyperv/Makefile           |   2 +-
>  arch/x86/hyperv/hv_init.c          |   2 +
>  arch/x86/hyperv/mmu.c              | 128
> +++++++++++++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h    |   2 +
>  arch/x86/include/uapi/asm/hyperv.h |   7 ++
>  arch/x86/kernel/cpu/mshyperv.c     |   1 +
>  6 files changed, 141 insertions(+), 1 deletion(-)
>  create mode 100644 arch/x86/hyperv/mmu.c
> 
> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> index 171ae09..367a820 100644
> --- a/arch/x86/hyperv/Makefile
> +++ b/arch/x86/hyperv/Makefile
> @@ -1 +1 @@
> -obj-y		:= hv_init.o
> +obj-y		:= hv_init.o mmu.o
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index 1c14088..2cf8a98 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -163,6 +163,8 @@ void hyperv_init(void)
>  	hypercall_msr.guest_physical_address =
> vmalloc_to_pfn(hv_hypercall_pg);
>  	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> 
> +	hyper_alloc_mmu();
> +
>  	/*
>  	 * Register Hyper-V specific clocksource.
>  	 */
> diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
> new file mode 100644
> index 0000000..fb487cb
> --- /dev/null
> +++ b/arch/x86/hyperv/mmu.c
> @@ -0,0 +1,128 @@
> +#include <linux/types.h>
> +#include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <asm/mshyperv.h>
> +#include <asm/tlbflush.h>
> +#include <asm/msr.h>
> +#include <asm/fpu/api.h>
> +
> +/*
> + * Arbitrary number; we need to pre-allocate per-cpu struct for doing TLB
> + * flush hypercalls and we need to pick a size. '16' means we'll be able
> + * to flush 16 * 4096 pages (256MB) with one hypercall.
> + */
> +#define HV_MMU_MAX_GVAS 16

Did you experiment with different sizes here.
> +
> +/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls */
> +struct hv_flush_pcpu {
> +	struct {
> +		__u64 address_space;
> +		__u64 flags;
> +		__u64 processor_mask;
> +		__u64 gva_list[HV_MMU_MAX_GVAS];
> +	} flush;
> +
> +	spinlock_t lock;
> +};
> +
We may be supporting more than 64 CPUs in this hypercall. I am going to inquire with
the Windows folks and get back to you.

> +static struct hv_flush_pcpu __percpu *pcpu_flush;
> +
> +static void hyperv_flush_tlb_others(const struct cpumask *cpus,
> +				    struct mm_struct *mm, unsigned long
> start,
> +				    unsigned long end)
> +{
> +	struct hv_flush_pcpu *flush;
> +	unsigned long cur, flags;
> +	u64 status = -1ULL;
> +	int cpu, vcpu, gva_n;
> +
> +	if (!pcpu_flush || !hv_hypercall_pg)
> +		goto do_native;
> +
> +	if (cpumask_empty(cpus))
> +		return;
> +
> +	flush = this_cpu_ptr(pcpu_flush);
> +	spin_lock_irqsave(&flush->lock, flags);
> +
> +	flush->flush.address_space = virt_to_phys(mm->pgd);
> +	flush->flush.processor_mask = 0;
> +	if (cpumask_equal(cpus, cpu_present_mask)) {
> +		flush->flush.flags = HV_FLUSH_ALL_PROCESSORS;
> +	} else {
> +		flush->flush.flags = 0;
> +		for_each_cpu(cpu, cpus) {
> +			vcpu = vmbus_cpu_number_to_vp_number(cpu);
> +			if (vcpu != -1 && vcpu < 64)
> +				flush->flush.processor_mask |= 1 << vcpu;
> +			else
> +				goto unlock_do_native;
> +		}
> +	}
> +
> +	if (end == TLB_FLUSH_ALL) {
> +		flush->flush.flags =
> HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
> +		status =
> hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
> +					 &flush->flush, NULL);
> +	} else {
> +		cur = start;
> +more_gvas:
> +		gva_n = 0;
> +
> +		do {
> +			flush->flush.gva_list[gva_n] = cur & PAGE_MASK;
> +			/*
> +			 * Lower 12 bits encode the number of additional
> +			 * pages to flush (in addition to the 'cur' page).
> +			 */
> +			if (end >= cur + PAGE_SIZE * PAGE_SIZE)
> +				flush->flush.gva_list[gva_n] |=
> ~PAGE_MASK;
> +			else if (end > cur)
> +				flush->flush.gva_list[gva_n] |=
> +					(end - cur - 1) >> PAGE_SHIFT;
> +
> +			cur += PAGE_SIZE * PAGE_SIZE;
> +			++gva_n;
> +
> +		} while (cur < end && gva_n < HV_MMU_MAX_GVAS);
> +
> +		status =
> hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
> +					     gva_n, &flush->flush, NULL);
> +
> +		if (!(status & 0xffff) && cur < end)
> +			goto more_gvas;
> +	}
> +
> +unlock_do_native:
> +	spin_unlock_irqrestore(&flush->lock, flags);
> +
> +	if (!(status & 0xffff))
> +		return;
> +do_native:
> +	native_flush_tlb_others(cpus, mm, start, end);
> +}
> +
> +void hyperv_setup_mmu_ops(void)
> +{
> +	if (ms_hyperv.hints &
> HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
> +		pr_info("Hyper-V: Using hypercall for remote TLB flush\n");
> +		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
> +	}
> +}
> +
> +void hyper_alloc_mmu(void)
> +{
> +	int cpu;
> +	struct hv_flush_pcpu *flush;
> +
> +	if (ms_hyperv.hints &
> HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
> +		pcpu_flush = alloc_percpu(struct hv_flush_pcpu);
> +		if (!pcpu_flush)
> +			return;
> +
> +		for_each_possible_cpu(cpu) {
> +			flush = per_cpu_ptr(pcpu_flush, cpu);
> +			spin_lock_init(&flush->lock);
> +		}
> +	}
> +}
> diff --git a/arch/x86/include/asm/mshyperv.h
> b/arch/x86/include/asm/mshyperv.h
> index 1293c84..a5041c3 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -301,6 +301,8 @@ static inline int
> vmbus_cpu_number_to_vp_number(int cpu_number)
>  }
> 
>  void hyperv_init(void);
> +void hyperv_setup_mmu_ops(void);
> +void hyper_alloc_mmu(void);
>  void hyperv_report_panic(struct pt_regs *regs);
>  bool hv_is_hypercall_page_setup(void);
>  void hyperv_cleanup(void);
> diff --git a/arch/x86/include/uapi/asm/hyperv.h
> b/arch/x86/include/uapi/asm/hyperv.h
> index c87e900..3d44036 100644
> --- a/arch/x86/include/uapi/asm/hyperv.h
> +++ b/arch/x86/include/uapi/asm/hyperv.h
> @@ -239,6 +239,8 @@
>  		(~((1ull <<
> HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
> 
>  /* Declare the various hypercall operations. */
> +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
> +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
>  #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
>  #define HVCALL_POST_MESSAGE			0x005c
>  #define HVCALL_SIGNAL_EVENT			0x005d
> @@ -256,6 +258,11 @@
>  #define HV_PROCESSOR_POWER_STATE_C2		2
>  #define HV_PROCESSOR_POWER_STATE_C3		3
> 
> +#define HV_FLUSH_ALL_PROCESSORS			0x00000001
> +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	0x00000002
> +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	0x00000004
> +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	0x00000008
> +
>  /* Hypercall interface */
>  union hv_hypercall_input {
>  	u64 as_uint64;
> diff --git a/arch/x86/kernel/cpu/mshyperv.c
> b/arch/x86/kernel/cpu/mshyperv.c
> index 04cb8d3..fc228d8 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -233,6 +233,7 @@ static void __init ms_hyperv_init_platform(void)
>  	 * Setup the hook to get control post apic initialization.
>  	 */
>  	x86_platform.apic_post_init = hyperv_init;
> +	hyperv_setup_mmu_ops();
>  #endif
>  }
> 
> --
> 2.9.3

[toc] | [prev] | [next] | [standalone]

#1619990 — Re: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

KY Srinivasan <kys@microsoft.com> writes:

>> -----Original Message-----
>> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
>> Sent: Friday, April 7, 2017 4:27 AM
>> To: devel@linuxdriverproject.org; x86@kernel.org
>> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
>> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
>> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
>> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
>> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
>> Subject: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush
>> 
>> Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
>> this is supposed to work faster than IPIs.
>> 
>> Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
>> we need to put the input somewhere in memory and we don't really want to
>> have memory allocation on each call so we pre-allocate per cpu memory
>> areas
>> on boot. These areas are of fixes size, limit them with an arbitrary number
>> of 16 (16 gvas are able to specify 16 * 4096 pages).
>> 
>> pv_ops patching is happening very early so we need to separate
>> hyperv_setup_mmu_ops() and hyper_alloc_mmu().
>> 
>> It is possible and easy to implement local TLB flushing too and there is
>> even a hint for that. However, I don't see a room for optimization on the
>> host side as both hypercall and native tlb flush will result in vmexit. The
>> hint is also not set on modern Hyper-V versions.
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  arch/x86/hyperv/Makefile           |   2 +-
>>  arch/x86/hyperv/hv_init.c          |   2 +
>>  arch/x86/hyperv/mmu.c              | 128
>> +++++++++++++++++++++++++++++++++++++
>>  arch/x86/include/asm/mshyperv.h    |   2 +
>>  arch/x86/include/uapi/asm/hyperv.h |   7 ++
>>  arch/x86/kernel/cpu/mshyperv.c     |   1 +
>>  6 files changed, 141 insertions(+), 1 deletion(-)
>>  create mode 100644 arch/x86/hyperv/mmu.c
>> 
>> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
>> index 171ae09..367a820 100644
>> --- a/arch/x86/hyperv/Makefile
>> +++ b/arch/x86/hyperv/Makefile
>> @@ -1 +1 @@
>> -obj-y		:= hv_init.o
>> +obj-y		:= hv_init.o mmu.o
>> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
>> index 1c14088..2cf8a98 100644
>> --- a/arch/x86/hyperv/hv_init.c
>> +++ b/arch/x86/hyperv/hv_init.c
>> @@ -163,6 +163,8 @@ void hyperv_init(void)
>>  	hypercall_msr.guest_physical_address =
>> vmalloc_to_pfn(hv_hypercall_pg);
>>  	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> 
>> +	hyper_alloc_mmu();
>> +
>>  	/*
>>  	 * Register Hyper-V specific clocksource.
>>  	 */
>> diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
>> new file mode 100644
>> index 0000000..fb487cb
>> --- /dev/null
>> +++ b/arch/x86/hyperv/mmu.c
>> @@ -0,0 +1,128 @@
>> +#include <linux/types.h>
>> +#include <linux/hyperv.h>
>> +#include <linux/slab.h>
>> +#include <asm/mshyperv.h>
>> +#include <asm/tlbflush.h>
>> +#include <asm/msr.h>
>> +#include <asm/fpu/api.h>
>> +
>> +/*
>> + * Arbitrary number; we need to pre-allocate per-cpu struct for doing TLB
>> + * flush hypercalls and we need to pick a size. '16' means we'll be able
>> + * to flush 16 * 4096 pages (256MB) with one hypercall.
>> + */
>> +#define HV_MMU_MAX_GVAS 16
>
> Did you experiment with different sizes here.

Actually, I was never able to see kernel trying to flush more than 4096
pages so we can get away with HV_MMU_MAX_GVAS=1. I went through the code
and didn't see any 'limit' for the number of pages we can ask to flush
so it can be a coincidence. Each addition gva_list item requires 8 bytes
only so I put and arbitrary '16' here.

>> +
>> +/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls */
>> +struct hv_flush_pcpu {
>> +	struct {
>> +		__u64 address_space;
>> +		__u64 flags;
>> +		__u64 processor_mask;
>> +		__u64 gva_list[HV_MMU_MAX_GVAS];
>> +	} flush;
>> +
>> +	spinlock_t lock;
>> +};
>> +
> We may be supporting more than 64 CPUs in this hypercall. I am going to inquire with
> the Windows folks and get back to you.

Thanks! It is even specified in the specification:
"Future versions of the hypervisor may support more than 64 virtual processors per partition. In that
case, a new field will be added to the flags value that allows the caller to define the “processor bank” to
which the processor mask applies."

We, however, need to know where to put this in flags.

>
>> +static struct hv_flush_pcpu __percpu *pcpu_flush;
>> +
>> +static void hyperv_flush_tlb_others(const struct cpumask *cpus,
>> +				    struct mm_struct *mm, unsigned long
>> start,
>> +				    unsigned long end)
>> +{
>> +	struct hv_flush_pcpu *flush;
>> +	unsigned long cur, flags;
>> +	u64 status = -1ULL;
>> +	int cpu, vcpu, gva_n;
>> +
>> +	if (!pcpu_flush || !hv_hypercall_pg)
>> +		goto do_native;
>> +
>> +	if (cpumask_empty(cpus))
>> +		return;
>> +
>> +	flush = this_cpu_ptr(pcpu_flush);
>> +	spin_lock_irqsave(&flush->lock, flags);
>> +
>> +	flush->flush.address_space = virt_to_phys(mm->pgd);
>> +	flush->flush.processor_mask = 0;
>> +	if (cpumask_equal(cpus, cpu_present_mask)) {
>> +		flush->flush.flags = HV_FLUSH_ALL_PROCESSORS;
>> +	} else {
>> +		flush->flush.flags = 0;
>> +		for_each_cpu(cpu, cpus) {
>> +			vcpu = vmbus_cpu_number_to_vp_number(cpu);
>> +			if (vcpu != -1 && vcpu < 64)
>> +				flush->flush.processor_mask |= 1 << vcpu;
>> +			else
>> +				goto unlock_do_native;
>> +		}
>> +	}
>> +
>> +	if (end == TLB_FLUSH_ALL) {
>> +		flush->flush.flags =
>> HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
>> +		status =
>> hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
>> +					 &flush->flush, NULL);
>> +	} else {
>> +		cur = start;
>> +more_gvas:
>> +		gva_n = 0;
>> +
>> +		do {
>> +			flush->flush.gva_list[gva_n] = cur & PAGE_MASK;
>> +			/*
>> +			 * Lower 12 bits encode the number of additional
>> +			 * pages to flush (in addition to the 'cur' page).
>> +			 */
>> +			if (end >= cur + PAGE_SIZE * PAGE_SIZE)
>> +				flush->flush.gva_list[gva_n] |=
>> ~PAGE_MASK;
>> +			else if (end > cur)
>> +				flush->flush.gva_list[gva_n] |=
>> +					(end - cur - 1) >> PAGE_SHIFT;
>> +
>> +			cur += PAGE_SIZE * PAGE_SIZE;
>> +			++gva_n;
>> +
>> +		} while (cur < end && gva_n < HV_MMU_MAX_GVAS);
>> +
>> +		status =
>> hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
>> +					     gva_n, &flush->flush, NULL);
>> +
>> +		if (!(status & 0xffff) && cur < end)
>> +			goto more_gvas;
>> +	}
>> +
>> +unlock_do_native:
>> +	spin_unlock_irqrestore(&flush->lock, flags);
>> +
>> +	if (!(status & 0xffff))
>> +		return;
>> +do_native:
>> +	native_flush_tlb_others(cpus, mm, start, end);
>> +}
>> +
>> +void hyperv_setup_mmu_ops(void)
>> +{
>> +	if (ms_hyperv.hints &
>> HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
>> +		pr_info("Hyper-V: Using hypercall for remote TLB flush\n");
>> +		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
>> +	}
>> +}
>> +
>> +void hyper_alloc_mmu(void)
>> +{
>> +	int cpu;
>> +	struct hv_flush_pcpu *flush;
>> +
>> +	if (ms_hyperv.hints &
>> HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
>> +		pcpu_flush = alloc_percpu(struct hv_flush_pcpu);
>> +		if (!pcpu_flush)
>> +			return;
>> +
>> +		for_each_possible_cpu(cpu) {
>> +			flush = per_cpu_ptr(pcpu_flush, cpu);
>> +			spin_lock_init(&flush->lock);
>> +		}
>> +	}
>> +}
>> diff --git a/arch/x86/include/asm/mshyperv.h
>> b/arch/x86/include/asm/mshyperv.h
>> index 1293c84..a5041c3 100644
>> --- a/arch/x86/include/asm/mshyperv.h
>> +++ b/arch/x86/include/asm/mshyperv.h
>> @@ -301,6 +301,8 @@ static inline int
>> vmbus_cpu_number_to_vp_number(int cpu_number)
>>  }
>> 
>>  void hyperv_init(void);
>> +void hyperv_setup_mmu_ops(void);
>> +void hyper_alloc_mmu(void);
>>  void hyperv_report_panic(struct pt_regs *regs);
>>  bool hv_is_hypercall_page_setup(void);
>>  void hyperv_cleanup(void);
>> diff --git a/arch/x86/include/uapi/asm/hyperv.h
>> b/arch/x86/include/uapi/asm/hyperv.h
>> index c87e900..3d44036 100644
>> --- a/arch/x86/include/uapi/asm/hyperv.h
>> +++ b/arch/x86/include/uapi/asm/hyperv.h
>> @@ -239,6 +239,8 @@
>>  		(~((1ull <<
>> HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
>> 
>>  /* Declare the various hypercall operations. */
>> +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
>> +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
>>  #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
>>  #define HVCALL_POST_MESSAGE			0x005c
>>  #define HVCALL_SIGNAL_EVENT			0x005d
>> @@ -256,6 +258,11 @@
>>  #define HV_PROCESSOR_POWER_STATE_C2		2
>>  #define HV_PROCESSOR_POWER_STATE_C3		3
>> 
>> +#define HV_FLUSH_ALL_PROCESSORS			0x00000001
>> +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	0x00000002
>> +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	0x00000004
>> +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	0x00000008
>> +
>>  /* Hypercall interface */
>>  union hv_hypercall_input {
>>  	u64 as_uint64;
>> diff --git a/arch/x86/kernel/cpu/mshyperv.c
>> b/arch/x86/kernel/cpu/mshyperv.c
>> index 04cb8d3..fc228d8 100644
>> --- a/arch/x86/kernel/cpu/mshyperv.c
>> +++ b/arch/x86/kernel/cpu/mshyperv.c
>> @@ -233,6 +233,7 @@ static void __init ms_hyperv_init_platform(void)
>>  	 * Setup the hook to get control post apic initialization.
>>  	 */
>>  	x86_platform.apic_post_init = hyperv_init;
>> +	hyperv_setup_mmu_ops();
>>  #endif
>>  }
>> 
>> --
>> 2.9.3

-- 
  Vitaly

[toc] | [prev] | [next] | [standalone]

#1620503 — RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]

> > We may be supporting more than 64 CPUs in this hypercall. I am going
> > to inquire with the Windows folks and get back to you.
> 
> Thanks! It is even specified in the specification:
> "Future versions of the hypervisor may support more than 64 virtual
> processors per partition. In that case, a new field will be added to the flags
> value that allows the caller to define the “processor bank” to which the
> processor mask applies."
> 
> We, however, need to know where to put this in flags.

Would the HvFlushVirtualAddressListEx hypercall do? Is there a doc update/clarification needed?

Regards,
Jork

[toc] | [prev] | [next] | [standalone]

#1620757 — RE: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Monday, April 10, 2017 7:44 AM
> To: KY Srinivasan <kys@microsoft.com>
> Cc: devel@linuxdriverproject.org; x86@kernel.org; linux-
> kernel@vger.kernel.org; Haiyang Zhang <haiyangz@microsoft.com>;
> Stephen Hemminger <sthemmin@microsoft.com>; Thomas Gleixner
> <tglx@linutronix.de>; Ingo Molnar <mingo@redhat.com>; H. Peter Anvin
> <hpa@zytor.com>; Steven Rostedt <rostedt@goodmis.org>; Jork Loeser
> <Jork.Loeser@microsoft.com>
> Subject: Re: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush
> 
> KY Srinivasan <kys@microsoft.com> writes:
> 
> >> -----Original Message-----
> >> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> >> Sent: Friday, April 7, 2017 4:27 AM
> >> To: devel@linuxdriverproject.org; x86@kernel.org
> >> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> >> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> >> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>;
> Ingo
> >> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> >> Rostedt <rostedt@goodmis.org>; Jork Loeser
> <Jork.Loeser@microsoft.com>
> >> Subject: [PATCH 6/7] x86/hyper-v: use hypercall for remove TLB flush
> >>
> >> Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
> >> this is supposed to work faster than IPIs.
> >>
> >> Implementation details: to do HvFlushVirtualAddress{Space,List}
> hypercalls
> >> we need to put the input somewhere in memory and we don't really
> want to
> >> have memory allocation on each call so we pre-allocate per cpu memory
> >> areas
> >> on boot. These areas are of fixes size, limit them with an arbitrary number
> >> of 16 (16 gvas are able to specify 16 * 4096 pages).
> >>
> >> pv_ops patching is happening very early so we need to separate
> >> hyperv_setup_mmu_ops() and hyper_alloc_mmu().
> >>
> >> It is possible and easy to implement local TLB flushing too and there is
> >> even a hint for that. However, I don't see a room for optimization on the
> >> host side as both hypercall and native tlb flush will result in vmexit. The
> >> hint is also not set on modern Hyper-V versions.
> >>
> >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> >> ---
> >>  arch/x86/hyperv/Makefile           |   2 +-
> >>  arch/x86/hyperv/hv_init.c          |   2 +
> >>  arch/x86/hyperv/mmu.c              | 128
> >> +++++++++++++++++++++++++++++++++++++
> >>  arch/x86/include/asm/mshyperv.h    |   2 +
> >>  arch/x86/include/uapi/asm/hyperv.h |   7 ++
> >>  arch/x86/kernel/cpu/mshyperv.c     |   1 +
> >>  6 files changed, 141 insertions(+), 1 deletion(-)
> >>  create mode 100644 arch/x86/hyperv/mmu.c
> >>
> >> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> >> index 171ae09..367a820 100644
> >> --- a/arch/x86/hyperv/Makefile
> >> +++ b/arch/x86/hyperv/Makefile
> >> @@ -1 +1 @@
> >> -obj-y		:= hv_init.o
> >> +obj-y		:= hv_init.o mmu.o
> >> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> >> index 1c14088..2cf8a98 100644
> >> --- a/arch/x86/hyperv/hv_init.c
> >> +++ b/arch/x86/hyperv/hv_init.c
> >> @@ -163,6 +163,8 @@ void hyperv_init(void)
> >>  	hypercall_msr.guest_physical_address =
> >> vmalloc_to_pfn(hv_hypercall_pg);
> >>  	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >>
> >> +	hyper_alloc_mmu();
> >> +
> >>  	/*
> >>  	 * Register Hyper-V specific clocksource.
> >>  	 */
> >> diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
> >> new file mode 100644
> >> index 0000000..fb487cb
> >> --- /dev/null
> >> +++ b/arch/x86/hyperv/mmu.c
> >> @@ -0,0 +1,128 @@
> >> +#include <linux/types.h>
> >> +#include <linux/hyperv.h>
> >> +#include <linux/slab.h>
> >> +#include <asm/mshyperv.h>
> >> +#include <asm/tlbflush.h>
> >> +#include <asm/msr.h>
> >> +#include <asm/fpu/api.h>
> >> +
> >> +/*
> >> + * Arbitrary number; we need to pre-allocate per-cpu struct for doing
> TLB
> >> + * flush hypercalls and we need to pick a size. '16' means we'll be able
> >> + * to flush 16 * 4096 pages (256MB) with one hypercall.
> >> + */
> >> +#define HV_MMU_MAX_GVAS 16
> >
> > Did you experiment with different sizes here.
> 
> Actually, I was never able to see kernel trying to flush more than 4096
> pages so we can get away with HV_MMU_MAX_GVAS=1. I went through the
> code
> and didn't see any 'limit' for the number of pages we can ask to flush
> so it can be a coincidence. Each addition gva_list item requires 8 bytes
> only so I put and arbitrary '16' here.
> 
> >> +
> >> +/* HvFlushVirtualAddressSpace*, HvFlushVirtualAddressList hypercalls
> */
> >> +struct hv_flush_pcpu {
> >> +	struct {
> >> +		__u64 address_space;
> >> +		__u64 flags;
> >> +		__u64 processor_mask;
> >> +		__u64 gva_list[HV_MMU_MAX_GVAS];
> >> +	} flush;
> >> +
> >> +	spinlock_t lock;
> >> +};
> >> +
> > We may be supporting more than 64 CPUs in this hypercall. I am going to
> inquire with
> > the Windows folks and get back to you.
> 
> Thanks! It is even specified in the specification:
> "Future versions of the hypervisor may support more than 64 virtual
> processors per partition. In that
> case, a new field will be added to the flags value that allows the caller to
> define the “processor bank” to
> which the processor mask applies."
> 
> We, however, need to know where to put this in flags.

There is a new Hypercall for targeting more than 64 VCPUs. For now, we can check if the CPU mask
Is specifying more than 64 CPUs and use native call if that is the case.

K. Y

[toc] | [prev] | [next] | [standalone]

#1618693 — [PATCH 4/7] x86/hyperv: implement rep hypercalls

Rep hypercalls are normal hypercalls which perform multiple actions at
once. Hyper-V guarantees to return exectution to the caller in not more
than 50us and the caller needs to use hypercall continuation. Touch NMI
watchdog between hypercall invocations.

This is going to be used for HvFlushVirtualAddressList hypercall for
remote TLB flushing.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/include/asm/mshyperv.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 9a5f58b..a2c996b 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -4,6 +4,7 @@
 #include <linux/types.h>
 #include <linux/interrupt.h>
 #include <linux/clocksource.h>
+#include <linux/nmi.h>
 #include <asm/hyperv.h>
 
 /*
@@ -253,6 +254,26 @@ static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1)
 #endif
 }
 
+static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, void *input,
+				      void *output)
+{
+	union hv_hypercall_input hc_input = { .code = code,
+					      .rep_count = rep_count};
+	u64 status;
+
+	do {
+		status = hv_do_hypercall(hc_input.as_uint64, input, output);
+		if ((status & 0xffff) != HV_STATUS_SUCCESS)
+			return status;
+
+		hc_input.rep_start = (status >> 32) & 0xfff;
+
+		touch_nmi_watchdog();
+	} while (hc_input.rep_start < hc_input.rep_count);
+
+	return status;
+}
+
 void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1619068 — RE: [PATCH 4/7] x86/hyperv: implement rep hypercalls

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Friday, April 7, 2017 04:27
> To: devel@linuxdriverproject.org; x86@kernel.org
> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
> Subject: [PATCH 4/7] x86/hyperv: implement rep hypercalls
 
> diff --git a/arch/x86/include/asm/mshyperv.h
> b/arch/x86/include/asm/mshyperv.h index 9a5f58b..a2c996b 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -4,6 +4,7 @@
>  #include <linux/types.h>
>  #include <linux/interrupt.h>
>  #include <linux/clocksource.h>
> +#include <linux/nmi.h>
>  #include <asm/hyperv.h>
> 
>  /*
> @@ -253,6 +254,26 @@ static inline u64 hv_do_fast_hypercall8(u16 code,
> u64 input1)  #endif  }
> 
> +static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, void
> *input,
> +				      void *output)
> +{
> +	union hv_hypercall_input hc_input = { .code = code,
> +					      .rep_count = rep_count};

Is there a way to statically verify the re-count not to exceed 12 bits? Could a dynamic check be justified? Perhaps a function comment?

[toc] | [prev] | [next] | [standalone]

#1619744 — Re: [PATCH 4/7] x86/hyperv: implement rep hypercalls

Jork Loeser <Jork.Loeser@microsoft.com> writes:

>> -----Original Message-----
>> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
>> Sent: Friday, April 7, 2017 04:27
>> To: devel@linuxdriverproject.org; x86@kernel.org
>> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
>> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
>> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
>> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
>> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
>> Subject: [PATCH 4/7] x86/hyperv: implement rep hypercalls
>
>> diff --git a/arch/x86/include/asm/mshyperv.h
>> b/arch/x86/include/asm/mshyperv.h index 9a5f58b..a2c996b 100644
>> --- a/arch/x86/include/asm/mshyperv.h
>> +++ b/arch/x86/include/asm/mshyperv.h
>> @@ -4,6 +4,7 @@
>>  #include <linux/types.h>
>>  #include <linux/interrupt.h>
>>  #include <linux/clocksource.h>
>> +#include <linux/nmi.h>
>>  #include <asm/hyperv.h>
>> 
>>  /*
>> @@ -253,6 +254,26 @@ static inline u64 hv_do_fast_hypercall8(u16 code,
>> u64 input1)  #endif  }
>> 
>> +static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, void
>> *input,
>> +				      void *output)
>> +{
>> +	union hv_hypercall_input hc_input = { .code = code,
>> +					      .rep_count = rep_count};
>
> Is there a way to statically verify the re-count not to exceed 12 bits? Could a dynamic check be justified? Perhaps a function comment?

I'd like to avoid dynamic checks here to keep this as fast as
possible. Static check is probably not an option as even the only user
we have now calculates this parameter dynamically. I'll add a comment to
the function, thanks!

-- 
  Vitaly

[toc] | [prev] | [next] | [standalone]

#1618694 — [PATCH 5/7] hyper-v: globalize vp_index

To support implementing remote TLB flushing on Hyper-V with a hypercall
we need to make vp_index available outside of vmbus module. Rename and
globalize.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/hyperv/hv_init.c       | 34 +++++++++++++++++++++++++++++++++-
 arch/x86/include/asm/mshyperv.h | 26 ++++++++++++++++++++++++++
 drivers/hv/channel_mgmt.c       |  7 +++----
 drivers/hv/connection.c         |  3 ++-
 drivers/hv/hv.c                 |  9 ---------
 drivers/hv/hyperv_vmbus.h       | 11 -----------
 drivers/hv/vmbus_drv.c          | 17 -----------------
 include/linux/hyperv.h          |  1 -
 8 files changed, 64 insertions(+), 44 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 7d961d4..1c14088 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -26,6 +26,8 @@
 #include <linux/mm.h>
 #include <linux/clockchips.h>
 #include <linux/hyperv.h>
+#include <linux/slab.h>
+#include <linux/cpuhotplug.h>
 
 #ifdef CONFIG_X86_64
 
@@ -103,6 +105,20 @@ EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 struct clocksource *hyperv_cs;
 EXPORT_SYMBOL_GPL(hyperv_cs);
 
+u32 *hv_vp_index;
+EXPORT_SYMBOL_GPL(hv_vp_index);
+
+static int hv_cpu_init(unsigned int cpu)
+{
+	u64 msr_vp_index;
+
+	hv_get_vp_index(msr_vp_index);
+
+	hv_vp_index[smp_processor_id()] = (u32)msr_vp_index;
+
+	return 0;
+}
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
@@ -118,6 +134,16 @@ void hyperv_init(void)
 	if (x86_hyper != &x86_hyper_ms_hyperv)
 		return;
 
+	/* Allocate percpu VP index */
+	hv_vp_index = kcalloc(num_possible_cpus(), sizeof(*hv_vp_index),
+			      GFP_KERNEL);
+	if (!hv_vp_index)
+		return;
+
+	if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:online",
+			      hv_cpu_init, NULL) < 0)
+		goto free_vp_index;
+
 	/*
 	 * Setup the hypercall page and enable hypercalls.
 	 * 1. Register the guest ID
@@ -129,7 +155,7 @@ void hyperv_init(void)
 	hv_hypercall_pg  = __vmalloc(PAGE_SIZE, GFP_KERNEL, PAGE_KERNEL_RX);
 	if (hv_hypercall_pg == NULL) {
 		wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
-		return;
+		goto free_vp_index;
 	}
 
 	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
@@ -169,6 +195,12 @@ void hyperv_init(void)
 	hyperv_cs = &hyperv_cs_msr;
 	if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
 		clocksource_register_hz(&hyperv_cs_msr, NSEC_PER_SEC/100);
+
+	return;
+
+free_vp_index:
+	kfree(hv_vp_index);
+	hv_vp_index = NULL;
 }
 
 /*
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index a2c996b..1293c84 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -274,6 +274,32 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, void *input,
 	return status;
 }
 
+/*
+ * Hypervisor's notion of virtual processor ID is different from
+ * Linux' notion of CPU ID. This information can only be retrieved
+ * in the context of the calling CPU. Setup a map for easy access
+ * to this information.
+ */
+extern u32 __percpu *hv_vp_index;
+
+/**
+ * vmbus_cpu_number_to_vp_number() - Map CPU to VP.
+ * @cpu_number: CPU number in Linux terms
+ *
+ * This function returns the mapping between the Linux processor
+ * number and the hypervisor's virtual processor number, useful
+ * in making hypercalls and such that talk about specific
+ * processors.
+ *
+ * Return: Virtual processor number in Hyper-V terms
+ */
+static inline int vmbus_cpu_number_to_vp_number(int cpu_number)
+{
+	WARN_ON(hv_vp_index[cpu_number] == -1);
+
+	return hv_vp_index[cpu_number];
+}
+
 void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 6cfa297..9969c82 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -599,7 +599,7 @@ static void init_vp_index(struct vmbus_channel *channel, u16 dev_type)
 		 */
 		channel->numa_node = 0;
 		channel->target_cpu = 0;
-		channel->target_vp = hv_context.vp_index[0];
+		channel->target_vp = vmbus_cpu_number_to_vp_number(0);
 		return;
 	}
 
@@ -683,7 +683,7 @@ static void init_vp_index(struct vmbus_channel *channel, u16 dev_type)
 	}
 
 	channel->target_cpu = cur_cpu;
-	channel->target_vp = hv_context.vp_index[cur_cpu];
+	channel->target_vp = vmbus_cpu_number_to_vp_number(cur_cpu);
 }
 
 static void vmbus_wait_for_unload(void)
@@ -1187,8 +1187,7 @@ struct vmbus_channel *vmbus_get_outgoing_channel(struct vmbus_channel *primary)
 		return outgoing_channel;
 	}
 
-	cur_cpu = hv_context.vp_index[get_cpu()];
-	put_cpu();
+	cur_cpu = vmbus_cpu_number_to_vp_number(smp_processor_id());
 	list_for_each_safe(cur, tmp, &primary->sc_list) {
 		cur_channel = list_entry(cur, struct vmbus_channel, sc_list);
 		if (cur_channel->state != CHANNEL_OPENED_STATE)
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 545f2a4..7026d13 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -96,7 +96,8 @@ static int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo,
 	 * the CPU attempting to connect may not be CPU 0.
 	 */
 	if (version >= VERSION_WIN8_1)
-		msg->target_vcpu = hv_context.vp_index[smp_processor_id()];
+		msg->target_vcpu =
+			vmbus_cpu_number_to_vp_number(smp_processor_id());
 	else
 		msg->target_vcpu = 0;
 
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index 12e7bae..7e67ef4 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -229,7 +229,6 @@ int hv_synic_init(unsigned int cpu)
 	union hv_synic_siefp siefp;
 	union hv_synic_sint shared_sint;
 	union hv_synic_scontrol sctrl;
-	u64 vp_index;
 
 	/* Setup the Synic's message page */
 	hv_get_simp(simp.as_uint64);
@@ -271,14 +270,6 @@ int hv_synic_init(unsigned int cpu)
 	hv_context.synic_initialized = true;
 
 	/*
-	 * Setup the mapping between Hyper-V's notion
-	 * of cpuid and Linux' notion of cpuid.
-	 * This array will be indexed using Linux cpuid.
-	 */
-	hv_get_vp_index(vp_index);
-	hv_context.vp_index[cpu] = (u32)vp_index;
-
-	/*
 	 * Register the per-cpu clockevent source.
 	 */
 	if (ms_hyperv.features & HV_X64_MSR_SYNTIMER_AVAILABLE)
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 6113e91..d624526 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -229,17 +229,6 @@ struct hv_context {
 	struct hv_per_cpu_context __percpu *cpu_context;
 
 	/*
-	 * Hypervisor's notion of virtual processor ID is different from
-	 * Linux' notion of CPU ID. This information can only be retrieved
-	 * in the context of the calling CPU. Setup a map for easy access
-	 * to this information:
-	 *
-	 * vp_index[a] is the Hyper-V's processor ID corresponding to
-	 * Linux cpuid 'a'.
-	 */
-	u32 vp_index[NR_CPUS];
-
-	/*
 	 * To manage allocations in a NUMA node.
 	 * Array indexed by numa node ID.
 	 */
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 0087b49..63e743d 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1455,23 +1455,6 @@ void vmbus_free_mmio(resource_size_t start, resource_size_t size)
 }
 EXPORT_SYMBOL_GPL(vmbus_free_mmio);
 
-/**
- * vmbus_cpu_number_to_vp_number() - Map CPU to VP.
- * @cpu_number: CPU number in Linux terms
- *
- * This function returns the mapping between the Linux processor
- * number and the hypervisor's virtual processor number, useful
- * in making hypercalls and such that talk about specific
- * processors.
- *
- * Return: Virtual processor number in Hyper-V terms
- */
-int vmbus_cpu_number_to_vp_number(int cpu_number)
-{
-	return hv_context.vp_index[cpu_number];
-}
-EXPORT_SYMBOL_GPL(vmbus_cpu_number_to_vp_number);
-
 static int vmbus_acpi_add(struct acpi_device *device)
 {
 	acpi_status result;
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 5d6777c..2450e07 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -1184,7 +1184,6 @@ int vmbus_allocate_mmio(struct resource **new, struct hv_device *device_obj,
 			resource_size_t size, resource_size_t align,
 			bool fb_overlap_ok);
 void vmbus_free_mmio(resource_size_t start, resource_size_t size);
-int vmbus_cpu_number_to_vp_number(int cpu_number);
 
 /*
  * GUID definitions of various offer types - services offered to the guest.
-- 
2.9.3

[toc] | [prev] | [next] | [standalone]

#1619314 — RE: [PATCH 5/7] hyper-v: globalize vp_index

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Friday, April 7, 2017 4:27 AM
> To: devel@linuxdriverproject.org; x86@kernel.org
> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
> Subject: [PATCH 5/7] hyper-v: globalize vp_index
> 
> To support implementing remote TLB flushing on Hyper-V with a hypercall
> we need to make vp_index available outside of vmbus module. Rename and
> globalize.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  arch/x86/hyperv/hv_init.c       | 34
> +++++++++++++++++++++++++++++++++-
>  arch/x86/include/asm/mshyperv.h | 26 ++++++++++++++++++++++++++
>  drivers/hv/channel_mgmt.c       |  7 +++----
>  drivers/hv/connection.c         |  3 ++-
>  drivers/hv/hv.c                 |  9 ---------
>  drivers/hv/hyperv_vmbus.h       | 11 -----------
>  drivers/hv/vmbus_drv.c          | 17 -----------------
>  include/linux/hyperv.h          |  1 -
>  8 files changed, 64 insertions(+), 44 deletions(-)
> 
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index 7d961d4..1c14088 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -26,6 +26,8 @@
>  #include <linux/mm.h>
>  #include <linux/clockchips.h>
>  #include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <linux/cpuhotplug.h>
> 
>  #ifdef CONFIG_X86_64
> 
> @@ -103,6 +105,20 @@ EXPORT_SYMBOL_GPL(hv_hypercall_pg);
>  struct clocksource *hyperv_cs;
>  EXPORT_SYMBOL_GPL(hyperv_cs);
> 
> +u32 *hv_vp_index;
> +EXPORT_SYMBOL_GPL(hv_vp_index);
> +
> +static int hv_cpu_init(unsigned int cpu)
> +{
> +	u64 msr_vp_index;
> +
> +	hv_get_vp_index(msr_vp_index);
> +
> +	hv_vp_index[smp_processor_id()] = (u32)msr_vp_index;
> +
> +	return 0;
> +}
> +
>  /*
>   * This function is to be invoked early in the boot sequence after the
>   * hypervisor has been detected.
> @@ -118,6 +134,16 @@ void hyperv_init(void)
>  	if (x86_hyper != &x86_hyper_ms_hyperv)
>  		return;
> 
> +	/* Allocate percpu VP index */
> +	hv_vp_index = kcalloc(num_possible_cpus(), sizeof(*hv_vp_index),
> +			      GFP_KERNEL);
> +	if (!hv_vp_index)
> +		return;
> +
> +	if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> "x86/hyperv_init:online",
> +			      hv_cpu_init, NULL) < 0)
> +		goto free_vp_index;
> +
>  	/*
>  	 * Setup the hypercall page and enable hypercalls.
>  	 * 1. Register the guest ID
> @@ -129,7 +155,7 @@ void hyperv_init(void)
>  	hv_hypercall_pg  = __vmalloc(PAGE_SIZE, GFP_KERNEL,
> PAGE_KERNEL_RX);
>  	if (hv_hypercall_pg == NULL) {
>  		wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
> -		return;
> +		goto free_vp_index;
>  	}
> 
>  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> @@ -169,6 +195,12 @@ void hyperv_init(void)
>  	hyperv_cs = &hyperv_cs_msr;
>  	if (ms_hyperv.features &
> HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
>  		clocksource_register_hz(&hyperv_cs_msr,
> NSEC_PER_SEC/100);
> +
> +	return;
> +
> +free_vp_index:
> +	kfree(hv_vp_index);
> +	hv_vp_index = NULL;
>  }
> 
>  /*
> diff --git a/arch/x86/include/asm/mshyperv.h
> b/arch/x86/include/asm/mshyperv.h
> index a2c996b..1293c84 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -274,6 +274,32 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16
> rep_count, void *input,
>  	return status;
>  }
> 
> +/*
> + * Hypervisor's notion of virtual processor ID is different from
> + * Linux' notion of CPU ID. This information can only be retrieved
> + * in the context of the calling CPU. Setup a map for easy access
> + * to this information.
> + */
> +extern u32 __percpu *hv_vp_index;
> +
> +/**
> + * vmbus_cpu_number_to_vp_number() - Map CPU to VP.
> + * @cpu_number: CPU number in Linux terms
> + *
> + * This function returns the mapping between the Linux processor
> + * number and the hypervisor's virtual processor number, useful
> + * in making hypercalls and such that talk about specific
> + * processors.
> + *
> + * Return: Virtual processor number in Hyper-V terms
> + */
> +static inline int vmbus_cpu_number_to_vp_number(int cpu_number)
> +{
> +	WARN_ON(hv_vp_index[cpu_number] == -1);
> +
> +	return hv_vp_index[cpu_number];
> +}

Now that we have moved this functionality into the Hyper-V specific 
base kernel, a hv prefix maybe more appropriate for the function.

> +
>  void hyperv_init(void);
>  void hyperv_report_panic(struct pt_regs *regs);
>  bool hv_is_hypercall_page_setup(void);
> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> index 6cfa297..9969c82 100644
> --- a/drivers/hv/channel_mgmt.c
> +++ b/drivers/hv/channel_mgmt.c
> @@ -599,7 +599,7 @@ static void init_vp_index(struct vmbus_channel
> *channel, u16 dev_type)
>  		 */
>  		channel->numa_node = 0;
>  		channel->target_cpu = 0;
> -		channel->target_vp = hv_context.vp_index[0];
> +		channel->target_vp =
> vmbus_cpu_number_to_vp_number(0);
>  		return;
>  	}
> 
> @@ -683,7 +683,7 @@ static void init_vp_index(struct vmbus_channel
> *channel, u16 dev_type)
>  	}
> 
>  	channel->target_cpu = cur_cpu;
> -	channel->target_vp = hv_context.vp_index[cur_cpu];
> +	channel->target_vp =
> vmbus_cpu_number_to_vp_number(cur_cpu);
>  }
> 
>  static void vmbus_wait_for_unload(void)
> @@ -1187,8 +1187,7 @@ struct vmbus_channel
> *vmbus_get_outgoing_channel(struct vmbus_channel *primary)
>  		return outgoing_channel;
>  	}
> 
> -	cur_cpu = hv_context.vp_index[get_cpu()];
> -	put_cpu();
> +	cur_cpu =
> vmbus_cpu_number_to_vp_number(smp_processor_id());
>  	list_for_each_safe(cur, tmp, &primary->sc_list) {
>  		cur_channel = list_entry(cur, struct vmbus_channel, sc_list);
>  		if (cur_channel->state != CHANNEL_OPENED_STATE)
> diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
> index 545f2a4..7026d13 100644
> --- a/drivers/hv/connection.c
> +++ b/drivers/hv/connection.c
> @@ -96,7 +96,8 @@ static int vmbus_negotiate_version(struct
> vmbus_channel_msginfo *msginfo,
>  	 * the CPU attempting to connect may not be CPU 0.
>  	 */
>  	if (version >= VERSION_WIN8_1)
> -		msg->target_vcpu =
> hv_context.vp_index[smp_processor_id()];
> +		msg->target_vcpu =
> +
> 	vmbus_cpu_number_to_vp_number(smp_processor_id());
>  	else
>  		msg->target_vcpu = 0;
> 
> diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
> index 12e7bae..7e67ef4 100644
> --- a/drivers/hv/hv.c
> +++ b/drivers/hv/hv.c
> @@ -229,7 +229,6 @@ int hv_synic_init(unsigned int cpu)
>  	union hv_synic_siefp siefp;
>  	union hv_synic_sint shared_sint;
>  	union hv_synic_scontrol sctrl;
> -	u64 vp_index;
> 
>  	/* Setup the Synic's message page */
>  	hv_get_simp(simp.as_uint64);
> @@ -271,14 +270,6 @@ int hv_synic_init(unsigned int cpu)
>  	hv_context.synic_initialized = true;
> 
>  	/*
> -	 * Setup the mapping between Hyper-V's notion
> -	 * of cpuid and Linux' notion of cpuid.
> -	 * This array will be indexed using Linux cpuid.
> -	 */
> -	hv_get_vp_index(vp_index);
> -	hv_context.vp_index[cpu] = (u32)vp_index;
> -
> -	/*
>  	 * Register the per-cpu clockevent source.
>  	 */
>  	if (ms_hyperv.features & HV_X64_MSR_SYNTIMER_AVAILABLE)
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 6113e91..d624526 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -229,17 +229,6 @@ struct hv_context {
>  	struct hv_per_cpu_context __percpu *cpu_context;
> 
>  	/*
> -	 * Hypervisor's notion of virtual processor ID is different from
> -	 * Linux' notion of CPU ID. This information can only be retrieved
> -	 * in the context of the calling CPU. Setup a map for easy access
> -	 * to this information:
> -	 *
> -	 * vp_index[a] is the Hyper-V's processor ID corresponding to
> -	 * Linux cpuid 'a'.
> -	 */
> -	u32 vp_index[NR_CPUS];
> -
> -	/*
>  	 * To manage allocations in a NUMA node.
>  	 * Array indexed by numa node ID.
>  	 */
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 0087b49..63e743d 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -1455,23 +1455,6 @@ void vmbus_free_mmio(resource_size_t start,
> resource_size_t size)
>  }
>  EXPORT_SYMBOL_GPL(vmbus_free_mmio);
> 
> -/**
> - * vmbus_cpu_number_to_vp_number() - Map CPU to VP.
> - * @cpu_number: CPU number in Linux terms
> - *
> - * This function returns the mapping between the Linux processor
> - * number and the hypervisor's virtual processor number, useful
> - * in making hypercalls and such that talk about specific
> - * processors.
> - *
> - * Return: Virtual processor number in Hyper-V terms
> - */
> -int vmbus_cpu_number_to_vp_number(int cpu_number)
> -{
> -	return hv_context.vp_index[cpu_number];
> -}
> -EXPORT_SYMBOL_GPL(vmbus_cpu_number_to_vp_number);
> -
>  static int vmbus_acpi_add(struct acpi_device *device)
>  {
>  	acpi_status result;
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index 5d6777c..2450e07 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -1184,7 +1184,6 @@ int vmbus_allocate_mmio(struct resource **new,
> struct hv_device *device_obj,
>  			resource_size_t size, resource_size_t align,
>  			bool fb_overlap_ok);
>  void vmbus_free_mmio(resource_size_t start, resource_size_t size);
> -int vmbus_cpu_number_to_vp_number(int cpu_number);
> 
>  /*
>   * GUID definitions of various offer types - services offered to the guest.
> --
> 2.9.3

[toc] | [prev] | [next] | [standalone]

#1619295 — RE: [PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and hypercall improvements

> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Friday, April 7, 2017 4:27 AM
> To: devel@linuxdriverproject.org; x86@kernel.org
> Cc: linux-kernel@vger.kernel.org; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Thomas Gleixner <tglx@linutronix.de>; Ingo
> Molnar <mingo@redhat.com>; H. Peter Anvin <hpa@zytor.com>; Steven
> Rostedt <rostedt@goodmis.org>; Jork Loeser <Jork.Loeser@microsoft.com>
> Subject: [PATCH 0/7] Hyper-V: praravirtualized remote TLB flushing and
> hypercall improvements
> 
> Hi,
> 
> Hyper-V supports hypercalls for doing local and remote TLB flushing and
> gives its guests hints when using hypercall is preferred. While doing
> hypercalls for local TLB flushes is probably not practical (and is not
> being suggested by modern Hyper-V versions) remote TLB flush with a
> hypercall brings significant improvement.
> 
> To test the series I wrote a special 'TLB trasher': on a 16 vCPU guest I
> was creating 32 threads which were doing 100000 mmap/munmaps each on
> some
> big file. Here are the results:
> 
> Before:
> # time ./pthread_mmap ./randfile
> real	3m44.994s
> user	0m3.829s
> sys	3m36.323s
> 
> After:
> # time ./pthread_mmap ./randfile
> real	2m57.145s
> user	0m3.797s
> sys	2m34.812s
> 
> This series brings a number of small improvements along the way: fast
> hypercall implementation and using it for event signaling, rep hypercalls
> implementation, hyperv tracing subsystem (which only traces the newly
> added
> remote TLB flush for now).

Thanks Vitaly. We are currently testing these patches on Azure and other Hyper-V
platforms and will report back.

K. Y 
> 
> Vitaly Kuznetsov (7):
>   x86/hyperv: make hv_do_hypercall() inline
>   x86/hyper-v: fast hypercall implementation
>   hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT
>   x86/hyperv: implement rep hypercalls
>   hyper-v: globalize vp_index
>   x86/hyper-v: use hypercall for remove TLB flush
>   tracing/hyper-v: trace hyperv_mmu_flush_tlb_others()
> 
>  MAINTAINERS                        |   1 +
>  arch/x86/hyperv/Makefile           |   2 +-
>  arch/x86/hyperv/hv_init.c          |  90 +++++++++++--------------
>  arch/x86/hyperv/mmu.c              | 134
> +++++++++++++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h    | 131
> ++++++++++++++++++++++++++++++++++++
>  arch/x86/include/uapi/asm/hyperv.h |  26 +++++++
>  arch/x86/kernel/cpu/mshyperv.c     |   1 +
>  drivers/hv/channel_mgmt.c          |  22 +++---
>  drivers/hv/connection.c            |   8 ++-
>  drivers/hv/hv.c                    |   9 ---
>  drivers/hv/hyperv_vmbus.h          |  11 ---
>  drivers/hv/vmbus_drv.c             |  17 -----
>  include/linux/hyperv.h             |  21 +++---
>  include/trace/events/hyperv.h      |  30 +++++++++
>  14 files changed, 386 insertions(+), 117 deletions(-)
>  create mode 100644 arch/x86/hyperv/mmu.c
>  create mode 100644 include/trace/events/hyperv.h
> 
> --
> 2.9.3

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web