Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1702217 > unrolled thread

[PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements

Back to article view | Back to linux.kernel

Contents

  [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:10 +0200
    [PATCH v10 1/9] x86/hyper-v: include hyperv/ only when CONFIG_HYPERV is set Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:10 +0200
      [tip:x86/platform] x86/hyper-v: Include hyperv/ only when  CONFIG_HYPERV is set tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
    [PATCH v10 3/9] x86/hyper-v: fast hypercall implementation Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:10 +0200
      [tip:x86/platform] x86/hyper-v: Introduce fast hypercall  implementation tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
    [PATCH v10 7/9] x86/hyper-v: use hypercall for remote TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
      [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
      [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 20:30 +0200
        Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-10 21:00 +0200
          Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-10 21:30 +0200
            Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 11:10 +0200
              Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush "Kirill A. Shutemov" <kirill@shutemov.name> - 2017-08-11 13:30 +0200
              Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush Linus Torvalds <torvalds@linux-foundation.org> - 2017-08-11 18:20 +0200
                Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 18:30 +0200
            Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-11 11:30 +0200
              Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 13:00 +0200
                Re: [Xen-devel] [tip:x86/platform] x86/hyper-v: Use hypercall for  remote TLB flush Andrew Cooper <andrew.cooper3@citrix.com> - 2017-08-11 13:10 +0200
                  Re: [Xen-devel] [tip:x86/platform] x86/hyper-v: Use hypercall for  remote TLB flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 14:10 +0200
                Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Juergen Gross <jgross@suse.com> - 2017-08-11 14:30 +0200
                  Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 14:40 +0200
                    Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Juergen Gross <jgross@suse.com> - 2017-08-11 14:50 +0200
                      Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 15:00 +0200
                        Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Juergen Gross <jgross@suse.com> - 2017-08-11 15:10 +0200
                          Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB  flush Peter Zijlstra <peterz@infradead.org> - 2017-08-11 15:50 +0200
    [PATCH v10 4/9] hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
      [tip:x86/platform] hyper-v: Use fast hypercall for  HVCALL_SIGNAL_EVENT tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
    [PATCH v10 5/9] x86/hyper-v: implement rep hypercalls Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
      [tip:x86/platform] x86/hyper-v: Implement rep hypercalls tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
    [PATCH v10 8/9] x86/hyper-v: support extended CPU ranges for TLB flush hypercalls Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
    [PATCH v10 9/9] tracing/hyper-v: trace hyperv_mmu_flush_tlb_others() Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
    [PATCH v10 6/9] hyper-v: globalize vp_index Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-02 18:20 +0200
      [tip:x86/platform] hyper-v: Globalize vp_index tip-bot for Vitaly Kuznetsov <tipbot@zytor.com> - 2017-08-10 18:50 +0200
    Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-10 14:00 +0200
      Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and  hypercall improvements Ingo Molnar <mingo@kernel.org> - 2017-08-10 17:20 +0200
        Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-10 17:20 +0200
    Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements Vitaly Kuznetsov <vkuznets@redhat.com> - 2017-08-10 19:10 +0200

Page 1 of 2  [1] 2  Next page →

#1702217 — [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements

Changes since v9:
- Rebase to 4.13-rc3.
- Drop PATCH1 as it was already taken by Greg to char-misc tree. There're no
  functional dependencies on this patch so the series can go through a different tree
  (and it actually belongs to x86 if I got Ingo's comment right).
- Add in missing void return type in PATCH1 [Colin King, Ingo Molnar, Greg KH]
- A few minor fixes in what is now PATCH7: add pr_fmt, tiny style fix in
  hyperv_flush_tlb_others() [Andy Shevchenko]
- Fix "error: implicit declaration of function 'virt_to_phys'" in PATCH2
  reported by kbuild test robot (#include <asm/io.h>)
- Add Steven's 'Reviewed-by:' to PATCH9.

Original description:

Hyper-V supports hypercalls for doing local and remote TLB flushing and
gives its guests hints when using hypercall is preferred. While doing
hypercalls for local TLB flushes is probably not practical (and is not
being suggested by modern Hyper-V versions) remote TLB flush with a
hypercall brings significant improvement.

To test the series I wrote a special 'TLB trasher': on a 16 vCPU guest I
was creating 32 threads which were doing 100000 mmap/munmaps each on some
big file. Here are the results:

Before:
# time ./pthread_mmap ./randfile 
real	3m33.118s
user	0m3.698s
sys	3m16.624s

After:
# time ./pthread_mmap ./randfile 
real	2m19.920s
user	0m2.662s
sys	2m9.948s

This series brings a number of small improvements along the way: fast
hypercall implementation and using it for event signaling, rep hypercalls
implementation, hyperv tracing subsystem (which only traces the newly added
remote TLB flush for now).

Vitaly Kuznetsov (9):
  x86/hyper-v: include hyperv/ only when CONFIG_HYPERV is set
  x86/hyper-v: make hv_do_hypercall() inline
  x86/hyper-v: fast hypercall implementation
  hyper-v: use fast hypercall for HVCALL_SIGNAL_EVENT
  x86/hyper-v: implement rep hypercalls
  hyper-v: globalize vp_index
  x86/hyper-v: use hypercall for remote TLB flush
  x86/hyper-v: support extended CPU ranges for TLB flush hypercalls
  tracing/hyper-v: trace hyperv_mmu_flush_tlb_others()

 MAINTAINERS                         |   1 +
 arch/x86/Kbuild                     |   2 +-
 arch/x86/hyperv/Makefile            |   2 +-
 arch/x86/hyperv/hv_init.c           |  90 ++++++------
 arch/x86/hyperv/mmu.c               | 272 ++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h     | 147 ++++++++++++++++++-
 arch/x86/include/asm/trace/hyperv.h |  40 ++++++
 arch/x86/include/uapi/asm/hyperv.h  |  17 +++
 arch/x86/kernel/cpu/mshyperv.c      |   1 +
 drivers/hv/channel_mgmt.c           |  20 +--
 drivers/hv/connection.c             |   7 +-
 drivers/hv/hv.c                     |   9 --
 drivers/hv/hyperv_vmbus.h           |  11 --
 drivers/hv/vmbus_drv.c              |  17 ---
 drivers/pci/host/pci-hyperv.c       |  54 +------
 include/linux/hyperv.h              |  17 +--
 16 files changed, 533 insertions(+), 174 deletions(-)
 create mode 100644 arch/x86/hyperv/mmu.c
 create mode 100644 arch/x86/include/asm/trace/hyperv.h

-- 
2.13.3

[toc] | [next] | [standalone]

#1702222 — [PATCH v10 1/9] x86/hyper-v: include hyperv/ only when CONFIG_HYPERV is set

Code is arch/x86/hyperv/ is only needed when CONFIG_HYPERV is set, the
'basic' support and detection lives in arch/x86/kernel/cpu/mshyperv.c
which is included when CONFIG_HYPERVISOR_GUEST is set.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
---
 arch/x86/Kbuild                 | 2 +-
 arch/x86/include/asm/mshyperv.h | 7 ++++++-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kbuild b/arch/x86/Kbuild
index 586b786b3edf..3e6f64073005 100644
--- a/arch/x86/Kbuild
+++ b/arch/x86/Kbuild
@@ -8,7 +8,7 @@ obj-$(CONFIG_KVM) += kvm/
 obj-$(CONFIG_XEN) += xen/
 
 # Hyper-V paravirtualization support
-obj-$(CONFIG_HYPERVISOR_GUEST) += hyperv/
+obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/
 
 # lguest paravirtualization support
 obj-$(CONFIG_LGUEST_GUEST) += lguest/
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 2b58c8c1eeaa..baea2679a0aa 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -173,7 +173,12 @@ void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
-#endif
+#else /* CONFIG_HYPERV */
+static inline void hyperv_init(void) {}
+static inline bool hv_is_hypercall_page_setup(void) { return false; }
+static inline void hyperv_cleanup(void) {}
+#endif /* CONFIG_HYPERV */
+
 #ifdef CONFIG_HYPERV_TSCPAGE
 struct ms_hyperv_tsc_page *hv_get_tsc_page(void);
 static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
-- 
2.13.3

[toc] | [prev] | [next] | [standalone]

#1708865 — [tip:x86/platform] x86/hyper-v: Include hyperv/ only when CONFIG_HYPERV is set

Commit-ID:  79cadff2d92bb8b1448f6dba6861d15adc3dc4cb
Gitweb:     http://git.kernel.org/tip/79cadff2d92bb8b1448f6dba6861d15adc3dc4cb
Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
AuthorDate: Wed, 2 Aug 2017 18:09:13 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 10 Aug 2017 16:50:22 +0200

x86/hyper-v: Include hyperv/ only when CONFIG_HYPERV is set

Code is arch/x86/hyperv/ is only needed when CONFIG_HYPERV is set, the
'basic' support and detection lives in arch/x86/kernel/cpu/mshyperv.c
which is included when CONFIG_HYPERVISOR_GUEST is set.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jork Loeser <Jork.Loeser@microsoft.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: devel@linuxdriverproject.org
Link: http://lkml.kernel.org/r/20170802160921.21791-2-vkuznets@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/Kbuild                 | 2 +-
 arch/x86/include/asm/mshyperv.h | 7 ++++++-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kbuild b/arch/x86/Kbuild
index 586b786..3e6f640 100644
--- a/arch/x86/Kbuild
+++ b/arch/x86/Kbuild
@@ -8,7 +8,7 @@ obj-$(CONFIG_KVM) += kvm/
 obj-$(CONFIG_XEN) += xen/
 
 # Hyper-V paravirtualization support
-obj-$(CONFIG_HYPERVISOR_GUEST) += hyperv/
+obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/
 
 # lguest paravirtualization support
 obj-$(CONFIG_LGUEST_GUEST) += lguest/
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 2b58c8c..baea267 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -173,7 +173,12 @@ void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
-#endif
+#else /* CONFIG_HYPERV */
+static inline void hyperv_init(void) {}
+static inline bool hv_is_hypercall_page_setup(void) { return false; }
+static inline void hyperv_cleanup(void) {}
+#endif /* CONFIG_HYPERV */
+
 #ifdef CONFIG_HYPERV_TSCPAGE
 struct ms_hyperv_tsc_page *hv_get_tsc_page(void);
 static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)

[toc] | [prev] | [next] | [standalone]

#1702224 — [PATCH v10 3/9] x86/hyper-v: fast hypercall implementation

Hyper-V supports 'fast' hypercalls when all parameters are passed through
registers. Implement an inline version of a simpliest of these calls:
hypercall with one 8-byte input and no output.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
---
 arch/x86/include/asm/mshyperv.h | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 6fa5e342cc86..e484255bd9de 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -209,6 +209,40 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
 	return hv_status;
 }
 
+#define HV_HYPERCALL_FAST_BIT		BIT(16)
+
+/* Fast hypercall with 8 bytes of input and no output */
+static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1)
+{
+	u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT;
+	register void *__sp asm(_ASM_SP);
+
+#ifdef CONFIG_X86_64
+	{
+		__asm__ __volatile__("call *%4"
+				     : "=a" (hv_status), "+r" (__sp),
+				       "+c" (control), "+d" (input1)
+				     : "m" (hv_hypercall_pg)
+				     : "cc", "r8", "r9", "r10", "r11");
+	}
+#else
+	{
+		u32 input1_hi = upper_32_bits(input1);
+		u32 input1_lo = lower_32_bits(input1);
+
+		__asm__ __volatile__ ("call *%5"
+				      : "=A"(hv_status),
+					"+c"(input1_lo),
+					"+r"(__sp)
+				      :	"A" (control),
+					"b" (input1_hi),
+					"m" (hv_hypercall_pg)
+				      : "cc", "edi", "esi");
+	}
+#endif
+		return hv_status;
+}
+
 void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
-- 
2.13.3

[toc] | [prev] | [next] | [standalone]

#1708880 — [tip:x86/platform] x86/hyper-v: Introduce fast hypercall implementation

Commit-ID:  6a8edbd0c54ae266b12f4f63e406313481c9d4bc
Gitweb:     http://git.kernel.org/tip/6a8edbd0c54ae266b12f4f63e406313481c9d4bc
Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
AuthorDate: Wed, 2 Aug 2017 18:09:15 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 10 Aug 2017 16:50:22 +0200

x86/hyper-v: Introduce fast hypercall implementation

Hyper-V supports 'fast' hypercalls when all parameters are passed through
registers. Implement an inline version of a simpliest of these calls:
hypercall with one 8-byte input and no output.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jork Loeser <Jork.Loeser@microsoft.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: devel@linuxdriverproject.org
Link: http://lkml.kernel.org/r/20170802160921.21791-4-vkuznets@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/include/asm/mshyperv.h | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 6fa5e34..e484255 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -209,6 +209,40 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
 	return hv_status;
 }
 
+#define HV_HYPERCALL_FAST_BIT		BIT(16)
+
+/* Fast hypercall with 8 bytes of input and no output */
+static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1)
+{
+	u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT;
+	register void *__sp asm(_ASM_SP);
+
+#ifdef CONFIG_X86_64
+	{
+		__asm__ __volatile__("call *%4"
+				     : "=a" (hv_status), "+r" (__sp),
+				       "+c" (control), "+d" (input1)
+				     : "m" (hv_hypercall_pg)
+				     : "cc", "r8", "r9", "r10", "r11");
+	}
+#else
+	{
+		u32 input1_hi = upper_32_bits(input1);
+		u32 input1_lo = lower_32_bits(input1);
+
+		__asm__ __volatile__ ("call *%5"
+				      : "=A"(hv_status),
+					"+c"(input1_lo),
+					"+r"(__sp)
+				      :	"A" (control),
+					"b" (input1_hi),
+					"m" (hv_hypercall_pg)
+				      : "cc", "edi", "esi");
+	}
+#endif
+		return hv_status;
+}
+
 void hyperv_init(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);

[toc] | [prev] | [next] | [standalone]

#1702240 — [PATCH v10 7/9] x86/hyper-v: use hypercall for remote TLB flush

Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
this is supposed to work faster than IPIs.

Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
we need to put the input somewhere in memory and we don't really want to
have memory allocation on each call so we pre-allocate per cpu memory areas
on boot.

pv_ops patching is happening very early so we need to separate
hyperv_setup_mmu_ops() and hyper_alloc_mmu().

It is possible and easy to implement local TLB flushing too and there is
even a hint for that. However, I don't see a room for optimization on the
host side as both hypercall and native tlb flush will result in vmexit. The
hint is also not set on modern Hyper-V versions.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   2 +
 arch/x86/hyperv/mmu.c              | 138 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    |   3 +
 arch/x86/include/uapi/asm/hyperv.h |   7 ++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 6 files changed, 152 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/hyperv/mmu.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 171ae09864d7..367a8203cfcf 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1 +1 @@
-obj-y		:= hv_init.o
+obj-y		:= hv_init.o mmu.o
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index e93b9a0b1b10..1a8eb550c40f 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -140,6 +140,8 @@ void hyperv_init(void)
 	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
 	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 
+	hyper_alloc_mmu();
+
 	/*
 	 * Register Hyper-V specific clocksource.
 	 */
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
new file mode 100644
index 000000000000..9419a20b1d75
--- /dev/null
+++ b/arch/x86/hyperv/mmu.c
@@ -0,0 +1,138 @@
+#define pr_fmt(fmt)  "Hyper-V: " fmt
+
+#include <linux/hyperv.h>
+#include <linux/log2.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include <asm/fpu/api.h>
+#include <asm/mshyperv.h>
+#include <asm/msr.h>
+#include <asm/tlbflush.h>
+
+/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
+struct hv_flush_pcpu {
+	u64 address_space;
+	u64 flags;
+	u64 processor_mask;
+	u64 gva_list[];
+};
+
+/* Each gva in gva_list encodes up to 4096 pages to flush */
+#define HV_TLB_FLUSH_UNIT (4096 * PAGE_SIZE)
+
+static struct hv_flush_pcpu __percpu *pcpu_flush;
+
+/*
+ * Fills in gva_list starting from offset. Returns the number of items added.
+ */
+static inline int fill_gva_list(u64 gva_list[], int offset,
+				unsigned long start, unsigned long end)
+{
+	int gva_n = offset;
+	unsigned long cur = start, diff;
+
+	do {
+		diff = end > cur ? end - cur : 0;
+
+		gva_list[gva_n] = cur & PAGE_MASK;
+		/*
+		 * Lower 12 bits encode the number of additional
+		 * pages to flush (in addition to the 'cur' page).
+		 */
+		if (diff >= HV_TLB_FLUSH_UNIT)
+			gva_list[gva_n] |= ~PAGE_MASK;
+		else if (diff)
+			gva_list[gva_n] |= (diff - 1) >> PAGE_SHIFT;
+
+		cur += HV_TLB_FLUSH_UNIT;
+		gva_n++;
+
+	} while (cur < end);
+
+	return gva_n - offset;
+}
+
+static void hyperv_flush_tlb_others(const struct cpumask *cpus,
+				    const struct flush_tlb_info *info)
+{
+	int cpu, vcpu, gva_n, max_gvas;
+	struct hv_flush_pcpu *flush;
+	u64 status = U64_MAX;
+	unsigned long flags;
+
+	if (!pcpu_flush || !hv_hypercall_pg)
+		goto do_native;
+
+	if (cpumask_empty(cpus))
+		return;
+
+	local_irq_save(flags);
+
+	flush = this_cpu_ptr(pcpu_flush);
+
+	if (info->mm) {
+		flush->address_space = virt_to_phys(info->mm->pgd);
+		flush->flags = 0;
+	} else {
+		flush->address_space = 0;
+		flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES;
+	}
+
+	flush->processor_mask = 0;
+	if (cpumask_equal(cpus, cpu_present_mask)) {
+		flush->flags |= HV_FLUSH_ALL_PROCESSORS;
+	} else {
+		for_each_cpu(cpu, cpus) {
+			vcpu = hv_cpu_number_to_vp_number(cpu);
+			if (vcpu >= 64)
+				goto do_native;
+
+			__set_bit(vcpu, (unsigned long *)
+				  &flush->processor_mask);
+		}
+	}
+
+	/*
+	 * We can flush not more than max_gvas with one hypercall. Flush the
+	 * whole address space if we were asked to do more.
+	 */
+	max_gvas = (PAGE_SIZE - sizeof(*flush)) / sizeof(flush->gva_list[0]);
+
+	if (info->end == TLB_FLUSH_ALL) {
+		flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else if (info->end &&
+		   ((info->end - info->start)/HV_TLB_FLUSH_UNIT) > max_gvas) {
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else {
+		gva_n = fill_gva_list(flush->gva_list, 0,
+				      info->start, info->end);
+		status = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
+					     gva_n, 0, flush, NULL);
+	}
+
+	local_irq_restore(flags);
+
+	if (!(status & HV_HYPERCALL_RESULT_MASK))
+		return;
+do_native:
+	native_flush_tlb_others(cpus, info);
+}
+
+void hyperv_setup_mmu_ops(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pr_info("Using hypercall for remote TLB flush\n");
+		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
+		setup_clear_cpu_cap(X86_FEATURE_PCID);
+	}
+}
+
+void hyper_alloc_mmu(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED)
+		pcpu_flush = __alloc_percpu(PAGE_SIZE, PAGE_SIZE);
+}
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index efd2f80d3353..0d4b01c5e438 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -307,6 +307,8 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)
 }
 
 void hyperv_init(void);
+void hyperv_setup_mmu_ops(void);
+void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
@@ -314,6 +316,7 @@ void hyperv_cleanup(void);
 static inline void hyperv_init(void) {}
 static inline bool hv_is_hypercall_page_setup(void) { return false; }
 static inline void hyperv_cleanup(void) {}
+static inline void hyperv_setup_mmu_ops(void) {}
 #endif /* CONFIG_HYPERV */
 
 #ifdef CONFIG_HYPERV_TSCPAGE
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 127ddadee1a5..a6fdd3b82b4a 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -242,6 +242,8 @@
 		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
 
 /* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
 #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
 #define HVCALL_POST_MESSAGE			0x005c
 #define HVCALL_SIGNAL_EVENT			0x005d
@@ -259,6 +261,11 @@
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
+#define HV_FLUSH_ALL_PROCESSORS			BIT(0)
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1)
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2)
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3)
+
 /* hypercall status code */
 #define HV_STATUS_SUCCESS			0
 #define HV_STATUS_INVALID_HYPERCALL_CODE	2
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 70e717fccdd6..daefd67a66c7 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -249,6 +249,7 @@ static void __init ms_hyperv_init_platform(void)
 	 * Setup the hook to get control post apic initialization.
 	 */
 	x86_platform.apic_post_init = hyperv_init;
+	hyperv_setup_mmu_ops();
 #endif
 }
 
-- 
2.13.3

[toc] | [prev] | [next] | [standalone]

#1708881 — [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

Commit-ID:  88b46342eb037d35decda4d651cfee5216f4f822
Gitweb:     http://git.kernel.org/tip/88b46342eb037d35decda4d651cfee5216f4f822
Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
AuthorDate: Wed, 2 Aug 2017 18:09:19 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 10 Aug 2017 16:50:23 +0200

x86/hyper-v: Use hypercall for remote TLB flush

Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
this is supposed to work faster than IPIs.

Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
we need to put the input somewhere in memory and we don't really want to
have memory allocation on each call so we pre-allocate per cpu memory areas
on boot.

pv_ops patching is happening very early so we need to separate
hyperv_setup_mmu_ops() and hyper_alloc_mmu().

It is possible and easy to implement local TLB flushing too and there is
even a hint for that. However, I don't see a room for optimization on the
host side as both hypercall and native tlb flush will result in vmexit. The
hint is also not set on modern Hyper-V versions.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jork Loeser <Jork.Loeser@microsoft.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: devel@linuxdriverproject.org
Link: http://lkml.kernel.org/r/20170802160921.21791-8-vkuznets@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   2 +
 arch/x86/hyperv/mmu.c              | 138 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    |   3 +
 arch/x86/include/uapi/asm/hyperv.h |   7 ++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 6 files changed, 152 insertions(+), 1 deletion(-)

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 171ae09..367a820 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1 +1 @@
-obj-y		:= hv_init.o
+obj-y		:= hv_init.o mmu.o
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index e93b9a0..1a8eb55 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -140,6 +140,8 @@ void hyperv_init(void)
 	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
 	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 
+	hyper_alloc_mmu();
+
 	/*
 	 * Register Hyper-V specific clocksource.
 	 */
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
new file mode 100644
index 0000000..9419a20
--- /dev/null
+++ b/arch/x86/hyperv/mmu.c
@@ -0,0 +1,138 @@
+#define pr_fmt(fmt)  "Hyper-V: " fmt
+
+#include <linux/hyperv.h>
+#include <linux/log2.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include <asm/fpu/api.h>
+#include <asm/mshyperv.h>
+#include <asm/msr.h>
+#include <asm/tlbflush.h>
+
+/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
+struct hv_flush_pcpu {
+	u64 address_space;
+	u64 flags;
+	u64 processor_mask;
+	u64 gva_list[];
+};
+
+/* Each gva in gva_list encodes up to 4096 pages to flush */
+#define HV_TLB_FLUSH_UNIT (4096 * PAGE_SIZE)
+
+static struct hv_flush_pcpu __percpu *pcpu_flush;
+
+/*
+ * Fills in gva_list starting from offset. Returns the number of items added.
+ */
+static inline int fill_gva_list(u64 gva_list[], int offset,
+				unsigned long start, unsigned long end)
+{
+	int gva_n = offset;
+	unsigned long cur = start, diff;
+
+	do {
+		diff = end > cur ? end - cur : 0;
+
+		gva_list[gva_n] = cur & PAGE_MASK;
+		/*
+		 * Lower 12 bits encode the number of additional
+		 * pages to flush (in addition to the 'cur' page).
+		 */
+		if (diff >= HV_TLB_FLUSH_UNIT)
+			gva_list[gva_n] |= ~PAGE_MASK;
+		else if (diff)
+			gva_list[gva_n] |= (diff - 1) >> PAGE_SHIFT;
+
+		cur += HV_TLB_FLUSH_UNIT;
+		gva_n++;
+
+	} while (cur < end);
+
+	return gva_n - offset;
+}
+
+static void hyperv_flush_tlb_others(const struct cpumask *cpus,
+				    const struct flush_tlb_info *info)
+{
+	int cpu, vcpu, gva_n, max_gvas;
+	struct hv_flush_pcpu *flush;
+	u64 status = U64_MAX;
+	unsigned long flags;
+
+	if (!pcpu_flush || !hv_hypercall_pg)
+		goto do_native;
+
+	if (cpumask_empty(cpus))
+		return;
+
+	local_irq_save(flags);
+
+	flush = this_cpu_ptr(pcpu_flush);
+
+	if (info->mm) {
+		flush->address_space = virt_to_phys(info->mm->pgd);
+		flush->flags = 0;
+	} else {
+		flush->address_space = 0;
+		flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES;
+	}
+
+	flush->processor_mask = 0;
+	if (cpumask_equal(cpus, cpu_present_mask)) {
+		flush->flags |= HV_FLUSH_ALL_PROCESSORS;
+	} else {
+		for_each_cpu(cpu, cpus) {
+			vcpu = hv_cpu_number_to_vp_number(cpu);
+			if (vcpu >= 64)
+				goto do_native;
+
+			__set_bit(vcpu, (unsigned long *)
+				  &flush->processor_mask);
+		}
+	}
+
+	/*
+	 * We can flush not more than max_gvas with one hypercall. Flush the
+	 * whole address space if we were asked to do more.
+	 */
+	max_gvas = (PAGE_SIZE - sizeof(*flush)) / sizeof(flush->gva_list[0]);
+
+	if (info->end == TLB_FLUSH_ALL) {
+		flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else if (info->end &&
+		   ((info->end - info->start)/HV_TLB_FLUSH_UNIT) > max_gvas) {
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else {
+		gva_n = fill_gva_list(flush->gva_list, 0,
+				      info->start, info->end);
+		status = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
+					     gva_n, 0, flush, NULL);
+	}
+
+	local_irq_restore(flags);
+
+	if (!(status & HV_HYPERCALL_RESULT_MASK))
+		return;
+do_native:
+	native_flush_tlb_others(cpus, info);
+}
+
+void hyperv_setup_mmu_ops(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pr_info("Using hypercall for remote TLB flush\n");
+		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
+		setup_clear_cpu_cap(X86_FEATURE_PCID);
+	}
+}
+
+void hyper_alloc_mmu(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED)
+		pcpu_flush = __alloc_percpu(PAGE_SIZE, PAGE_SIZE);
+}
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index efd2f80..0d4b01c 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -307,6 +307,8 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)
 }
 
 void hyperv_init(void);
+void hyperv_setup_mmu_ops(void);
+void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
@@ -314,6 +316,7 @@ void hyperv_cleanup(void);
 static inline void hyperv_init(void) {}
 static inline bool hv_is_hypercall_page_setup(void) { return false; }
 static inline void hyperv_cleanup(void) {}
+static inline void hyperv_setup_mmu_ops(void) {}
 #endif /* CONFIG_HYPERV */
 
 #ifdef CONFIG_HYPERV_TSCPAGE
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 127ddad..a6fdd3b 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -242,6 +242,8 @@
 		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
 
 /* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
 #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
 #define HVCALL_POST_MESSAGE			0x005c
 #define HVCALL_SIGNAL_EVENT			0x005d
@@ -259,6 +261,11 @@
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
+#define HV_FLUSH_ALL_PROCESSORS			BIT(0)
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1)
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2)
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3)
+
 /* hypercall status code */
 #define HV_STATUS_SUCCESS			0
 #define HV_STATUS_INVALID_HYPERCALL_CODE	2
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 70e717f..daefd67 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -249,6 +249,7 @@ static void __init ms_hyperv_init_platform(void)
 	 * Setup the hook to get control post apic initialization.
 	 */
 	x86_platform.apic_post_init = hyperv_init;
+	hyperv_setup_mmu_ops();
 #endif
 }
 

[toc] | [prev] | [next] | [standalone]

#1708953 — [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

Commit-ID:  2ffd9e33ce4af4e8cfa3e17bf493defe8474e2eb
Gitweb:     http://git.kernel.org/tip/2ffd9e33ce4af4e8cfa3e17bf493defe8474e2eb
Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
AuthorDate: Wed, 2 Aug 2017 18:09:19 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 10 Aug 2017 20:16:44 +0200

x86/hyper-v: Use hypercall for remote TLB flush

Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
this is supposed to work faster than IPIs.

Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
we need to put the input somewhere in memory and we don't really want to
have memory allocation on each call so we pre-allocate per cpu memory areas
on boot.

pv_ops patching is happening very early so we need to separate
hyperv_setup_mmu_ops() and hyper_alloc_mmu().

It is possible and easy to implement local TLB flushing too and there is
even a hint for that. However, I don't see a room for optimization on the
host side as both hypercall and native tlb flush will result in vmexit. The
hint is also not set on modern Hyper-V versions.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jork Loeser <Jork.Loeser@microsoft.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: devel@linuxdriverproject.org
Link: http://lkml.kernel.org/r/20170802160921.21791-8-vkuznets@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/hyperv/Makefile           |   2 +-
 arch/x86/hyperv/hv_init.c          |   2 +
 arch/x86/hyperv/mmu.c              | 138 +++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h    |   3 +
 arch/x86/include/uapi/asm/hyperv.h |   7 ++
 arch/x86/kernel/cpu/mshyperv.c     |   1 +
 drivers/hv/Kconfig                 |   1 +
 7 files changed, 153 insertions(+), 1 deletion(-)

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 171ae09..367a820 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1 +1 @@
-obj-y		:= hv_init.o
+obj-y		:= hv_init.o mmu.o
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index e93b9a0..1a8eb55 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -140,6 +140,8 @@ void hyperv_init(void)
 	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
 	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 
+	hyper_alloc_mmu();
+
 	/*
 	 * Register Hyper-V specific clocksource.
 	 */
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
new file mode 100644
index 0000000..9419a20
--- /dev/null
+++ b/arch/x86/hyperv/mmu.c
@@ -0,0 +1,138 @@
+#define pr_fmt(fmt)  "Hyper-V: " fmt
+
+#include <linux/hyperv.h>
+#include <linux/log2.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include <asm/fpu/api.h>
+#include <asm/mshyperv.h>
+#include <asm/msr.h>
+#include <asm/tlbflush.h>
+
+/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */
+struct hv_flush_pcpu {
+	u64 address_space;
+	u64 flags;
+	u64 processor_mask;
+	u64 gva_list[];
+};
+
+/* Each gva in gva_list encodes up to 4096 pages to flush */
+#define HV_TLB_FLUSH_UNIT (4096 * PAGE_SIZE)
+
+static struct hv_flush_pcpu __percpu *pcpu_flush;
+
+/*
+ * Fills in gva_list starting from offset. Returns the number of items added.
+ */
+static inline int fill_gva_list(u64 gva_list[], int offset,
+				unsigned long start, unsigned long end)
+{
+	int gva_n = offset;
+	unsigned long cur = start, diff;
+
+	do {
+		diff = end > cur ? end - cur : 0;
+
+		gva_list[gva_n] = cur & PAGE_MASK;
+		/*
+		 * Lower 12 bits encode the number of additional
+		 * pages to flush (in addition to the 'cur' page).
+		 */
+		if (diff >= HV_TLB_FLUSH_UNIT)
+			gva_list[gva_n] |= ~PAGE_MASK;
+		else if (diff)
+			gva_list[gva_n] |= (diff - 1) >> PAGE_SHIFT;
+
+		cur += HV_TLB_FLUSH_UNIT;
+		gva_n++;
+
+	} while (cur < end);
+
+	return gva_n - offset;
+}
+
+static void hyperv_flush_tlb_others(const struct cpumask *cpus,
+				    const struct flush_tlb_info *info)
+{
+	int cpu, vcpu, gva_n, max_gvas;
+	struct hv_flush_pcpu *flush;
+	u64 status = U64_MAX;
+	unsigned long flags;
+
+	if (!pcpu_flush || !hv_hypercall_pg)
+		goto do_native;
+
+	if (cpumask_empty(cpus))
+		return;
+
+	local_irq_save(flags);
+
+	flush = this_cpu_ptr(pcpu_flush);
+
+	if (info->mm) {
+		flush->address_space = virt_to_phys(info->mm->pgd);
+		flush->flags = 0;
+	} else {
+		flush->address_space = 0;
+		flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES;
+	}
+
+	flush->processor_mask = 0;
+	if (cpumask_equal(cpus, cpu_present_mask)) {
+		flush->flags |= HV_FLUSH_ALL_PROCESSORS;
+	} else {
+		for_each_cpu(cpu, cpus) {
+			vcpu = hv_cpu_number_to_vp_number(cpu);
+			if (vcpu >= 64)
+				goto do_native;
+
+			__set_bit(vcpu, (unsigned long *)
+				  &flush->processor_mask);
+		}
+	}
+
+	/*
+	 * We can flush not more than max_gvas with one hypercall. Flush the
+	 * whole address space if we were asked to do more.
+	 */
+	max_gvas = (PAGE_SIZE - sizeof(*flush)) / sizeof(flush->gva_list[0]);
+
+	if (info->end == TLB_FLUSH_ALL) {
+		flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY;
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else if (info->end &&
+		   ((info->end - info->start)/HV_TLB_FLUSH_UNIT) > max_gvas) {
+		status = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE,
+					 flush, NULL);
+	} else {
+		gva_n = fill_gva_list(flush->gva_list, 0,
+				      info->start, info->end);
+		status = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST,
+					     gva_n, 0, flush, NULL);
+	}
+
+	local_irq_restore(flags);
+
+	if (!(status & HV_HYPERCALL_RESULT_MASK))
+		return;
+do_native:
+	native_flush_tlb_others(cpus, info);
+}
+
+void hyperv_setup_mmu_ops(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) {
+		pr_info("Using hypercall for remote TLB flush\n");
+		pv_mmu_ops.flush_tlb_others = hyperv_flush_tlb_others;
+		setup_clear_cpu_cap(X86_FEATURE_PCID);
+	}
+}
+
+void hyper_alloc_mmu(void)
+{
+	if (ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED)
+		pcpu_flush = __alloc_percpu(PAGE_SIZE, PAGE_SIZE);
+}
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index efd2f80..0d4b01c 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -307,6 +307,8 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)
 }
 
 void hyperv_init(void);
+void hyperv_setup_mmu_ops(void);
+void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
@@ -314,6 +316,7 @@ void hyperv_cleanup(void);
 static inline void hyperv_init(void) {}
 static inline bool hv_is_hypercall_page_setup(void) { return false; }
 static inline void hyperv_cleanup(void) {}
+static inline void hyperv_setup_mmu_ops(void) {}
 #endif /* CONFIG_HYPERV */
 
 #ifdef CONFIG_HYPERV_TSCPAGE
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 127ddad..a6fdd3b 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -242,6 +242,8 @@
 		(~((1ull << HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT) - 1))
 
 /* Declare the various hypercall operations. */
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE	0x0002
+#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST	0x0003
 #define HVCALL_NOTIFY_LONG_SPIN_WAIT		0x0008
 #define HVCALL_POST_MESSAGE			0x005c
 #define HVCALL_SIGNAL_EVENT			0x005d
@@ -259,6 +261,11 @@
 #define HV_PROCESSOR_POWER_STATE_C2		2
 #define HV_PROCESSOR_POWER_STATE_C3		3
 
+#define HV_FLUSH_ALL_PROCESSORS			BIT(0)
+#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES	BIT(1)
+#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY	BIT(2)
+#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT	BIT(3)
+
 /* hypercall status code */
 #define HV_STATUS_SUCCESS			0
 #define HV_STATUS_INVALID_HYPERCALL_CODE	2
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 70e717f..daefd67 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -249,6 +249,7 @@ static void __init ms_hyperv_init_platform(void)
 	 * Setup the hook to get control post apic initialization.
 	 */
 	x86_platform.apic_post_init = hyperv_init;
+	hyperv_setup_mmu_ops();
 #endif
 }
 
diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
index c29cd53..50b89ea 100644
--- a/drivers/hv/Kconfig
+++ b/drivers/hv/Kconfig
@@ -3,6 +3,7 @@ menu "Microsoft Hyper-V guest support"
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
 	depends on X86 && ACPI && PCI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+	select PARAVIRT
 	help
 	  Select this option to run Linux as a Hyper-V client operating
 	  system.

[toc] | [prev] | [next] | [standalone]

#1708967 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Thu, Aug 10, 2017 at 11:21:49AM -0700, tip-bot for Vitaly Kuznetsov wrote:
> Commit-ID:  2ffd9e33ce4af4e8cfa3e17bf493defe8474e2eb
> Gitweb:     http://git.kernel.org/tip/2ffd9e33ce4af4e8cfa3e17bf493defe8474e2eb
> Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
> AuthorDate: Wed, 2 Aug 2017 18:09:19 +0200
> Committer:  Ingo Molnar <mingo@kernel.org>
> CommitDate: Thu, 10 Aug 2017 20:16:44 +0200
> 
> x86/hyper-v: Use hypercall for remote TLB flush
> 
> Hyper-V host can suggest us to use hypercall for doing remote TLB flush,
> this is supposed to work faster than IPIs.
> 
> Implementation details: to do HvFlushVirtualAddress{Space,List} hypercalls
> we need to put the input somewhere in memory and we don't really want to
> have memory allocation on each call so we pre-allocate per cpu memory areas
> on boot.
> 
> pv_ops patching is happening very early so we need to separate
> hyperv_setup_mmu_ops() and hyper_alloc_mmu().
> 
> It is possible and easy to implement local TLB flushing too and there is
> even a hint for that. However, I don't see a room for optimization on the
> host side as both hypercall and native tlb flush will result in vmexit. The
> hint is also not set on modern Hyper-V versions.

Hold on.. if we don't IPI for TLB invalidation. What serializes our
software page table walkers like fast_gup() ?

[toc] | [prev] | [next] | [standalone]

#1708982 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Thu, Aug 10, 2017 at 07:08:22PM +0000, Jork Loeser wrote:

> > > Subject: Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush
> 
> > > Hold on.. if we don't IPI for TLB invalidation. What serializes our
> > > software page table walkers like fast_gup() ?
> > 
> > Hypervisor may implement this functionality via an IPI.
> > 
> > K. Y
> 
> HvFlushVirtualAddressList() states:
> This call guarantees that by the time control returns back to the
> caller, the observable effects of all flushes on the specified virtual
> processors have occurred.
> 
> HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding sparse target VP lists.
> 
> Is this enough of a guarantee, or do you see other races?

That's nowhere near enough. We need the remote CPU to have completed any
guest IF section that was in progress at the time of the call.

So if a host IPI can interrupt a guest while the guest has IF cleared,
and we then process the host IPI -- clear the TLBs -- before resuming the
guest, which still has IF cleared, we've got a problem.

Because at that point, our software page-table walker, that relies on IF
being clear to guarantee the page-tables exist, because it holds off the
TLB invalidate and thereby the freeing of the pages, gets its pages
ripped out from under it.

[toc] | [prev] | [next] | [standalone]

#1709389 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 01:15:18AM +0000, Jork Loeser wrote:

> > > HvFlushVirtualAddressList() states:
> > > This call guarantees that by the time control returns back to the
> > > caller, the observable effects of all flushes on the specified virtual
> > > processors have occurred.
> > >
> > > HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding
> > > sparse target VP lists.
> > >
> > > Is this enough of a guarantee, or do you see other races?
> > 
> > That's nowhere near enough. We need the remote CPU to have completed any
> > guest IF section that was in progress at the time of the call.
> > 
> > So if a host IPI can interrupt a guest while the guest has IF cleared, and we then
> > process the host IPI -- clear the TLBs -- before resuming the guest, which still has
> > IF cleared, we've got a problem.
> > 
> > Because at that point, our software page-table walker, that relies on IF being
> > clear to guarantee the page-tables exist, because it holds off the TLB invalidate
> > and thereby the freeing of the pages, gets its pages ripped out from under it.
> 
> I see, IF is used as a locking mechanism for the pages. Would
> CONFIG_HAVE_RCU_TABLE_FREE be an option for x86? There are caveats
> (statically enabled, RCU for page-free), yet if the resulting perf is
> still a gain it would be worthwhile for Hyper-V targeted kernels.

I'm sure we talked about using HAVE_RCU_TABLE_FREE for x86 (and yes that
would make it work again), but this was some years ago and I cannot
readily find those emails.

Kirill would you have any opinions?

[toc] | [prev] | [next] | [standalone]

#1709474 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 11:03:36AM +0200, Peter Zijlstra wrote:
> On Fri, Aug 11, 2017 at 01:15:18AM +0000, Jork Loeser wrote:
> 
> > > > HvFlushVirtualAddressList() states:
> > > > This call guarantees that by the time control returns back to the
> > > > caller, the observable effects of all flushes on the specified virtual
> > > > processors have occurred.
> > > >
> > > > HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding
> > > > sparse target VP lists.
> > > >
> > > > Is this enough of a guarantee, or do you see other races?
> > > 
> > > That's nowhere near enough. We need the remote CPU to have completed any
> > > guest IF section that was in progress at the time of the call.
> > > 
> > > So if a host IPI can interrupt a guest while the guest has IF cleared, and we then
> > > process the host IPI -- clear the TLBs -- before resuming the guest, which still has
> > > IF cleared, we've got a problem.
> > > 
> > > Because at that point, our software page-table walker, that relies on IF being
> > > clear to guarantee the page-tables exist, because it holds off the TLB invalidate
> > > and thereby the freeing of the pages, gets its pages ripped out from under it.
> > 
> > I see, IF is used as a locking mechanism for the pages. Would
> > CONFIG_HAVE_RCU_TABLE_FREE be an option for x86? There are caveats
> > (statically enabled, RCU for page-free), yet if the resulting perf is
> > still a gain it would be worthwhile for Hyper-V targeted kernels.
> 
> I'm sure we talked about using HAVE_RCU_TABLE_FREE for x86 (and yes that
> would make it work again), but this was some years ago and I cannot
> readily find those emails.
> 
> Kirill would you have any opinions?

I guess we can try this. The main question is what would be performance
implications of such move.

-- 
 Kirill A. Shutemov

[toc] | [prev] | [next] | [standalone]

#1709811 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 2:03 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>
> I'm sure we talked about using HAVE_RCU_TABLE_FREE for x86 (and yes that
> would make it work again), but this was some years ago and I cannot
> readily find those emails.

I think the only time we really talked about HAVE_RCU_TABLE_FREE for
x86 (at least that I was cc'd on) was not because of RCU freeing, but
because we just wanted to use the generic page table lookup code on
x86 *despite* not using RCU freeing.

And we just ended up renaming HAVE_GENERIC_RCU_GUP as HAVE_GENERIC_GUP.

There was only passing mention of maybe making x86 use RCU, but the
discussion was really about why the IF flag meant that x86 didn't need
to, iirc.

I don't recall us ever discussing *really* making x86 use RCU.

                    Linus

[toc] | [prev] | [next] | [standalone]

#1709823 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 09:16:29AM -0700, Linus Torvalds wrote:
> On Fri, Aug 11, 2017 at 2:03 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > I'm sure we talked about using HAVE_RCU_TABLE_FREE for x86 (and yes that
> > would make it work again), but this was some years ago and I cannot
> > readily find those emails.
> 
> I think the only time we really talked about HAVE_RCU_TABLE_FREE for
> x86 (at least that I was cc'd on) was not because of RCU freeing, but
> because we just wanted to use the generic page table lookup code on
> x86 *despite* not using RCU freeing.
> 
> And we just ended up renaming HAVE_GENERIC_RCU_GUP as HAVE_GENERIC_GUP.
> 
> There was only passing mention of maybe making x86 use RCU, but the
> discussion was really about why the IF flag meant that x86 didn't need
> to, iirc.
> 
> I don't recall us ever discussing *really* making x86 use RCU.

Google finds me this:

  https://lwn.net/Articles/500188/

Which includes:

  http://www.mail-archive.com/kvm@vger.kernel.org/msg72918.html

which does as was suggested here, selects HAVE_RCU_TABLE_FREE for
PARAVIRT_TLB_FLUSH.

But yes, this is very much virt specific nonsense, native would never
need this.

[toc] | [prev] | [next] | [standalone]

#1709403 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

Peter Zijlstra <peterz@infradead.org> writes:

> On Thu, Aug 10, 2017 at 07:08:22PM +0000, Jork Loeser wrote:
>
>> > > Subject: Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush
>> 
>> > > Hold on.. if we don't IPI for TLB invalidation. What serializes our
>> > > software page table walkers like fast_gup() ?
>> > 
>> > Hypervisor may implement this functionality via an IPI.
>> > 
>> > K. Y
>> 
>> HvFlushVirtualAddressList() states:
>> This call guarantees that by the time control returns back to the
>> caller, the observable effects of all flushes on the specified virtual
>> processors have occurred.
>> 
>> HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding sparse target VP lists.
>> 
>> Is this enough of a guarantee, or do you see other races?
>
> That's nowhere near enough. We need the remote CPU to have completed any
> guest IF section that was in progress at the time of the call.
>
> So if a host IPI can interrupt a guest while the guest has IF cleared,
> and we then process the host IPI -- clear the TLBs -- before resuming the
> guest, which still has IF cleared, we've got a problem.
>
> Because at that point, our software page-table walker, that relies on IF
> being clear to guarantee the page-tables exist, because it holds off the
> TLB invalidate and thereby the freeing of the pages, gets its pages
> ripped out from under it.

Oh, I see your concern. Hyper-V, however, is not the first x86
hypervisor trying to avoid IPIs on remote TLB flush, Xen does this
too. Briefly looking at xen_flush_tlb_others() I don't see anything
special, do we know how serialization is achieved there?

-- 
  Vitaly

[toc] | [prev] | [next] | [standalone]

#1709457 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 11:23:10AM +0200, Vitaly Kuznetsov wrote:
> Peter Zijlstra <peterz@infradead.org> writes:
> 
> > On Thu, Aug 10, 2017 at 07:08:22PM +0000, Jork Loeser wrote:
> >
> >> > > Subject: Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush
> >> 
> >> > > Hold on.. if we don't IPI for TLB invalidation. What serializes our
> >> > > software page table walkers like fast_gup() ?
> >> > 
> >> > Hypervisor may implement this functionality via an IPI.
> >> > 
> >> > K. Y
> >> 
> >> HvFlushVirtualAddressList() states:
> >> This call guarantees that by the time control returns back to the
> >> caller, the observable effects of all flushes on the specified virtual
> >> processors have occurred.
> >> 
> >> HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding sparse target VP lists.
> >> 
> >> Is this enough of a guarantee, or do you see other races?
> >
> > That's nowhere near enough. We need the remote CPU to have completed any
> > guest IF section that was in progress at the time of the call.
> >
> > So if a host IPI can interrupt a guest while the guest has IF cleared,
> > and we then process the host IPI -- clear the TLBs -- before resuming the
> > guest, which still has IF cleared, we've got a problem.
> >
> > Because at that point, our software page-table walker, that relies on IF
> > being clear to guarantee the page-tables exist, because it holds off the
> > TLB invalidate and thereby the freeing of the pages, gets its pages
> > ripped out from under it.
> 
> Oh, I see your concern. Hyper-V, however, is not the first x86
> hypervisor trying to avoid IPIs on remote TLB flush, Xen does this
> too. Briefly looking at xen_flush_tlb_others() I don't see anything
> special, do we know how serialization is achieved there?

No idea on how Xen works, I always just hope it goes away :-) But lets
ask some Xen folks.

[toc] | [prev] | [next] | [standalone]

#1709461 — Re: [Xen-devel] [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On 11/08/17 11:56, Peter Zijlstra wrote:
> On Fri, Aug 11, 2017 at 11:23:10AM +0200, Vitaly Kuznetsov wrote:
>> Peter Zijlstra <peterz@infradead.org> writes:
>>
>>> On Thu, Aug 10, 2017 at 07:08:22PM +0000, Jork Loeser wrote:
>>>
>>>>>> Subject: Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush
>>>>>> Hold on.. if we don't IPI for TLB invalidation. What serializes our
>>>>>> software page table walkers like fast_gup() ?
>>>>> Hypervisor may implement this functionality via an IPI.
>>>>>
>>>>> K. Y
>>>> HvFlushVirtualAddressList() states:
>>>> This call guarantees that by the time control returns back to the
>>>> caller, the observable effects of all flushes on the specified virtual
>>>> processors have occurred.
>>>>
>>>> HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding sparse target VP lists.
>>>>
>>>> Is this enough of a guarantee, or do you see other races?
>>> That's nowhere near enough. We need the remote CPU to have completed any
>>> guest IF section that was in progress at the time of the call.
>>>
>>> So if a host IPI can interrupt a guest while the guest has IF cleared,
>>> and we then process the host IPI -- clear the TLBs -- before resuming the
>>> guest, which still has IF cleared, we've got a problem.
>>>
>>> Because at that point, our software page-table walker, that relies on IF
>>> being clear to guarantee the page-tables exist, because it holds off the
>>> TLB invalidate and thereby the freeing of the pages, gets its pages
>>> ripped out from under it.
>> Oh, I see your concern. Hyper-V, however, is not the first x86
>> hypervisor trying to avoid IPIs on remote TLB flush, Xen does this
>> too. Briefly looking at xen_flush_tlb_others() I don't see anything
>> special, do we know how serialization is achieved there?
> No idea on how Xen works, I always just hope it goes away :-) But lets
> ask some Xen folks.

How is the software pagewalker relying on IF being clear safe at all (on
native, let alone under virtualisation)?  Hardware has no architectural
requirement to keep entries in the TLB.

In the virtualisation case, at any point the vcpu can be scheduled on a
different pcpu even during a critical region like that, so the TLB
really can empty itself under your feet.

~Andrew

[toc] | [prev] | [next] | [standalone]

#1709521 — Re: [Xen-devel] [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 12:05:45PM +0100, Andrew Cooper wrote:
> >> Oh, I see your concern. Hyper-V, however, is not the first x86
> >> hypervisor trying to avoid IPIs on remote TLB flush, Xen does this
> >> too. Briefly looking at xen_flush_tlb_others() I don't see anything
> >> special, do we know how serialization is achieved there?
> > No idea on how Xen works, I always just hope it goes away :-) But lets
> > ask some Xen folks.
> 
> How is the software pagewalker relying on IF being clear safe at all (on
> native, let alone under virtualisation)?  Hardware has no architectural
> requirement to keep entries in the TLB.

No, but it _can_, therefore when we unhook pages we _must_ invalidate.

It goes like:

	CPU0			CPU1

	unhook page
				cli
				traverse page tables
	TLB invalidate --->	<IF clear, therefore CPU0 waits>
				sti
				<IPI>
				 TLB invalidate
			<------	 complete
				</IPI>
	free page

So the CPU1 page-table walker gets an existence guarantee of the
page-tables by clearing IF.

> In the virtualisation case, at any point the vcpu can be scheduled on a
> different pcpu even during a critical region like that, so the TLB
> really can empty itself under your feet.

Not the point.

[toc] | [prev] | [next] | [standalone]

#1709528 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On 11/08/17 12:56, Peter Zijlstra wrote:
> On Fri, Aug 11, 2017 at 11:23:10AM +0200, Vitaly Kuznetsov wrote:
>> Peter Zijlstra <peterz@infradead.org> writes:
>>
>>> On Thu, Aug 10, 2017 at 07:08:22PM +0000, Jork Loeser wrote:
>>>
>>>>>> Subject: Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush
>>>>
>>>>>> Hold on.. if we don't IPI for TLB invalidation. What serializes our
>>>>>> software page table walkers like fast_gup() ?
>>>>>
>>>>> Hypervisor may implement this functionality via an IPI.
>>>>>
>>>>> K. Y
>>>>
>>>> HvFlushVirtualAddressList() states:
>>>> This call guarantees that by the time control returns back to the
>>>> caller, the observable effects of all flushes on the specified virtual
>>>> processors have occurred.
>>>>
>>>> HvFlushVirtualAddressListEx() refers to HvFlushVirtualAddressList() as adding sparse target VP lists.
>>>>
>>>> Is this enough of a guarantee, or do you see other races?
>>>
>>> That's nowhere near enough. We need the remote CPU to have completed any
>>> guest IF section that was in progress at the time of the call.
>>>
>>> So if a host IPI can interrupt a guest while the guest has IF cleared,
>>> and we then process the host IPI -- clear the TLBs -- before resuming the
>>> guest, which still has IF cleared, we've got a problem.
>>>
>>> Because at that point, our software page-table walker, that relies on IF
>>> being clear to guarantee the page-tables exist, because it holds off the
>>> TLB invalidate and thereby the freeing of the pages, gets its pages
>>> ripped out from under it.
>>
>> Oh, I see your concern. Hyper-V, however, is not the first x86
>> hypervisor trying to avoid IPIs on remote TLB flush, Xen does this
>> too. Briefly looking at xen_flush_tlb_others() I don't see anything
>> special, do we know how serialization is achieved there?
> 
> No idea on how Xen works, I always just hope it goes away :-) But lets
> ask some Xen folks.

Wait - the TLB can be cleared at any time, as Andrew was pointing out.
No cpu can rely on an address being accessible just because IF is being
cleared. All that matters is the existing and valid page table entry.

So clearing IF on a cpu isn't meant to secure the TLB from being
cleared, but just to avoid interrupts (as the name of the flag is
suggesting).

In the Xen case the hypervisor does the following:

- it checks whether any of the vcpus specified in the cpumask of the
  flush request is running on any physical cpu
- if any running vcpu is found an IPI will be sent to the physical cpu
  and the hypervisor will do the TLB flush there
- any vcpu addressed by the flush and not running will be flagged to
  flush its TLB when being scheduled the next time

This ensures no TLB entry to be flushed can be used after return of
xen_flush_tlb_others().


Juergen

[toc] | [prev] | [next] | [standalone]

#1709538 — Re: [tip:x86/platform] x86/hyper-v: Use hypercall for remote TLB flush

On Fri, Aug 11, 2017 at 02:22:25PM +0200, Juergen Gross wrote:
> Wait - the TLB can be cleared at any time, as Andrew was pointing out.
> No cpu can rely on an address being accessible just because IF is being
> cleared. All that matters is the existing and valid page table entry.
> 
> So clearing IF on a cpu isn't meant to secure the TLB from being
> cleared, but just to avoid interrupts (as the name of the flag is
> suggesting).

Yes, but by holding off the TLB invalidate IPI, we hold off the freeing
of the concurrently unhooked page-table.

> In the Xen case the hypervisor does the following:
> 
> - it checks whether any of the vcpus specified in the cpumask of the
>   flush request is running on any physical cpu
> - if any running vcpu is found an IPI will be sent to the physical cpu
>   and the hypervisor will do the TLB flush there

And this will preempt a vcpu which could have IF cleared, right?

> - any vcpu addressed by the flush and not running will be flagged to
>   flush its TLB when being scheduled the next time
> 
> This ensures no TLB entry to be flushed can be used after return of
> xen_flush_tlb_others().

But that is not a sufficient guarantee. We need the IF to hold off the
TLB invalidate and thereby hold off the freeing of our page-table pages.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web