Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen

From	Haozhong Zhang <haozhong.zhang@intel.com>
Newsgroups	linux.kernel
Subject	Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen
Date	2016-10-11 09:20 +0200
Message-ID	<sqZvX-3En-7@gated-at.bofh.it> (permalink)
References	<sqwNj-2FC-3@gated-at.bofh.it> <sqzLb-4wh-5@gated-at.bofh.it> <sqCpH-6ic-9@gated-at.bofh.it> <sqLCG-3rZ-13@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On 10/10/16 09:24, Dan Williams wrote:
> On Sun, Oct 9, 2016 at 11:32 PM, Haozhong Zhang
> <haozhong.zhang@intel.com> wrote:
> > On 10/09/16 20:45, Dan Williams wrote:
> >> On Sun, Oct 9, 2016 at 5:35 PM, Haozhong Zhang <haozhong.zhang@intel.com> wrote:
> >> > Overview
> >> > ========
> >> > This RFC kernel patch series along with corresponding patch series of
> >> > Xen, QEMU and ndctl implements Xen vNVDIMM, which can map the host
> >> > NVDIMM devices to Xen HVM domU as vNVDIMM devices.
> >> >
> >> > Xen hypervisor does not include an NVDIMM driver, so it needs the
> >> > assistance from the driver in Dom0 Linux kernel to manage NVDIMM
> >> > devices. We currently only supports NVDIMM devices in pmem mode.
> >> >
> >> > Design and Implementation
> >> > =========================
> >> > The complete design can be found at
> >> >   https://lists.xenproject.org/archives/html/xen-devel/2016-07/msg01921.html.
> >>
> >> The KVM enabling for persistent memory does not need this support from
> >> the kernel, and as far as I can see neither does Xen. If the
> >> hypervisor needs to reserve some space it can simply trim the amount
> >> that it hands to the guest.
> >>
> >
> > Xen does not have the NVDIMM driver, so it cannot operate on NVDIMM
> > devices by itself. Instead it relies on the driver in Dom0 Linux to
> > probe NVDIMM and make the reservation.
> 
> I'm missing something because the design document talks about mmap'ing
> files on a DAX filesystem.  So, I'm assuming it is similar to the KVM
> NVDIMM virtualization case where an mmap range in dom0 is translated
> into a guest physical range.  The suggestion is to reserve some memory
> out of that mapping rather than introduce a new info block /
> reservation type to the sub-system.

Just like struct page to linux, Xen hypervisor uses a struct page_info
for its memory management. We are facing the same problem as linux
kernel: where we store those structs for pmem, and decided to put them
on a reserved area on pmem, similar to what pfn device in kernel does.

Reserving at the moment of mmap and out of what is mapped does not
work. It's a bootstrap problem: Xen needs the information of those
pages, which are stored in struct page_info, at the moment of
mapping. That is, page_info structs for pmem pages should be prepared
before those pages are actually used.

However, as the ongoing discussion in another thread with Andrew
Cooper, if Xen hypervisor turns to treat pmem pages as MMIO, then the
reservation may not be needed. Let's see what conclusion will be
reached there.

Thanks,
Haozhong

Thread

[RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-10 02:40 +0200
  Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-10 05:50 +0200
    Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-10 08:40 +0200
      Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-10 18:30 +0200
        Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-11 09:20 +0200
  Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-10 18:50 +0200
    Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-11 08:00 +0200
      Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:50 +0200
      Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 21:00 +0200
      Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 21:00 +0200
    Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support  for Xen "Jan Beulich" <jbeulich@suse.com> - 2016-10-11 15:10 +0200
      Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 18:00 +0200
        Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 19:00 +0200
          Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 20:00 +0200
            Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 20:30 +0200
              Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:50 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 21:50 +0200
            Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:40 +0200
              Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 21:30 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 22:00 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 22:20 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 22:20 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-12 12:40 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support  for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 13:40 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for  Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-12 17:10 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support  for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 17:40 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-12 17:50 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support  for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 18:30 +0200
                Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-12 18:30 +0200
        Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support  for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 09:30 +0200

csiph-web