Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1499750
| From | Haozhong Zhang <haozhong.zhang@intel.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen |
| Date | 2016-10-12 17:10 +0200 |
| Message-ID | <srtkl-688-5@gated-at.bofh.it> (permalink) |
| References | (5 earlier) <sraUp-2hG-1@gated-at.bofh.it> <srbnr-2rH-23@gated-at.bofh.it> <srbGN-2Nu-1@gated-at.bofh.it> <srp73-3aC-9@gated-at.bofh.it> <srq37-3LI-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
On 10/12/16 05:32 -0600, Jan Beulich wrote:
>>>> On 12.10.16 at 12:33, <haozhong.zhang@intel.com> wrote:
>> The layout is shown as the following diagram.
>>
>> +---------------+-----------+-------+----------+--------------+
>> | whatever used | Partition | Super | Reserved | /dev/pmem0p1 |
>> | by kernel | Table | Block | for Xen | |
>> +---------------+-----------+-------+----------+--------------+
>> \_____________________ _______________________/
>> V
>> /dev/pmem0
>
>I have to admit that I dislike this, for not being OS-agnostic.
>Neither should there be any Xen-specific region, nor should the
>"whatever used by kernel" one be restricted to just Linux. What
>I could see is an OS-reserved area ahead of the partition table,
>the exact usage of which depends on which OS is currently
>running (and in the Xen case this might be both Xen _and_ the
>Dom0 kernel, arbitrated by a tbd protocol). After all, when
>running under Xen, the Dom0 may not have a need for as much
>control data as it has when running on bare hardware, for it
>controlling less (if any) of the actual memory ranges when Xen
>is present.
>
Isn't this OS-reserved area still not OS-agnostic, as it requires OS
to know where the reserved area is? Or do you mean it's not if it's
defined by a protocol that is accepted by all OSes?
Let me list another two methods just coming to my mind.
1. The first method extends the usage of the super block used by
current Linux kernel to reserve space on pmem.
Current Linux kernel places a super block of the following
structure near the beginning of a pmem namespace.
struct nd_pfn_sb {
u8 signature[PFN_SIG_LEN];
u8 uuid[16];
u8 parent_uuid[16];
__le32 flags;
__le16 version_major;
__le16 version_minor;
__le64 dataoff; /* relative to namespace_base + start_pad */
__le64 npfns;
__le32 mode;
/* minor-version-1 additions for section alignment */
__le32 start_pad;
__le32 end_trunc;
/* minor-version-2 record the base alignment of the mapping */
__le32 align;
u8 padding[4000];
__le64 checksum;
}
Two interesting fields here are 'dataoff' and 'mode':
- 'dataoff' indicates the offset where the data area starts,
ie. IIUC, the part that can be accessed via /dev/pmemN or
/dev/daxN.
- 'mode' indicates whether Linux puts struct page for this
namespace in the ram (= PFN_MODE_RAM) or on the device (=
PFN_MODE_PMEM).
Currently for Linux, only 'mode' is customizable, while 'dataoff'
is not. If mode == PFN_MODE_RAM, no reservation for struct page is
made on the device, and dataoff starts almost immediately after
the super block except a small reserved area in between for other
structures and alignment. If mode == PFN_MODE_PMEM, the size of
the reservation is decided by kernel, i.e. 64 bytes per struct
page.
I propose to make the size of the reserved area customizable,
e.g. via ioctl and ndctl.
- If mode == PFN_MODE_PMEM and
* if the given reserved size is large enough to hold what an OS
(not limited to Linux) wants to put in, then the OS just
starts use it as desired;
* if the given reserved size is not enough, then the OS reports
error and may take other fallback actions.
- If mode == PFN_MODE_RAM and
* if the reserved size is zero, then it's the current way that
Linux uses the device;
* if the reserved size is non-zero, I would like to reserve this
case for hypervisor (right now, namely Xen hypervisor)
usage. That is, the OS should not use the reserved area. For
Xen, we could add a function in xen driver in kernel to report
the reserved area to hypervisor.
I guess this might be the OS-agnostic way Jan expects, but Dan may
object to.
2. Lay another pseudo device on the block device (e.g. /dev/pmemN)
provided by the NVDIMM driver.
This pseudo device can reserve the size according to user's
requirement. The reservation information can be persistently
recorded in a super block before the reserved area.
This pseudo device also implements another pseudo block device to
allow the non-reserved area be accessed as a block device (we can
even implement it as DAX-capable).
pseudo block device
/---------^-----------\
+------------------+-------+---------------+-----------------------+
| whatever used | Super | reserved by | |
| by NVDIMM driver | Block | pseudo device | |
+------------------+-------+---------------+-----------------------+
\_____________________ _______________________/
V
/dev/pmem0
(provided by NVDIMM driver)
In order to make it work across difference OSes, it requires other
OS recognizes the same types of pmem block devices made by Linux,
and implements the driver for the pseudo device.
This is inspired by Dan's reply at
https://lists.xenproject.org/archives/html/xen-devel/2016-10/msg00651.html.
However, it's essentially the same as my partition solution, so I guess
Jan will still dislike.
Any comments?
>The assumption of course is that the reserved area holds no
>persistent data. If that assumption didn't hold, you'd have to
>have per-OS reserved areas anyway (as many of them as
>there might be OSes [planned to get] installed on a particular
>system).
>
No persistent data should be placed in the reserved area.
Thanks,
Haozhong
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-10 02:40 +0200
Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-10 05:50 +0200
Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-10 08:40 +0200
Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-10 18:30 +0200
Re: [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-11 09:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-10 18:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-11 08:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 21:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 21:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <jbeulich@suse.com> - 2016-10-11 15:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 18:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 19:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 20:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 20:30 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 21:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 20:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 21:30 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-11 22:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-11 22:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-11 22:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-12 12:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 13:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-12 17:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 17:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-12 17:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 18:30 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-12 18:30 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-13 10:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-13 11:00 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-13 11:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-13 11:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-13 17:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-14 12:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-20 11:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-13 17:50 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-13 18:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Dan Williams <dan.j.williams@intel.com> - 2016-10-13 21:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-13 21:40 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-14 09:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-14 14:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Haozhong Zhang <haozhong.zhang@intel.com> - 2016-10-20 11:20 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen Andrew Cooper <andrew.cooper3@citrix.com> - 2016-10-21 00:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-14 12:10 +0200
Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen "Jan Beulich" <JBeulich@suse.com> - 2016-10-12 09:30 +0200
csiph-web