Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1253514 > unrolled thread

[PATCH 0/5] block, dax: updates for 4.4

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/5] block, dax: updates for 4.4 Dan Williams <dan.j.williams@intel.com> - 2015-10-22 08:50 +0200
    [PATCH 2/5] dax: increase granularity of dax_clear_blocks()  operations Dan Williams <dan.j.williams@intel.com> - 2015-10-22 08:50 +0200
      Re: [PATCH 2/5] dax: increase granularity of dax_clear_blocks()  operations Jan Kara <jack@suse.cz> - 2015-10-22 11:30 +0200
    [PATCH 5/5] block: enable dax for raw block devices Dan Williams <dan.j.williams@intel.com> - 2015-10-22 08:50 +0200
      Re: [PATCH 5/5] block: enable dax for raw block devices Jan Kara <jack@suse.cz> - 2015-10-22 11:40 +0200
        Re: [PATCH 5/5] block: enable dax for raw block devices "Williams, Dan J" <dan.j.williams@intel.com> - 2015-10-22 18:10 +0200
          Re: [PATCH 5/5] block: enable dax for raw block devices Jan Kara <jack@suse.cz> - 2015-10-22 23:10 +0200
            Re: [PATCH 5/5] block: enable dax for raw block devices "Williams, Dan J" <dan.j.williams@intel.com> - 2015-10-23 01:50 +0200
              Re: [PATCH 5/5] block: enable dax for raw block devices Jan Kara <jack@suse.cz> - 2015-10-24 16:30 +0200
            Re: [PATCH 5/5] block: enable dax for raw block devices Dan Williams <dan.j.williams@intel.com> - 2015-10-24 01:40 +0200
              Re: [PATCH 5/5] block: enable dax for raw block devices Jan Kara <jack@suse.cz> - 2015-10-24 16:50 +0200
            Re: [PATCH 5/5] block: enable dax for raw block devices Dave Chinner <david@fromorbit.com> - 2015-10-25 22:30 +0100
              Re: [PATCH 5/5] block: enable dax for raw block devices Dan Williams <dan.j.williams@intel.com> - 2015-10-26 03:50 +0100
                Re: [PATCH 5/5] block: enable dax for raw block devices Dave Chinner <david@fromorbit.com> - 2015-10-26 07:30 +0100
                  Re: [PATCH 5/5] block: enable dax for raw block devices Jan Kara <jack@suse.cz> - 2015-10-26 08:30 +0100
                  Re: [PATCH 5/5] block: enable dax for raw block devices Dan Williams <dan.j.williams@intel.com> - 2015-10-26 10:00 +0100
                    Re: [PATCH 5/5] block: enable dax for raw block devices Dave Chinner <david@fromorbit.com> - 2015-10-26 23:30 +0100
                      Re: [PATCH 5/5] block: enable dax for raw block devices Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-10-28 00:00 +0100

#1253514 — [PATCH 0/5] block, dax: updates for 4.4

As requested [1], break out the block specific updates from the dax-gup
series [2], to merge via the block tree.

1/ Enable dax mappings for raw block devices.  This addresses the review
   comments (from Ross and Honza) from the RFC [3].

2/ Introduce dax_map_atomic() to fix races between device teardown and
   new mapping requests.  This depends on commit 2a9067a91825 "block:
   generic request_queue reference counting" in for-4.4/integrity branch
   of the block tree.

3/ Cleanup clear_pmem() and its usage in dax.  This depends on commit
   0f90cc6609c7 "mm, dax: fix DAX deadlocks" that was merged into v4.3-rc6.

These pass the nvdimm unit tests and have passed a 0day-kbuild-robot run.

[1]: https://lists.01.org/pipermail/linux-nvdimm/2015-October/002531.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2015-October/002387.html
[3]: https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html

---

Dan Williams (5):
      pmem, dax: clean up clear_pmem()
      dax: increase granularity of dax_clear_blocks() operations
      block, dax: fix lifetime of in-kernel dax mappings with dax_map_atomic()
      block: introduce file_bd_inode()
      block: enable dax for raw block devices


 arch/x86/include/asm/pmem.h |    7 --
 block/blk.h                 |    2 
 fs/block_dev.c              |   73 ++++++++++++++--
 fs/dax.c                    |  196 +++++++++++++++++++++++++++----------------
 include/linux/blkdev.h      |    2 
 5 files changed, 191 insertions(+), 89 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1253515 — [PATCH 2/5] dax: increase granularity of dax_clear_blocks() operations

dax_clear_blocks is currently performing a cond_resched() after every
PAGE_SIZE memset.  We need not check so frequently, for example md-raid
only calls cond_resched() at stripe granularity.  Also, in preparation
for introducing a dax_map_atomic() operation that temporarily pins a dax
mapping move the call to cond_resched() to the outer loop.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 fs/dax.c |   27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/fs/dax.c b/fs/dax.c
index 5dc33d788d50..f8e543839e5c 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -28,6 +28,7 @@
 #include <linux/sched.h>
 #include <linux/uio.h>
 #include <linux/vmstat.h>
+#include <linux/sizes.h>
 
 int dax_clear_blocks(struct inode *inode, sector_t block, long size)
 {
@@ -38,24 +39,20 @@ int dax_clear_blocks(struct inode *inode, sector_t block, long size)
 	do {
 		void __pmem *addr;
 		unsigned long pfn;
-		long count;
+		long count, sz;
 
-		count = bdev_direct_access(bdev, sector, &addr, &pfn, size);
+		sz = min_t(long, size, SZ_1M);
+		count = bdev_direct_access(bdev, sector, &addr, &pfn, sz);
 		if (count < 0)
 			return count;
-		BUG_ON(size < count);
-		while (count > 0) {
-			unsigned pgsz = PAGE_SIZE - offset_in_page(addr);
-			if (pgsz > count)
-				pgsz = count;
-			clear_pmem(addr, pgsz);
-			addr += pgsz;
-			size -= pgsz;
-			count -= pgsz;
-			BUG_ON(pgsz & 511);
-			sector += pgsz / 512;
-			cond_resched();
-		}
+		if (count < sz)
+			sz = count;
+		clear_pmem(addr, sz);
+		addr += sz;
+		size -= sz;
+		BUG_ON(sz & 511);
+		sector += sz / 512;
+		cond_resched();
 	} while (size);
 
 	wmb_pmem();

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1253639 — Re: [PATCH 2/5] dax: increase granularity of dax_clear_blocks() operations

On Thu 22-10-15 02:41:54, Dan Williams wrote:
> dax_clear_blocks is currently performing a cond_resched() after every
> PAGE_SIZE memset.  We need not check so frequently, for example md-raid
> only calls cond_resched() at stripe granularity.  Also, in preparation
> for introducing a dax_map_atomic() operation that temporarily pins a dax
> mapping move the call to cond_resched() to the outer loop.
> 
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>

The patch looks good to me. You can add:

Reviewed-by: Jan Kara <jack@suse.com>

								Honza
> ---
>  fs/dax.c |   27 ++++++++++++---------------
>  1 file changed, 12 insertions(+), 15 deletions(-)
> 
> diff --git a/fs/dax.c b/fs/dax.c
> index 5dc33d788d50..f8e543839e5c 100644
> --- a/fs/dax.c
> +++ b/fs/dax.c
> @@ -28,6 +28,7 @@
>  #include <linux/sched.h>
>  #include <linux/uio.h>
>  #include <linux/vmstat.h>
> +#include <linux/sizes.h>
>  
>  int dax_clear_blocks(struct inode *inode, sector_t block, long size)
>  {
> @@ -38,24 +39,20 @@ int dax_clear_blocks(struct inode *inode, sector_t block, long size)
>  	do {
>  		void __pmem *addr;
>  		unsigned long pfn;
> -		long count;
> +		long count, sz;
>  
> -		count = bdev_direct_access(bdev, sector, &addr, &pfn, size);
> +		sz = min_t(long, size, SZ_1M);
> +		count = bdev_direct_access(bdev, sector, &addr, &pfn, sz);
>  		if (count < 0)
>  			return count;
> -		BUG_ON(size < count);
> -		while (count > 0) {
> -			unsigned pgsz = PAGE_SIZE - offset_in_page(addr);
> -			if (pgsz > count)
> -				pgsz = count;
> -			clear_pmem(addr, pgsz);
> -			addr += pgsz;
> -			size -= pgsz;
> -			count -= pgsz;
> -			BUG_ON(pgsz & 511);
> -			sector += pgsz / 512;
> -			cond_resched();
> -		}
> +		if (count < sz)
> +			sz = count;
> +		clear_pmem(addr, sz);
> +		addr += sz;
> +		size -= sz;
> +		BUG_ON(sz & 511);
> +		sector += sz / 512;
> +		cond_resched();
>  	} while (size);
>  
>  	wmb_pmem();
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1253516 — [PATCH 5/5] block: enable dax for raw block devices

If an application wants exclusive access to all of the persistent memory
provided by an NVDIMM namespace it can use this raw-block-dax facility
to forgo establishing a filesystem.  This capability is targeted
primarily to hypervisors wanting to provision persistent memory for
guests.

Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 fs/block_dev.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 53 insertions(+), 1 deletion(-)

diff --git a/fs/block_dev.c b/fs/block_dev.c
index 3255dcec96b4..c27cd1a21a13 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -1687,13 +1687,65 @@ static const struct address_space_operations def_blk_aops = {
 	.is_dirty_writeback = buffer_check_dirty_writeback,
 };
 
+#ifdef CONFIG_FS_DAX
+/*
+ * In the raw block case we do not need to contend with truncation nor
+ * unwritten file extents.  Without those concerns there is no need for
+ * additional locking beyond the mmap_sem context that these routines
+ * are already executing under.
+ *
+ * Note, there is no protection if the block device is dynamically
+ * resized (partition grow/shrink) during a fault. A stable block device
+ * size is already not enforced in the blkdev_direct_IO path.
+ *
+ * For DAX, it is the responsibility of the block device driver to
+ * ensure the whole-disk device size is stable while requests are in
+ * flight.
+ *
+ * Finally, these paths do not synchronize against freezing
+ * (sb_start_pagefault(), etc...) since bdev_sops does not support
+ * freezing.
+ */
+static int blkdev_dax_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+	return __dax_fault(vma, vmf, blkdev_get_block, NULL);
+}
+
+static int blkdev_dax_pmd_fault(struct vm_area_struct *vma, unsigned long addr,
+		pmd_t *pmd, unsigned int flags)
+{
+	return __dax_pmd_fault(vma, addr, pmd, flags, blkdev_get_block, NULL);
+}
+
+static const struct vm_operations_struct blkdev_dax_vm_ops = {
+	.page_mkwrite	= blkdev_dax_fault,
+	.fault		= blkdev_dax_fault,
+	.pmd_fault	= blkdev_dax_pmd_fault,
+};
+
+static int blkdev_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct inode *bd_inode = file_bd_inode(file);
+
+	if (!IS_DAX(bd_inode))
+		return generic_file_mmap(file, vma);
+
+	file_accessed(file);
+	vma->vm_ops = &blkdev_dax_vm_ops;
+	vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+	return 0;
+}
+#else
+#define blkdev_mmap generic_file_mmap
+#endif
+
 const struct file_operations def_blk_fops = {
 	.open		= blkdev_open,
 	.release	= blkdev_close,
 	.llseek		= block_llseek,
 	.read_iter	= blkdev_read_iter,
 	.write_iter	= blkdev_write_iter,
-	.mmap		= generic_file_mmap,
+	.mmap		= blkdev_mmap,
 	.fsync		= blkdev_fsync,
 	.unlocked_ioctl	= block_ioctl,
 #ifdef CONFIG_COMPAT

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1253663 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Thu 22-10-15 02:42:11, Dan Williams wrote:
> If an application wants exclusive access to all of the persistent memory
> provided by an NVDIMM namespace it can use this raw-block-dax facility
> to forgo establishing a filesystem.  This capability is targeted
> primarily to hypervisors wanting to provision persistent memory for
> guests.
> 
> Cc: Jan Kara <jack@suse.cz>
> Cc: Jeff Moyer <jmoyer@redhat.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Dave Chinner <david@fromorbit.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
>  fs/block_dev.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 53 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/block_dev.c b/fs/block_dev.c
> index 3255dcec96b4..c27cd1a21a13 100644
> --- a/fs/block_dev.c
> +++ b/fs/block_dev.c
> @@ -1687,13 +1687,65 @@ static const struct address_space_operations def_blk_aops = {
>  	.is_dirty_writeback = buffer_check_dirty_writeback,
>  };
>  
> +#ifdef CONFIG_FS_DAX
> +/*
> + * In the raw block case we do not need to contend with truncation nor
> + * unwritten file extents.  Without those concerns there is no need for
> + * additional locking beyond the mmap_sem context that these routines
> + * are already executing under.
> + *
> + * Note, there is no protection if the block device is dynamically
> + * resized (partition grow/shrink) during a fault. A stable block device
> + * size is already not enforced in the blkdev_direct_IO path.
> + *
> + * For DAX, it is the responsibility of the block device driver to
> + * ensure the whole-disk device size is stable while requests are in
> + * flight.
> + *
> + * Finally, these paths do not synchronize against freezing
> + * (sb_start_pagefault(), etc...) since bdev_sops does not support
> + * freezing.

Well, for devices freezing is handled directly in the block layer code
(blk_stop_queue()) since there's no need to put some metadata structures
into a consistent state. So the comment about bdev_sops is somewhat
strange. Otherwise the patch looks good to me. You can add:

Reviewed-by: Jan Kara <jack@suse.com>

								Honza
 
> + */
> +static int blkdev_dax_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
> +{
> +	return __dax_fault(vma, vmf, blkdev_get_block, NULL);
> +}
> +
> +static int blkdev_dax_pmd_fault(struct vm_area_struct *vma, unsigned long addr,
> +		pmd_t *pmd, unsigned int flags)
> +{
> +	return __dax_pmd_fault(vma, addr, pmd, flags, blkdev_get_block, NULL);
> +}
> +
> +static const struct vm_operations_struct blkdev_dax_vm_ops = {
> +	.page_mkwrite	= blkdev_dax_fault,
> +	.fault		= blkdev_dax_fault,
> +	.pmd_fault	= blkdev_dax_pmd_fault,
> +};
> +
> +static int blkdev_mmap(struct file *file, struct vm_area_struct *vma)
> +{
> +	struct inode *bd_inode = file_bd_inode(file);
> +
> +	if (!IS_DAX(bd_inode))
> +		return generic_file_mmap(file, vma);
> +
> +	file_accessed(file);
> +	vma->vm_ops = &blkdev_dax_vm_ops;
> +	vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
> +	return 0;
> +}
> +#else
> +#define blkdev_mmap generic_file_mmap
> +#endif
> +
>  const struct file_operations def_blk_fops = {
>  	.open		= blkdev_open,
>  	.release	= blkdev_close,
>  	.llseek		= block_llseek,
>  	.read_iter	= blkdev_read_iter,
>  	.write_iter	= blkdev_write_iter,
> -	.mmap		= generic_file_mmap,
> +	.mmap		= blkdev_mmap,
>  	.fsync		= blkdev_fsync,
>  	.unlocked_ioctl	= block_ioctl,
>  #ifdef CONFIG_COMPAT
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1253944 — Re: [PATCH 5/5] block: enable dax for raw block devices

T24gVGh1LCAyMDE1LTEwLTIyIGF0IDExOjM1ICswMjAwLCBKYW4gS2FyYSB3cm90ZToNCj4gT24g
VGh1IDIyLTEwLTE1IDAyOjQyOjExLCBEYW4gV2lsbGlhbXMgd3JvdGU6DQo+ID4gSWYgYW4gYXBw
bGljYXRpb24gd2FudHMgZXhjbHVzaXZlIGFjY2VzcyB0byBhbGwgb2YgdGhlIHBlcnNpc3RlbnQg
bWVtb3J5DQo+ID4gcHJvdmlkZWQgYnkgYW4gTlZESU1NIG5hbWVzcGFjZSBpdCBjYW4gdXNlIHRo
aXMgcmF3LWJsb2NrLWRheCBmYWNpbGl0eQ0KPiA+IHRvIGZvcmdvIGVzdGFibGlzaGluZyBhIGZp
bGVzeXN0ZW0uICBUaGlzIGNhcGFiaWxpdHkgaXMgdGFyZ2V0ZWQNCj4gPiBwcmltYXJpbHkgdG8g
aHlwZXJ2aXNvcnMgd2FudGluZyB0byBwcm92aXNpb24gcGVyc2lzdGVudCBtZW1vcnkgZm9yDQo+
ID4gZ3Vlc3RzLg0KPiA+IA0KPiA+IENjOiBKYW4gS2FyYSA8amFja0BzdXNlLmN6Pg0KPiA+IENj
OiBKZWZmIE1veWVyIDxqbW95ZXJAcmVkaGF0LmNvbT4NCj4gPiBDYzogQ2hyaXN0b3BoIEhlbGx3
aWcgPGhjaEBsc3QuZGU+DQo+ID4gQ2M6IERhdmUgQ2hpbm5lciA8ZGF2aWRAZnJvbW9yYml0LmNv
bT4NCj4gPiBDYzogQW5kcmV3IE1vcnRvbiA8YWtwbUBsaW51eC1mb3VuZGF0aW9uLm9yZz4NCj4g
PiBDYzogUm9zcyBad2lzbGVyIDxyb3NzLnp3aXNsZXJAbGludXguaW50ZWwuY29tPg0KPiA+IFNp
Z25lZC1vZmYtYnk6IERhbiBXaWxsaWFtcyA8ZGFuLmoud2lsbGlhbXNAaW50ZWwuY29tPg0KPiA+
IC0tLQ0KPiA+ICBmcy9ibG9ja19kZXYuYyB8ICAgNTQgKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKystDQo+ID4gIDEgZmlsZSBjaGFuZ2VkLCA1MyBp
bnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pDQo+ID4gDQo+ID4gZGlmZiAtLWdpdCBhL2ZzL2Js
b2NrX2Rldi5jIGIvZnMvYmxvY2tfZGV2LmMNCj4gPiBpbmRleCAzMjU1ZGNlYzk2YjQuLmMyN2Nk
MWEyMWExMyAxMDA2NDQNCj4gPiAtLS0gYS9mcy9ibG9ja19kZXYuYw0KPiA+ICsrKyBiL2ZzL2Js
b2NrX2Rldi5jDQo+ID4gQEAgLTE2ODcsMTMgKzE2ODcsNjUgQEAgc3RhdGljIGNvbnN0IHN0cnVj
dCBhZGRyZXNzX3NwYWNlX29wZXJhdGlvbnMgZGVmX2Jsa19hb3BzID0gew0KPiA+ICAJLmlzX2Rp
cnR5X3dyaXRlYmFjayA9IGJ1ZmZlcl9jaGVja19kaXJ0eV93cml0ZWJhY2ssDQo+ID4gIH07DQo+
ID4gIA0KPiA+ICsjaWZkZWYgQ09ORklHX0ZTX0RBWA0KPiA+ICsvKg0KPiA+ICsgKiBJbiB0aGUg
cmF3IGJsb2NrIGNhc2Ugd2UgZG8gbm90IG5lZWQgdG8gY29udGVuZCB3aXRoIHRydW5jYXRpb24g
bm9yDQo+ID4gKyAqIHVud3JpdHRlbiBmaWxlIGV4dGVudHMuICBXaXRob3V0IHRob3NlIGNvbmNl
cm5zIHRoZXJlIGlzIG5vIG5lZWQgZm9yDQo+ID4gKyAqIGFkZGl0aW9uYWwgbG9ja2luZyBiZXlv
bmQgdGhlIG1tYXBfc2VtIGNvbnRleHQgdGhhdCB0aGVzZSByb3V0aW5lcw0KPiA+ICsgKiBhcmUg
YWxyZWFkeSBleGVjdXRpbmcgdW5kZXIuDQo+ID4gKyAqDQo+ID4gKyAqIE5vdGUsIHRoZXJlIGlz
IG5vIHByb3RlY3Rpb24gaWYgdGhlIGJsb2NrIGRldmljZSBpcyBkeW5hbWljYWxseQ0KPiA+ICsg
KiByZXNpemVkIChwYXJ0aXRpb24gZ3Jvdy9zaHJpbmspIGR1cmluZyBhIGZhdWx0LiBBIHN0YWJs
ZSBibG9jayBkZXZpY2UNCj4gPiArICogc2l6ZSBpcyBhbHJlYWR5IG5vdCBlbmZvcmNlZCBpbiB0
aGUgYmxrZGV2X2RpcmVjdF9JTyBwYXRoLg0KPiA+ICsgKg0KPiA+ICsgKiBGb3IgREFYLCBpdCBp
cyB0aGUgcmVzcG9uc2liaWxpdHkgb2YgdGhlIGJsb2NrIGRldmljZSBkcml2ZXIgdG8NCj4gPiAr
ICogZW5zdXJlIHRoZSB3aG9sZS1kaXNrIGRldmljZSBzaXplIGlzIHN0YWJsZSB3aGlsZSByZXF1
ZXN0cyBhcmUgaW4NCj4gPiArICogZmxpZ2h0Lg0KPiA+ICsgKg0KPiA+ICsgKiBGaW5hbGx5LCB0
aGVzZSBwYXRocyBkbyBub3Qgc3luY2hyb25pemUgYWdhaW5zdCBmcmVlemluZw0KPiA+ICsgKiAo
c2Jfc3RhcnRfcGFnZWZhdWx0KCksIGV0Yy4uLikgc2luY2UgYmRldl9zb3BzIGRvZXMgbm90IHN1
cHBvcnQNCj4gPiArICogZnJlZXppbmcuDQo+IA0KPiBXZWxsLCBmb3IgZGV2aWNlcyBmcmVlemlu
ZyBpcyBoYW5kbGVkIGRpcmVjdGx5IGluIHRoZSBibG9jayBsYXllciBjb2RlDQo+IChibGtfc3Rv
cF9xdWV1ZSgpKSBzaW5jZSB0aGVyZSdzIG5vIG5lZWQgdG8gcHV0IHNvbWUgbWV0YWRhdGEgc3Ry
dWN0dXJlcw0KPiBpbnRvIGEgY29uc2lzdGVudCBzdGF0ZS4gU28gdGhlIGNvbW1lbnQgYWJvdXQg
YmRldl9zb3BzIGlzIHNvbWV3aGF0DQo+IHN0cmFuZ2UuDQoNClRoaXMgdGV4dCB3YXMgYWltZWQg
YXQgdGhlIHJlcXVlc3QgZnJvbSBSb3NzIHRvIGRvY3VtZW50IHRoZSBkaWZmZXJlbmNlcw0KdnMg
dGhlIGdlbmVyaWNfZmlsZV9tbWFwKCkgcGF0aC4gIElzIHRoZSBmb2xsb3dpbmcgaW5jcmVtZW50
YWwgY2hhbmdlDQptb3JlIGNsZWFyPw0KDQpkaWZmIC0tZ2l0IGEvZnMvYmxvY2tfZGV2LmMgYi9m
cy9ibG9ja19kZXYuYw0KaW5kZXggODQwYWNkNDM4MGQ0Li40YWU4ZmE1NWJkMWUgMTAwNjQ0DQot
LS0gYS9mcy9ibG9ja19kZXYuYw0KKysrIGIvZnMvYmxvY2tfZGV2LmMNCkBAIC0xNzAyLDkgKzE3
MDIsMTUgQEAgc3RhdGljIGNvbnN0IHN0cnVjdCBhZGRyZXNzX3NwYWNlX29wZXJhdGlvbnMgZGVm
X2Jsa19hb3BzID0gew0KICAqIGVuc3VyZSB0aGUgd2hvbGUtZGlzayBkZXZpY2Ugc2l6ZSBpcyBz
dGFibGUgd2hpbGUgcmVxdWVzdHMgYXJlIGluDQogICogZmxpZ2h0Lg0KICAqDQotICogRmluYWxs
eSwgdGhlc2UgcGF0aHMgZG8gbm90IHN5bmNocm9uaXplIGFnYWluc3QgZnJlZXppbmcNCi0gKiAo
c2Jfc3RhcnRfcGFnZWZhdWx0KCksIGV0Yy4uLikgc2luY2UgYmRldl9zb3BzIGRvZXMgbm90IHN1
cHBvcnQNCi0gKiBmcmVlemluZy4NCisgKiBGaW5hbGx5LCBpbiBjb250cmFzdCB0byB0aGUgZ2Vu
ZXJpY19maWxlX21tYXAoKSBwYXRoLCB0aGVyZSBhcmUgbm8NCisgKiBjYWxscyB0byBzYl9zdGFy
dF9wYWdlZmF1bHQoKS4gIFRoYXQgaXMgbWVhbnQgdG8gc3luY2hyb25pemUgd3JpdGUNCisgKiBm
YXVsdHMgYWdhaW5zdCByZXF1ZXN0cyB0byBmcmVlemUgdGhlIGNvbnRlbnRzIG9mIHRoZSBmaWxl
c3lzdGVtDQorICogaG9zdGluZyB2bWEtPnZtX2ZpbGUuICBIb3dldmVyLCBpbiB0aGUgY2FzZSBv
ZiBhIGJsb2NrIGRldmljZSBzcGVjaWFsDQorICogZmlsZSwgaXQgaXMgYSAwLXNpemVkIGRldmlj
ZSBub2RlIHVzdWFsbHkgaG9zdGVkIG9uIGRldnRtcGZzLCBpLmUuDQorICogbm90aGluZyB0byBk
byB3aXRoIHRoZSBzdXBlcl9ibG9jayBmb3IgYmRldl9maWxlX2lub2RlKHZtYS0+dm1fZmlsZSku
DQorICogV2UgY291bGQgY2FsbCBnZXRfc3VwZXIoKSBpbiB0aGlzIHBhdGggdG8gcmV0cmlldmUg
dGhlIHJpZ2h0DQorICogc3VwZXJfYmxvY2ssIGJ1dCB0aGUgZ2VuZXJpY19maWxlX21tYXAoKSBw
YXRoIGRvZXMgbm90IGRvIHRoaXMgZm9yDQorICogdGhlIENPTkZJR19GU19EQVg9biBjYXNlLg0K
ICAqLw0KIHN0YXRpYyBpbnQgYmxrZGV2X2RheF9mYXVsdChzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3Qg
KnZtYSwgc3RydWN0IHZtX2ZhdWx0ICp2bWYpDQogew0KDQo=
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1254181 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Thu 22-10-15 16:05:46, Williams, Dan J wrote:
> On Thu, 2015-10-22 at 11:35 +0200, Jan Kara wrote:
> > On Thu 22-10-15 02:42:11, Dan Williams wrote:
> > > If an application wants exclusive access to all of the persistent memory
> > > provided by an NVDIMM namespace it can use this raw-block-dax facility
> > > to forgo establishing a filesystem.  This capability is targeted
> > > primarily to hypervisors wanting to provision persistent memory for
> > > guests.
> > > 
> > > Cc: Jan Kara <jack@suse.cz>
> > > Cc: Jeff Moyer <jmoyer@redhat.com>
> > > Cc: Christoph Hellwig <hch@lst.de>
> > > Cc: Dave Chinner <david@fromorbit.com>
> > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> > > ---
> > >  fs/block_dev.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
> > >  1 file changed, 53 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/fs/block_dev.c b/fs/block_dev.c
> > > index 3255dcec96b4..c27cd1a21a13 100644
> > > --- a/fs/block_dev.c
> > > +++ b/fs/block_dev.c
> > > @@ -1687,13 +1687,65 @@ static const struct address_space_operations def_blk_aops = {
> > >  	.is_dirty_writeback = buffer_check_dirty_writeback,
> > >  };
> > >  
> > > +#ifdef CONFIG_FS_DAX
> > > +/*
> > > + * In the raw block case we do not need to contend with truncation nor
> > > + * unwritten file extents.  Without those concerns there is no need for
> > > + * additional locking beyond the mmap_sem context that these routines
> > > + * are already executing under.
> > > + *
> > > + * Note, there is no protection if the block device is dynamically
> > > + * resized (partition grow/shrink) during a fault. A stable block device
> > > + * size is already not enforced in the blkdev_direct_IO path.
> > > + *
> > > + * For DAX, it is the responsibility of the block device driver to
> > > + * ensure the whole-disk device size is stable while requests are in
> > > + * flight.
> > > + *
> > > + * Finally, these paths do not synchronize against freezing
> > > + * (sb_start_pagefault(), etc...) since bdev_sops does not support
> > > + * freezing.
> > 
> > Well, for devices freezing is handled directly in the block layer code
> > (blk_stop_queue()) since there's no need to put some metadata structures
> > into a consistent state. So the comment about bdev_sops is somewhat
> > strange.
> 
> This text was aimed at the request from Ross to document the differences
> vs the generic_file_mmap() path.  Is the following incremental change
> more clear?

Well, not really. I thought you'd just delete that paragraph :) The thing
is: When doing IO directly to the block device, it makes no sense to look
at a filesystem on top of it - hopefully there is none since you'd be
corrupting it. So the paragraph that would make sense to me would be:

 * Finally, in contrast to filemap_page_mkwrite(), we don't bother calling
 * sb_start_pagefault(). There is no filesystem which could be frozen here
 * and when bdev gets frozen, IO gets blocked in the request queue.

But when spelled out like this, I've realized that with DAX, this blocking
of requests in the request queue doesn't really block the IO to the device.
So block device freezing (aka blk_queue_stop()) doesn't work reliably with
DAX. That should be fixed but it's not easy as the only way to do that
would be to hook into blk_stop_queue() and unmap (or at least
write-protect) all the mappings of the device. Ugh...

Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
filesystems since there's nothing which writeprotects pages that are
writeably mapped. In normal path, page writeback does this but that doesn't
happen for DAX. I remember we once talked about this but it got lost.
We need something like walk all filesystem inodes during fs freeze and
writeprotect all pages that are mapped. But that's going to be slow...

								Honza

> diff --git a/fs/block_dev.c b/fs/block_dev.c
> index 840acd4380d4..4ae8fa55bd1e 100644
> --- a/fs/block_dev.c
> +++ b/fs/block_dev.c
> @@ -1702,9 +1702,15 @@ static const struct address_space_operations def_blk_aops = {
>   * ensure the whole-disk device size is stable while requests are in
>   * flight.
>   *
> - * Finally, these paths do not synchronize against freezing
> - * (sb_start_pagefault(), etc...) since bdev_sops does not support
> - * freezing.
> + * Finally, in contrast to the generic_file_mmap() path, there are no
> + * calls to sb_start_pagefault().  That is meant to synchronize write
> + * faults against requests to freeze the contents of the filesystem
> + * hosting vma->vm_file.  However, in the case of a block device special
> + * file, it is a 0-sized device node usually hosted on devtmpfs, i.e.
> + * nothing to do with the super_block for bdev_file_inode(vma->vm_file).
> + * We could call get_super() in this path to retrieve the right
> + * super_block, but the generic_file_mmap() path does not do this for
> + * the CONFIG_FS_DAX=n case.
>   */
>  static int blkdev_dax_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
>  {
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1254242 — Re: [PATCH 5/5] block: enable dax for raw block devices

T24gVGh1LCAyMDE1LTEwLTIyIGF0IDIzOjA4ICswMjAwLCBKYW4gS2FyYSB3cm90ZToNCj4gT24g
VGh1IDIyLTEwLTE1IDE2OjA1OjQ2LCBXaWxsaWFtcywgRGFuIEogd3JvdGU6DQo+ID4gT24gVGh1
LCAyMDE1LTEwLTIyIGF0IDExOjM1ICswMjAwLCBKYW4gS2FyYSB3cm90ZToNCj4gPiA+IE9uIFRo
dSAyMi0xMC0xNSAwMjo0MjoxMSwgRGFuIFdpbGxpYW1zIHdyb3RlOg0KPiA+ID4gPiBJZiBhbiBh
cHBsaWNhdGlvbiB3YW50cyBleGNsdXNpdmUgYWNjZXNzIHRvIGFsbCBvZiB0aGUgcGVyc2lzdGVu
dCBtZW1vcnkNCj4gPiA+ID4gcHJvdmlkZWQgYnkgYW4gTlZESU1NIG5hbWVzcGFjZSBpdCBjYW4g
dXNlIHRoaXMgcmF3LWJsb2NrLWRheCBmYWNpbGl0eQ0KPiA+ID4gPiB0byBmb3JnbyBlc3RhYmxp
c2hpbmcgYSBmaWxlc3lzdGVtLiAgVGhpcyBjYXBhYmlsaXR5IGlzIHRhcmdldGVkDQo+ID4gPiA+
IHByaW1hcmlseSB0byBoeXBlcnZpc29ycyB3YW50aW5nIHRvIHByb3Zpc2lvbiBwZXJzaXN0ZW50
IG1lbW9yeSBmb3INCj4gPiA+ID4gZ3Vlc3RzLg0KPiA+ID4gPiANCj4gPiA+ID4gQ2M6IEphbiBL
YXJhIDxqYWNrQHN1c2UuY3o+DQo+ID4gPiA+IENjOiBKZWZmIE1veWVyIDxqbW95ZXJAcmVkaGF0
LmNvbT4NCj4gPiA+ID4gQ2M6IENocmlzdG9waCBIZWxsd2lnIDxoY2hAbHN0LmRlPg0KPiA+ID4g
PiBDYzogRGF2ZSBDaGlubmVyIDxkYXZpZEBmcm9tb3JiaXQuY29tPg0KPiA+ID4gPiBDYzogQW5k
cmV3IE1vcnRvbiA8YWtwbUBsaW51eC1mb3VuZGF0aW9uLm9yZz4NCj4gPiA+ID4gQ2M6IFJvc3Mg
Wndpc2xlciA8cm9zcy56d2lzbGVyQGxpbnV4LmludGVsLmNvbT4NCj4gPiA+ID4gU2lnbmVkLW9m
Zi1ieTogRGFuIFdpbGxpYW1zIDxkYW4uai53aWxsaWFtc0BpbnRlbC5jb20+DQo+ID4gPiA+IC0t
LQ0KPiA+ID4gPiAgZnMvYmxvY2tfZGV2LmMgfCAgIDU0ICsrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLQ0KPiA+ID4gPiAgMSBmaWxlIGNoYW5nZWQs
IDUzIGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkNCj4gPiA+ID4gDQo+ID4gPiA+IGRpZmYg
LS1naXQgYS9mcy9ibG9ja19kZXYuYyBiL2ZzL2Jsb2NrX2Rldi5jDQo+ID4gPiA+IGluZGV4IDMy
NTVkY2VjOTZiNC4uYzI3Y2QxYTIxYTEzIDEwMDY0NA0KPiA+ID4gPiAtLS0gYS9mcy9ibG9ja19k
ZXYuYw0KPiA+ID4gPiArKysgYi9mcy9ibG9ja19kZXYuYw0KPiA+ID4gPiBAQCAtMTY4NywxMyAr
MTY4Nyw2NSBAQCBzdGF0aWMgY29uc3Qgc3RydWN0IGFkZHJlc3Nfc3BhY2Vfb3BlcmF0aW9ucyBk
ZWZfYmxrX2FvcHMgPSB7DQo+ID4gPiA+ICAJLmlzX2RpcnR5X3dyaXRlYmFjayA9IGJ1ZmZlcl9j
aGVja19kaXJ0eV93cml0ZWJhY2ssDQo+ID4gPiA+ICB9Ow0KPiA+ID4gPiAgDQo+ID4gPiA+ICsj
aWZkZWYgQ09ORklHX0ZTX0RBWA0KPiA+ID4gPiArLyoNCj4gPiA+ID4gKyAqIEluIHRoZSByYXcg
YmxvY2sgY2FzZSB3ZSBkbyBub3QgbmVlZCB0byBjb250ZW5kIHdpdGggdHJ1bmNhdGlvbiBub3IN
Cj4gPiA+ID4gKyAqIHVud3JpdHRlbiBmaWxlIGV4dGVudHMuICBXaXRob3V0IHRob3NlIGNvbmNl
cm5zIHRoZXJlIGlzIG5vIG5lZWQgZm9yDQo+ID4gPiA+ICsgKiBhZGRpdGlvbmFsIGxvY2tpbmcg
YmV5b25kIHRoZSBtbWFwX3NlbSBjb250ZXh0IHRoYXQgdGhlc2Ugcm91dGluZXMNCj4gPiA+ID4g
KyAqIGFyZSBhbHJlYWR5IGV4ZWN1dGluZyB1bmRlci4NCj4gPiA+ID4gKyAqDQo+ID4gPiA+ICsg
KiBOb3RlLCB0aGVyZSBpcyBubyBwcm90ZWN0aW9uIGlmIHRoZSBibG9jayBkZXZpY2UgaXMgZHlu
YW1pY2FsbHkNCj4gPiA+ID4gKyAqIHJlc2l6ZWQgKHBhcnRpdGlvbiBncm93L3NocmluaykgZHVy
aW5nIGEgZmF1bHQuIEEgc3RhYmxlIGJsb2NrIGRldmljZQ0KPiA+ID4gPiArICogc2l6ZSBpcyBh
bHJlYWR5IG5vdCBlbmZvcmNlZCBpbiB0aGUgYmxrZGV2X2RpcmVjdF9JTyBwYXRoLg0KPiA+ID4g
PiArICoNCj4gPiA+ID4gKyAqIEZvciBEQVgsIGl0IGlzIHRoZSByZXNwb25zaWJpbGl0eSBvZiB0
aGUgYmxvY2sgZGV2aWNlIGRyaXZlciB0bw0KPiA+ID4gPiArICogZW5zdXJlIHRoZSB3aG9sZS1k
aXNrIGRldmljZSBzaXplIGlzIHN0YWJsZSB3aGlsZSByZXF1ZXN0cyBhcmUgaW4NCj4gPiA+ID4g
KyAqIGZsaWdodC4NCj4gPiA+ID4gKyAqDQo+ID4gPiA+ICsgKiBGaW5hbGx5LCB0aGVzZSBwYXRo
cyBkbyBub3Qgc3luY2hyb25pemUgYWdhaW5zdCBmcmVlemluZw0KPiA+ID4gPiArICogKHNiX3N0
YXJ0X3BhZ2VmYXVsdCgpLCBldGMuLi4pIHNpbmNlIGJkZXZfc29wcyBkb2VzIG5vdCBzdXBwb3J0
DQo+ID4gPiA+ICsgKiBmcmVlemluZy4NCj4gPiA+IA0KPiA+ID4gV2VsbCwgZm9yIGRldmljZXMg
ZnJlZXppbmcgaXMgaGFuZGxlZCBkaXJlY3RseSBpbiB0aGUgYmxvY2sgbGF5ZXIgY29kZQ0KPiA+
ID4gKGJsa19zdG9wX3F1ZXVlKCkpIHNpbmNlIHRoZXJlJ3Mgbm8gbmVlZCB0byBwdXQgc29tZSBt
ZXRhZGF0YSBzdHJ1Y3R1cmVzDQo+ID4gPiBpbnRvIGEgY29uc2lzdGVudCBzdGF0ZS4gU28gdGhl
IGNvbW1lbnQgYWJvdXQgYmRldl9zb3BzIGlzIHNvbWV3aGF0DQo+ID4gPiBzdHJhbmdlLg0KPiA+
IA0KPiA+IFRoaXMgdGV4dCB3YXMgYWltZWQgYXQgdGhlIHJlcXVlc3QgZnJvbSBSb3NzIHRvIGRv
Y3VtZW50IHRoZSBkaWZmZXJlbmNlcw0KPiA+IHZzIHRoZSBnZW5lcmljX2ZpbGVfbW1hcCgpIHBh
dGguICBJcyB0aGUgZm9sbG93aW5nIGluY3JlbWVudGFsIGNoYW5nZQ0KPiA+IG1vcmUgY2xlYXI/
DQo+IA0KPiBXZWxsLCBub3QgcmVhbGx5LiBJIHRob3VnaHQgeW91J2QganVzdCBkZWxldGUgdGhh
dCBwYXJhZ3JhcGggOikgVGhlIHRoaW5nDQo+IGlzOiBXaGVuIGRvaW5nIElPIGRpcmVjdGx5IHRv
IHRoZSBibG9jayBkZXZpY2UsIGl0IG1ha2VzIG5vIHNlbnNlIHRvIGxvb2sNCj4gYXQgYSBmaWxl
c3lzdGVtIG9uIHRvcCBvZiBpdCAtIGhvcGVmdWxseSB0aGVyZSBpcyBub25lIHNpbmNlIHlvdSdk
IGJlDQo+IGNvcnJ1cHRpbmcgaXQuIFNvIHRoZSBwYXJhZ3JhcGggdGhhdCB3b3VsZCBtYWtlIHNl
bnNlIHRvIG1lIHdvdWxkIGJlOg0KPiANCj4gICogRmluYWxseSwgaW4gY29udHJhc3QgdG8gZmls
ZW1hcF9wYWdlX21rd3JpdGUoKSwgd2UgZG9uJ3QgYm90aGVyIGNhbGxpbmcNCj4gICogc2Jfc3Rh
cnRfcGFnZWZhdWx0KCkuIFRoZXJlIGlzIG5vIGZpbGVzeXN0ZW0gd2hpY2ggY291bGQgYmUgZnJv
emVuIGhlcmUNCj4gICogYW5kIHdoZW4gYmRldiBnZXRzIGZyb3plbiwgSU8gZ2V0cyBibG9ja2Vk
IGluIHRoZSByZXF1ZXN0IHF1ZXVlLg0KPiANCj4gQnV0IHdoZW4gc3BlbGxlZCBvdXQgbGlrZSB0
aGlzLCBJJ3ZlIHJlYWxpemVkIHRoYXQgd2l0aCBEQVgsIHRoaXMgYmxvY2tpbmcNCj4gb2YgcmVx
dWVzdHMgaW4gdGhlIHJlcXVlc3QgcXVldWUgZG9lc24ndCByZWFsbHkgYmxvY2sgdGhlIElPIHRv
IHRoZSBkZXZpY2UuDQo+IFNvIGJsb2NrIGRldmljZSBmcmVlemluZyAoYWthIGJsa19xdWV1ZV9z
dG9wKCkpIGRvZXNuJ3Qgd29yayByZWxpYWJseSB3aXRoDQo+IERBWC4gVGhhdCBzaG91bGQgYmUg
Zml4ZWQgYnV0IGl0J3Mgbm90IGVhc3kgYXMgdGhlIG9ubHkgd2F5IHRvIGRvIHRoYXQNCj4gd291
bGQgYmUgdG8gaG9vayBpbnRvIGJsa19zdG9wX3F1ZXVlKCkgYW5kIHVubWFwIChvciBhdCBsZWFz
dA0KPiB3cml0ZS1wcm90ZWN0KSBhbGwgdGhlIG1hcHBpbmdzIG9mIHRoZSBkZXZpY2UuIFVnaC4u
Lg0KPiANCj4gVWdoMjogTm93IEkgcmVhbGl6ZWQgdGhhdCBEQVggbW1hcCBpc24ndCBzYWZlIHdy
dCBmcyBmcmVlemluZyBldmVuIGZvcg0KPiBmaWxlc3lzdGVtcyBzaW5jZSB0aGVyZSdzIG5vdGhp
bmcgd2hpY2ggd3JpdGVwcm90ZWN0cyBwYWdlcyB0aGF0IGFyZQ0KPiB3cml0ZWFibHkgbWFwcGVk
LiBJbiBub3JtYWwgcGF0aCwgcGFnZSB3cml0ZWJhY2sgZG9lcyB0aGlzIGJ1dCB0aGF0IGRvZXNu
J3QNCj4gaGFwcGVuIGZvciBEQVguIEkgcmVtZW1iZXIgd2Ugb25jZSB0YWxrZWQgYWJvdXQgdGhp
cyBidXQgaXQgZ290IGxvc3QuDQo+IFdlIG5lZWQgc29tZXRoaW5nIGxpa2Ugd2FsayBhbGwgZmls
ZXN5c3RlbSBpbm9kZXMgZHVyaW5nIGZzIGZyZWV6ZSBhbmQNCj4gd3JpdGVwcm90ZWN0IGFsbCBw
YWdlcyB0aGF0IGFyZSBtYXBwZWQuIEJ1dCB0aGF0J3MgZ29pbmcgdG8gYmUgc2xvdy4uLg0KDQpU
aGlzIHNvdW5kcyBzdXNwaWNpb3VzbHkgbGlrZSB3aGF0IEknbSBwbGFubmluZyB0byBkbyBmb3Ig
dGhlIGRldmljZQ0KdGVhcmRvd24gcGF0aCB3aGVuIHdlJ3ZlIGR5bmFtaWNhbGx5IGFsbG9jYXRl
ZCBzdHJ1Y3QgcGFnZS4gIFRoZSBiYWNraW5nDQptZW1vcnkgZm9yIHRob3NlIHBhZ2VzIGlzIGZy
ZWVkIHdoZW4gdGhlIGRyaXZlciBydW5zIGl0cyAtPnJlbW92ZSgpDQpwYXRoLCBzbyB3ZSBoYXZl
IHRvIGJlIHN1cmUgdGhlcmUgYXJlIG5vIG91dHN0YW5kaW5nIHJlZmVyZW5jZXMgdG8gdGhlbS4N
Cg0KTXkgY3VycmVudCBwcm9wb3NhbCBmb3IgdGhlIHRlYXJkb3duIGNhc2UsIHRoYXQgd2UgbWln
aHQgcmUtcHVycG9zZSBmb3INCnRoaXMgZnJlZXplIGNhc2UsIGlzIGJlbG93LiAgSXQgcmVsaWVz
IG9uIHRoZSBwZXJjcHVfcmVmIGluIHRoZQ0KcmVxdWVzdF9xdWV1ZSB0byBibG9jayBuZXcgZmF1
bHRzIGFuZCB0aGVuIHVzZXMgdHJ1bmNhdGVfcGFnZWNhY2hlKCkgdG8NCnRlYXJkb3duIG1hcHBp
bmdzLiAgSG93ZXZlciwgdGhpcyBhc3N1bWVzIHdlJ3ZlIGluc2VydGVkIHBhZ2VzIGludG8gdGhl
DQphZGRyZXNzX3NwYWNlIHJhZGl4IGF0IGZhdWx0LCB3aGljaCB3ZSBkb24ndCBjdXJyZW50bHkg
ZG8uLi4NCg0KSW4gZ2VuZXJhbCwgYXMgdGhpcyBwYWdlLWJhY2tlZC1wbWVtIHN1cHBvcnQgbGFu
ZHMgdXBzdHJlYW0sIEknbSBvZiB0aGUNCm9waW5pb24gdGhhdCB0aGUgcGFnZS1sZXNzIERBWCBz
dXBwb3J0IGJlIGRlcHJlY2F0ZWQvZGlzYWJsZWQNCnVubGVzcy91bnRpbCBpdCBjYW4gYmUgbWFk
ZSBhcyBmdW5jdGlvbmFsbHkgY2FwYWJsZSBhcyB0aGUgcGFnZS1lbmFibGVkDQpwYXRocy4NCg0K
ODwtLS0tDQpTdWJqZWN0OiBtbSwgcG1lbTogZGV2bV9tZW11bm1hcF9wYWdlcygpLCB0cnVuY2F0
ZSBhbmQgdW5tYXAgWk9ORV9ERVZJQ0UgcGFnZXMNCg0KRnJvbTogRGFuIFdpbGxpYW1zIDxkYW4u
ai53aWxsaWFtc0BpbnRlbC5jb20+DQoNCkJlZm9yZSB3ZSBhbGxvdyBaT05FX0RFVklDRSBwYWdl
cyB0byBiZSBwdXQgaW50byBhY3RpdmUgdXNlIG91dHNpZGUgb2YNCnRoZSBwbWVtIGRyaXZlciwg
d2UgbmVlZCB0byBhcnJhbmdlIGZvciB0aGVtIHRvIGJlIHJlY2xhaW1lZCB3aGVuIHRoZQ0KZHJp
dmVyIGlzIHNodXRkb3duLiAgZGV2bV9tZW11bm1hcF9wYWdlcygpIG11c3Qgd2FpdCBmb3IgYWxs
IHBhZ2VzIHRvDQpyZXR1cm4gdG8gdGhlIGluaXRpYWwgbWFwY291bnQgb2YgMS4gIElmIGEgZ2l2
ZW4gcGFnZSBpcyBtYXBwZWQgYnkgYQ0KcHJvY2VzcyB3ZSB3aWxsIHRydW5jYXRlIGl0IG91dCBv
ZiBpdHMgaW5vZGUgbWFwcGluZyBhbmQgdW5tYXAgaXQgb3V0IG9mDQp0aGUgcHJvY2VzcyB2bWEu
DQoNClRoaXMgdHJ1bmNhdGlvbiBpcyBkb25lIHdoaWxlIHRoZSBkZXZfcGFnZW1hcCByZWZlcmVu
Y2UgY291bnQgaXMgImRlYWQiLA0KcHJldmVudGluZyBuZXcgcmVmZXJlbmNlcyBmcm9tIGJlaW5n
IHRha2VuIHdoaWxlIHRoZSB0cnVuY2F0ZSt1bm1hcCBzY2FuDQppcyBpbiBwcm9ncmVzcy4NCg0K
Q2M6IERhdmUgSGFuc2VuIDxkYXZlQHNyNzEubmV0Pg0KQ2M6IEFuZHJldyBNb3J0b24gPGFrcG1A
bGludXgtZm91bmRhdGlvbi5vcmc+DQpDYzogQ2hyaXN0b3BoIEhlbGx3aWcgPGhjaEBsc3QuZGU+
DQpDYzogUm9zcyBad2lzbGVyIDxyb3NzLnp3aXNsZXJAbGludXguaW50ZWwuY29tPg0KQ2M6IE1h
dHRoZXcgV2lsY294IDx3aWxseUBsaW51eC5pbnRlbC5jb20+DQpDYzogQWxleGFuZGVyIFZpcm8g
PHZpcm9AemVuaXYubGludXgub3JnLnVrPg0KQ2M6IERhdmUgQ2hpbm5lciA8ZGF2aWRAZnJvbW9y
Yml0LmNvbT4NClNpZ25lZC1vZmYtYnk6IERhbiBXaWxsaWFtcyA8ZGFuLmoud2lsbGlhbXNAaW50
ZWwuY29tPg0KLS0tDQogZHJpdmVycy9udmRpbW0vcG1lbS5jIHwgICA0MiArKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKystLS0tLS0NCiBmcy9kYXguYyAgICAgICAgICAgICAgfCAg
ICAyICsrDQogaW5jbHVkZS9saW51eC9tbS5oICAgIHwgICAgNSArKysrKw0KIGtlcm5lbC9tZW1y
ZW1hcC5jICAgICB8ICAgNDggKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrDQogNCBmaWxlcyBjaGFuZ2VkLCA5MSBpbnNlcnRpb25zKCspLCA2IGRlbGV0aW9u
cygtKQ0KDQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9udmRpbW0vcG1lbS5jIGIvZHJpdmVycy9udmRp
bW0vcG1lbS5jDQppbmRleCBmN2FjY2U1OTRmYTAuLjJjOWFlYmJjM2ZlYSAxMDA2NDQNCi0tLSBh
L2RyaXZlcnMvbnZkaW1tL3BtZW0uYw0KKysrIGIvZHJpdmVycy9udmRpbW0vcG1lbS5jDQpAQCAt
MjQsMTIgKzI0LDE1IEBADQogI2luY2x1ZGUgPGxpbnV4L21lbW9yeV9ob3RwbHVnLmg+DQogI2lu
Y2x1ZGUgPGxpbnV4L21vZHVsZXBhcmFtLmg+DQogI2luY2x1ZGUgPGxpbnV4L3ZtYWxsb2MuaD4N
CisjaW5jbHVkZSA8bGludXgvYXN5bmMuaD4NCiAjaW5jbHVkZSA8bGludXgvc2xhYi5oPg0KICNp
bmNsdWRlIDxsaW51eC9wbWVtLmg+DQogI2luY2x1ZGUgPGxpbnV4L25kLmg+DQogI2luY2x1ZGUg
InBmbi5oIg0KICNpbmNsdWRlICJuZC5oIg0KIA0KK3N0YXRpYyBBU1lOQ19ET01BSU5fRVhDTFVT
SVZFKGFzeW5jX3BtZW0pOw0KKw0KIHN0cnVjdCBwbWVtX2RldmljZSB7DQogCXN0cnVjdCByZXF1
ZXN0X3F1ZXVlCSpwbWVtX3F1ZXVlOw0KIAlzdHJ1Y3QgZ2VuZGlzawkJKnBtZW1fZGlzazsNCkBA
IC0xNjQsMTQgKzE2Nyw0MyBAQCBzdGF0aWMgc3RydWN0IHBtZW1fZGV2aWNlICpwbWVtX2FsbG9j
KHN0cnVjdCBkZXZpY2UgKmRldiwNCiAJcmV0dXJuIHBtZW07DQogfQ0KIA0KLXN0YXRpYyB2b2lk
IHBtZW1fZGV0YWNoX2Rpc2soc3RydWN0IHBtZW1fZGV2aWNlICpwbWVtKQ0KKw0KK3N0YXRpYyB2
b2lkIGFzeW5jX2Jsa19jbGVhbnVwX3F1ZXVlKHZvaWQgKmRhdGEsIGFzeW5jX2Nvb2tpZV90IGNv
b2tpZSkNCit7DQorCXN0cnVjdCBwbWVtX2RldmljZSAqcG1lbSA9IGRhdGE7DQorDQorCWJsa19j
bGVhbnVwX3F1ZXVlKHBtZW0tPnBtZW1fcXVldWUpOw0KK30NCisNCitzdGF0aWMgdm9pZCBwbWVt
X2RldGFjaF9kaXNrKHN0cnVjdCBkZXZpY2UgKmRldikNCiB7DQorCXN0cnVjdCBwbWVtX2Rldmlj
ZSAqcG1lbSA9IGRldl9nZXRfZHJ2ZGF0YShkZXYpOw0KKwlzdHJ1Y3QgcmVxdWVzdF9xdWV1ZSAq
cSA9IHBtZW0tPnBtZW1fcXVldWU7DQorDQogCWlmICghcG1lbS0+cG1lbV9kaXNrKQ0KIAkJcmV0
dXJuOw0KIA0KIAlkZWxfZ2VuZGlzayhwbWVtLT5wbWVtX2Rpc2spOw0KIAlwdXRfZGlzayhwbWVt
LT5wbWVtX2Rpc2spOw0KLQlibGtfY2xlYW51cF9xdWV1ZShwbWVtLT5wbWVtX3F1ZXVlKTsNCisJ
YXN5bmNfc2NoZWR1bGVfZG9tYWluKGFzeW5jX2Jsa19jbGVhbnVwX3F1ZXVlLCBwbWVtLCAmYXN5
bmNfcG1lbSk7DQorDQorCWlmIChwbWVtLT5wZm5fZmxhZ3MgJiBQRk5fTUFQKSB7DQorCQkvKg0K
KwkJICogV2FpdCBmb3IgcXVldWUgdG8gZ28gZGVhZCBzbyB0aGF0IHdlIGtub3cgbm8gbmV3DQor
CQkgKiByZWZlcmVuY2VzIHdpbGwgYmUgdGFrZW4gYWdhaW5zdCB0aGUgcGFnZXMgYWxsb2NhdGVk
DQorCQkgKiBieSBkZXZtX21lbXJlbWFwX3BhZ2VzKCkuDQorCQkgKi8NCisJCWJsa193YWl0X3F1
ZXVlX2RlYWQocSk7DQorDQorCQkvKg0KKwkJICogTWFudWFsbHkgcmVsZWFzZSB0aGUgcGFnZSBt
YXBwaW5nIHNvIHRoYXQNCisJCSAqIGJsa19jbGVhbnVwX3F1ZXVlKCkgY2FuIGNvbXBsZXRlIHF1
ZXVlIGRyYWluaW5nLg0KKwkJICovDQorCQlkZXZtX21lbXVubWFwX3BhZ2VzKGRldiwgKHZvaWQg
X19mb3JjZSAqKSBwbWVtLT52aXJ0X2FkZHIpOw0KKwl9DQorDQorCS8qIFdhaXQgZm9yIGJsa19j
bGVhbnVwX3F1ZXVlKCkgdG8gZmluaXNoICovDQorCWFzeW5jX3N5bmNocm9uaXplX2Z1bGxfZG9t
YWluKCZhc3luY19wbWVtKTsNCiB9DQogDQogc3RhdGljIGludCBwbWVtX2F0dGFjaF9kaXNrKHN0
cnVjdCBkZXZpY2UgKmRldiwNCkBAIC0yOTksMTEgKzMzMSw5IEBAIHN0YXRpYyBpbnQgbmRfcGZu
X2luaXQoc3RydWN0IG5kX3BmbiAqbmRfcGZuKQ0KIHN0YXRpYyBpbnQgbnZkaW1tX25hbWVzcGFj
ZV9kZXRhY2hfcGZuKHN0cnVjdCBuZF9uYW1lc3BhY2VfY29tbW9uICpuZG5zKQ0KIHsNCiAJc3Ry
dWN0IG5kX3BmbiAqbmRfcGZuID0gdG9fbmRfcGZuKG5kbnMtPmNsYWltKTsNCi0Jc3RydWN0IHBt
ZW1fZGV2aWNlICpwbWVtOw0KIA0KIAkvKiBmcmVlIHBtZW0gZGlzayAqLw0KLQlwbWVtID0gZGV2
X2dldF9kcnZkYXRhKCZuZF9wZm4tPmRldik7DQotCXBtZW1fZGV0YWNoX2Rpc2socG1lbSk7DQor
CXBtZW1fZGV0YWNoX2Rpc2soJm5kX3Bmbi0+ZGV2KTsNCiANCiAJLyogcmVsZWFzZSBuZF9wZm4g
cmVzb3VyY2VzICovDQogCWtmcmVlKG5kX3Bmbi0+cGZuX3NiKTsNCkBAIC00NDYsNyArNDc2LDcg
QEAgc3RhdGljIGludCBuZF9wbWVtX3JlbW92ZShzdHJ1Y3QgZGV2aWNlICpkZXYpDQogCWVsc2Ug
aWYgKGlzX25kX3BmbihkZXYpKQ0KIAkJbnZkaW1tX25hbWVzcGFjZV9kZXRhY2hfcGZuKHBtZW0t
Pm5kbnMpOw0KIAllbHNlDQotCQlwbWVtX2RldGFjaF9kaXNrKHBtZW0pOw0KKwkJcG1lbV9kZXRh
Y2hfZGlzayhkZXYpOw0KIA0KIAlyZXR1cm4gMDsNCiB9DQpkaWZmIC0tZ2l0IGEvZnMvZGF4LmMg
Yi9mcy9kYXguYw0KaW5kZXggOGQ3NTY1NjJmY2YwLi4wYmM5YjMxNWQxNmYgMTAwNjQ0DQotLS0g
YS9mcy9kYXguYw0KKysrIGIvZnMvZGF4LmMNCkBAIC00Niw2ICs0Niw3IEBAIHN0YXRpYyB2b2lk
IF9fcG1lbSAqX19kYXhfbWFwX2F0b21pYyhzdHJ1Y3QgYmxvY2tfZGV2aWNlICpiZGV2LCBzZWN0
b3JfdCBzZWN0b3IsDQogCQlibGtfcXVldWVfZXhpdChxKTsNCiAJCXJldHVybiAodm9pZCBfX3Bt
ZW0gKikgRVJSX1BUUihyYyk7DQogCX0NCisJcmN1X3JlYWRfbG9jaygpOw0KIAlyZXR1cm4gYWRk
cjsNCiB9DQogDQpAQCAtNjIsNiArNjMsNyBAQCBzdGF0aWMgdm9pZCBkYXhfdW5tYXBfYXRvbWlj
KHN0cnVjdCBibG9ja19kZXZpY2UgKmJkZXYsIHZvaWQgX19wbWVtICphZGRyKQ0KIAlpZiAoSVNf
RVJSKGFkZHIpKQ0KIAkJcmV0dXJuOw0KIAlibGtfcXVldWVfZXhpdChiZGV2LT5iZF9xdWV1ZSk7
DQorCXJjdV9yZWFkX3VubG9jaygpOw0KIH0NCiANCiBpbnQgZGF4X2NsZWFyX2Jsb2NrcyhzdHJ1
Y3QgaW5vZGUgKmlub2RlLCBzZWN0b3JfdCBibG9jaywgbG9uZyBzaXplKQ0KZGlmZiAtLWdpdCBh
L2luY2x1ZGUvbGludXgvbW0uaCBiL2luY2x1ZGUvbGludXgvbW0uaA0KaW5kZXggYTViNTI2N2Vh
ZTViLi4yOTQ1MThkZGY1YmMgMTAwNjQ0DQotLS0gYS9pbmNsdWRlL2xpbnV4L21tLmgNCisrKyBi
L2luY2x1ZGUvbGludXgvbW0uaA0KQEAgLTgwMSw2ICs4MDEsNyBAQCBzdHJ1Y3QgZGV2X3BhZ2Vt
YXAgew0KIA0KICNpZmRlZiBDT05GSUdfWk9ORV9ERVZJQ0UNCiBzdHJ1Y3QgZGV2X3BhZ2VtYXAg
Kl9fZ2V0X2Rldl9wYWdlbWFwKHJlc291cmNlX3NpemVfdCBwaHlzKTsNCit2b2lkIGRldm1fbWVt
dW5tYXBfcGFnZXMoc3RydWN0IGRldmljZSAqZGV2LCB2b2lkICphZGRyKTsNCiB2b2lkICpkZXZt
X21lbXJlbWFwX3BhZ2VzKHN0cnVjdCBkZXZpY2UgKmRldiwgc3RydWN0IHJlc291cmNlICpyZXMs
DQogCQlzdHJ1Y3QgcGVyY3B1X3JlZiAqcmVmLCBzdHJ1Y3Qgdm1lbV9hbHRtYXAgKmFsdG1hcCk7
DQogI2Vsc2UNCkBAIC04MDksNiArODEwLDEwIEBAIHN0YXRpYyBpbmxpbmUgc3RydWN0IGRldl9w
YWdlbWFwICpfX2dldF9kZXZfcGFnZW1hcChyZXNvdXJjZV9zaXplX3QgcGh5cykNCiAJcmV0dXJu
IE5VTEw7DQogfQ0KIA0KK3N0YXRpYyBpbmxpbmUgdm9pZCBkZXZtX21lbXVubWFwX3BhZ2VzKHN0
cnVjdCBkZXZpY2UgKmRldiwgdm9pZCAqYWRkcikNCit7DQorfQ0KKw0KIHN0YXRpYyBpbmxpbmUg
dm9pZCAqZGV2bV9tZW1yZW1hcF9wYWdlcyhzdHJ1Y3QgZGV2aWNlICpkZXYsIHN0cnVjdCByZXNv
dXJjZSAqcmVzLA0KIAkJc3RydWN0IHBlcmNwdV9yZWYgKnJlZiwgc3RydWN0IHZtZW1fYWx0bWFw
ICphbHRtYXApDQogew0KZGlmZiAtLWdpdCBhL2tlcm5lbC9tZW1yZW1hcC5jIGIva2VybmVsL21l
bXJlbWFwLmMNCmluZGV4IDQ2OTgwNzFhMWM0My4uYWM3NDMzNmU2ZDczIDEwMDY0NA0KLS0tIGEv
a2VybmVsL21lbXJlbWFwLmMNCisrKyBiL2tlcm5lbC9tZW1yZW1hcC5jDQpAQCAtMTMsNiArMTMs
NyBAQA0KICNpbmNsdWRlIDxsaW51eC9yY3VsaXN0Lmg+DQogI2luY2x1ZGUgPGxpbnV4L2Rldmlj
ZS5oPg0KICNpbmNsdWRlIDxsaW51eC90eXBlcy5oPg0KKyNpbmNsdWRlIDxsaW51eC9mcy5oPg0K
ICNpbmNsdWRlIDxsaW51eC9pby5oPg0KICNpbmNsdWRlIDxsaW51eC9tbS5oPg0KICNpbmNsdWRl
IDxsaW51eC9tZW1vcnlfaG90cGx1Zy5oPg0KQEAgLTE4NywxMCArMTg4LDM5IEBAIHN0YXRpYyB1
bnNpZ25lZCBsb25nIHBmbl9lbmQoc3RydWN0IGRldl9wYWdlbWFwICpwZ21hcCkNCiANCiBzdGF0
aWMgdm9pZCBkZXZtX21lbXJlbWFwX3BhZ2VzX3JlbGVhc2Uoc3RydWN0IGRldmljZSAqZGV2LCB2
b2lkICpkYXRhKQ0KIHsNCisJdW5zaWduZWQgbG9uZyBwZm47DQogCXN0cnVjdCBwYWdlX21hcCAq
cGFnZV9tYXAgPSBkYXRhOw0KIAlzdHJ1Y3QgcmVzb3VyY2UgKnJlcyA9ICZwYWdlX21hcC0+cmVz
Ow0KKwlzdHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZ19wcmV2ID0gTlVMTDsNCiAJc3RydWN0
IGRldl9wYWdlbWFwICpwZ21hcCA9ICZwYWdlX21hcC0+cGdtYXA7DQogDQorCWlmIChwZXJjcHVf
cmVmX3RyeWdldF9saXZlKHBnbWFwLT5yZWYpKSB7DQorCQlkZXZfV0FSTihkZXYsICIlczogcGFn
ZSBtYXBwaW5nIGlzIHN0aWxsIGxpdmUhXG4iLCBfX2Z1bmNfXyk7DQorCQlwZXJjcHVfcmVmX3B1
dChwZ21hcC0+cmVmKTsNCisJfQ0KKw0KKwkvKiBmbHVzaCBpbi1mbGlnaHQgZGF4X21hcF9hdG9t
aWMoKSBvcGVyYXRpb25zICovDQorCXN5bmNocm9uaXplX3JjdSgpOw0KKw0KKwlmb3JfZWFjaF9k
ZXZpY2VfcGZuKHBmbiwgcGdtYXApIHsNCisJCXN0cnVjdCBwYWdlICpwYWdlID0gcGZuX3RvX3Bh
Z2UocGZuKTsNCisJCXN0cnVjdCBhZGRyZXNzX3NwYWNlICptYXBwaW5nID0gcGFnZS0+bWFwcGlu
ZzsNCisJCXN0cnVjdCBpbm9kZSAqaW5vZGUgPSBtYXBwaW5nID8gbWFwcGluZy0+aG9zdCA6IE5V
TEw7DQorDQorCQlkZXZfV0FSTl9PTkNFKGRldiwgYXRvbWljX3JlYWQoJnBhZ2UtPl9jb3VudCkg
PCAxLA0KKwkJCQkiJXM6IFpPTkVfREVWSUNFIHBhZ2Ugd2FzIGZyZWVkIVxuIiwgX19mdW5jX18p
Ow0KKw0KKwkJaWYgKCFtYXBwaW5nIHx8ICFpbm9kZSB8fCBtYXBwaW5nID09IG1hcHBpbmdfcHJl
dikgew0KKwkJCWRldl9XQVJOX09OQ0UoZGV2LCBhdG9taWNfcmVhZCgmcGFnZS0+X2NvdW50KSA+
IDEsDQorCQkJCQkiJXM6IHVuZXhwZWN0ZWQgZWxldmF0ZWQgcGFnZSBjb3VudCBwZm46ICVseFxu
IiwNCisJCQkJCV9fZnVuY19fLCBwZm4pOw0KKwkJCWNvbnRpbnVlOw0KKwkJfQ0KKw0KKwkJdHJ1
bmNhdGVfcGFnZWNhY2hlKGlub2RlLCAwKTsNCisJCW1hcHBpbmdfcHJldiA9IG1hcHBpbmc7DQor
CX0NCisNCiAJLyogcGFnZXMgYXJlIGRlYWQgYW5kIHVudXNlZCwgdW5kbyB0aGUgYXJjaCBtYXBw
aW5nICovDQogCWFyY2hfcmVtb3ZlX21lbW9yeShyZXMtPnN0YXJ0LCByZXNvdXJjZV9zaXplKHJl
cykpOw0KIAlkZXZfV0FSTl9PTkNFKGRldiwgcGdtYXAtPmFsdG1hcCAmJiBwZ21hcC0+YWx0bWFw
LT5hbGxvYywNCkBAIC0yOTIsNiArMzIyLDI0IEBAIHZvaWQgKmRldm1fbWVtcmVtYXBfcGFnZXMo
c3RydWN0IGRldmljZSAqZGV2LCBzdHJ1Y3QgcmVzb3VyY2UgKnJlcywNCiAJcmV0dXJuIF9fdmEo
cmVzLT5zdGFydCk7DQogfQ0KIEVYUE9SVF9TWU1CT0woZGV2bV9tZW1yZW1hcF9wYWdlcyk7DQor
DQorc3RhdGljIGludCBwYWdlX21hcF9tYXRjaChzdHJ1Y3QgZGV2aWNlICpkZXYsIHZvaWQgKnJl
cywgdm9pZCAqbWF0Y2hfZGF0YSkNCit7DQorCXN0cnVjdCBwYWdlX21hcCAqcGFnZV9tYXAgPSBy
ZXM7DQorCXJlc291cmNlX3NpemVfdCBwaHlzID0gKihyZXNvdXJjZV9zaXplX3QgKikgbWF0Y2hf
ZGF0YTsNCisNCisJcmV0dXJuIHBhZ2VfbWFwLT5yZXMuc3RhcnQgPT0gcGh5czsNCit9DQorDQor
dm9pZCBkZXZtX21lbXVubWFwX3BhZ2VzKHN0cnVjdCBkZXZpY2UgKmRldiwgdm9pZCAqYWRkcikN
Cit7DQorCXJlc291cmNlX3NpemVfdCBzdGFydCA9IF9fcGEoYWRkcik7DQorDQorCWlmIChkZXZy
ZXNfcmVsZWFzZShkZXYsIGRldm1fbWVtcmVtYXBfcGFnZXNfcmVsZWFzZSwgcGFnZV9tYXBfbWF0
Y2gsDQorCQkJCSZzdGFydCkgIT0gMCkNCisJCWRldl9XQVJOKGRldiwgImZhaWxlZCB0byBmaW5k
IHBhZ2UgbWFwIHRvIHJlbGVhc2VcbiIpOw0KK30NCitFWFBPUlRfU1lNQk9MKGRldm1fbWVtdW5t
YXBfcGFnZXMpOw0KICNlbmRpZiAvKiBDT05GSUdfWk9ORV9ERVZJQ0UgKi8NCiANCiAjaWZkZWYg
Q09ORklHX1NQQVJTRU1FTV9WTUVNTUFQDQoNCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255190 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Thu 22-10-15 23:41:27, Williams, Dan J wrote:
> On Thu, 2015-10-22 at 23:08 +0200, Jan Kara wrote:
> > On Thu 22-10-15 16:05:46, Williams, Dan J wrote:
> > > On Thu, 2015-10-22 at 11:35 +0200, Jan Kara wrote:
> > > > On Thu 22-10-15 02:42:11, Dan Williams wrote:
> > > > > If an application wants exclusive access to all of the persistent memory
> > > > > provided by an NVDIMM namespace it can use this raw-block-dax facility
> > > > > to forgo establishing a filesystem.  This capability is targeted
> > > > > primarily to hypervisors wanting to provision persistent memory for
> > > > > guests.
> > > > > 
> > > > > Cc: Jan Kara <jack@suse.cz>
> > > > > Cc: Jeff Moyer <jmoyer@redhat.com>
> > > > > Cc: Christoph Hellwig <hch@lst.de>
> > > > > Cc: Dave Chinner <david@fromorbit.com>
> > > > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > > > Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
> > > > > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> > > > > ---
> > > > >  fs/block_dev.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
> > > > >  1 file changed, 53 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/fs/block_dev.c b/fs/block_dev.c
> > > > > index 3255dcec96b4..c27cd1a21a13 100644
> > > > > --- a/fs/block_dev.c
> > > > > +++ b/fs/block_dev.c
> > > > > @@ -1687,13 +1687,65 @@ static const struct address_space_operations def_blk_aops = {
> > > > >  	.is_dirty_writeback = buffer_check_dirty_writeback,
> > > > >  };
> > > > >  
> > > > > +#ifdef CONFIG_FS_DAX
> > > > > +/*
> > > > > + * In the raw block case we do not need to contend with truncation nor
> > > > > + * unwritten file extents.  Without those concerns there is no need for
> > > > > + * additional locking beyond the mmap_sem context that these routines
> > > > > + * are already executing under.
> > > > > + *
> > > > > + * Note, there is no protection if the block device is dynamically
> > > > > + * resized (partition grow/shrink) during a fault. A stable block device
> > > > > + * size is already not enforced in the blkdev_direct_IO path.
> > > > > + *
> > > > > + * For DAX, it is the responsibility of the block device driver to
> > > > > + * ensure the whole-disk device size is stable while requests are in
> > > > > + * flight.
> > > > > + *
> > > > > + * Finally, these paths do not synchronize against freezing
> > > > > + * (sb_start_pagefault(), etc...) since bdev_sops does not support
> > > > > + * freezing.
> > > > 
> > > > Well, for devices freezing is handled directly in the block layer code
> > > > (blk_stop_queue()) since there's no need to put some metadata structures
> > > > into a consistent state. So the comment about bdev_sops is somewhat
> > > > strange.
> > > 
> > > This text was aimed at the request from Ross to document the differences
> > > vs the generic_file_mmap() path.  Is the following incremental change
> > > more clear?
> > 
> > Well, not really. I thought you'd just delete that paragraph :) The thing
> > is: When doing IO directly to the block device, it makes no sense to look
> > at a filesystem on top of it - hopefully there is none since you'd be
> > corrupting it. So the paragraph that would make sense to me would be:
> > 
> >  * Finally, in contrast to filemap_page_mkwrite(), we don't bother calling
> >  * sb_start_pagefault(). There is no filesystem which could be frozen here
> >  * and when bdev gets frozen, IO gets blocked in the request queue.
> > 
> > But when spelled out like this, I've realized that with DAX, this blocking
> > of requests in the request queue doesn't really block the IO to the device.
> > So block device freezing (aka blk_queue_stop()) doesn't work reliably with
> > DAX. That should be fixed but it's not easy as the only way to do that
> > would be to hook into blk_stop_queue() and unmap (or at least
> > write-protect) all the mappings of the device. Ugh...
> > 
> > Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
> > filesystems since there's nothing which writeprotects pages that are
> > writeably mapped. In normal path, page writeback does this but that doesn't
> > happen for DAX. I remember we once talked about this but it got lost.
> > We need something like walk all filesystem inodes during fs freeze and
> > writeprotect all pages that are mapped. But that's going to be slow...
> 
> This sounds suspiciously like what I'm planning to do for the device
> teardown path when we've dynamically allocated struct page.  The backing
> memory for those pages is freed when the driver runs its ->remove()
> path, so we have to be sure there are no outstanding references to them.
> 
> My current proposal for the teardown case, that we might re-purpose for
> this freeze case, is below.  It relies on the percpu_ref in the
> request_queue to block new faults and then uses truncate_pagecache() to
> teardown mappings.  However, this assumes we've inserted pages into the
> address_space radix at fault, which we don't currently do...

Well, for the freeze case it is enough to call unmap_mapping_range() for
each inode->i_mapping on the frozen filesystem. Struct page or presence in
radix tree isn't needed for that to work. Less intrusive  solution would be
to do what unmap_mapping_range() does but writeprotect all the ptes instead
of invalidating them. But that would require some more coding.

> In general, as this page-backed-pmem support lands upstream, I'm of the
> opinion that the page-less DAX support be deprecated/disabled
> unless/until it can be made as functionally capable as the page-enabled
> paths.

I didn't get to reading those patches yet so I may be behind on what has
been agreed on. So far it seemed to me that we can get most of the
functionality work without struct page so that would be preferable so that
we don't have to allocate those pages, no? For stuff like get_user_pages()
allocating struct page is probably the least painful path so I agree with
struct page there. But that is relatively rare... We can talk about this at
KS.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255056 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Thu, Oct 22, 2015 at 2:08 PM, Jan Kara <jack@suse.cz> wrote:
> On Thu 22-10-15 16:05:46, Williams, Dan J wrote:
[..]
>> This text was aimed at the request from Ross to document the differences
>> vs the generic_file_mmap() path.  Is the following incremental change
>> more clear?
>
> Well, not really. I thought you'd just delete that paragraph :) The thing
> is: When doing IO directly to the block device, it makes no sense to look
> at a filesystem on top of it - hopefully there is none since you'd be
> corrupting it. So the paragraph that would make sense to me would be:
>
>  * Finally, in contrast to filemap_page_mkwrite(), we don't bother calling
>  * sb_start_pagefault(). There is no filesystem which could be frozen here
>  * and when bdev gets frozen, IO gets blocked in the request queue.

I'm not following this assertion that "IO gets blocked in the request
queue" when the device is frozen in the code.  As far as I can see
outside of tracking the freeze depth count the request_queue does not
check if the device is frozen.   freeze_bdev() is moot when no
filesystem is a present.

> But when spelled out like this, I've realized that with DAX, this blocking
> of requests in the request queue doesn't really block the IO to the device.
> So block device freezing (aka blk_queue_stop()) doesn't work reliably with
> DAX. That should be fixed but it's not easy as the only way to do that
> would be to hook into blk_stop_queue() and unmap (or at least
> write-protect) all the mappings of the device. Ugh...

Again I'm missing how this is guaranteed in the non-DAX case.
freeze_bdev() will sync_blockdev(), but it does nothing to prevent
re-dirtying through raw device mmaps while the fs in frozen.  Should
it?  That's at least a separate patch.

> Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
> filesystems since there's nothing which writeprotects pages that are
> writeably mapped. In normal path, page writeback does this but that doesn't
> happen for DAX. I remember we once talked about this but it got lost.
> We need something like walk all filesystem inodes during fs freeze and
> writeprotect all pages that are mapped. But that's going to be slow...

This is what I'm attempting to tackle with the next revision of this series...
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255193 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Fri 23-10-15 16:32:57, Dan Williams wrote:
> On Thu, Oct 22, 2015 at 2:08 PM, Jan Kara <jack@suse.cz> wrote:
> > On Thu 22-10-15 16:05:46, Williams, Dan J wrote:
> [..]
> >> This text was aimed at the request from Ross to document the differences
> >> vs the generic_file_mmap() path.  Is the following incremental change
> >> more clear?
> >
> > Well, not really. I thought you'd just delete that paragraph :) The thing
> > is: When doing IO directly to the block device, it makes no sense to look
> > at a filesystem on top of it - hopefully there is none since you'd be
> > corrupting it. So the paragraph that would make sense to me would be:
> >
> >  * Finally, in contrast to filemap_page_mkwrite(), we don't bother calling
> >  * sb_start_pagefault(). There is no filesystem which could be frozen here
> >  * and when bdev gets frozen, IO gets blocked in the request queue.
> 
> I'm not following this assertion that "IO gets blocked in the request
> queue" when the device is frozen in the code.  As far as I can see
> outside of tracking the freeze depth count the request_queue does not
> check if the device is frozen.   freeze_bdev() is moot when no
> filesystem is a present.

Yes, how e.g. dm freezes devices when it wants to do a snapshot is that it
first calls freeze_bdev() (to stop fs when there is one) and then calls
blk_stop_queue() to block all the IO requests in the request queue. In this
sense freeze_bdev() is somewhat a misnomer since it doesn't make sure no IO
is submitted to the bdev.

> > But when spelled out like this, I've realized that with DAX, this blocking
> > of requests in the request queue doesn't really block the IO to the device.
> > So block device freezing (aka blk_queue_stop()) doesn't work reliably with
> > DAX. That should be fixed but it's not easy as the only way to do that
> > would be to hook into blk_stop_queue() and unmap (or at least
> > write-protect) all the mappings of the device. Ugh...
> 
> Again I'm missing how this is guaranteed in the non-DAX case.
> freeze_bdev() will sync_blockdev(), but it does nothing to prevent
> re-dirtying through raw device mmaps while the fs in frozen.  Should
> it?  That's at least a separate patch.

It doesn't have to - after blk_stop_queue() is called no IO is submitted to
the device and snapshotting happens in the level below bdev page cache so
we don't care about modifications happening there. But with DAX things are
different as we directly map device pages into page cache so we have to
make sure no modifications of page cache happen.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255563 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Thu, Oct 22, 2015 at 11:08:18PM +0200, Jan Kara wrote:
> Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
> filesystems since there's nothing which writeprotects pages that are
> writeably mapped. In normal path, page writeback does this but that doesn't
> happen for DAX. I remember we once talked about this but it got lost.
> We need something like walk all filesystem inodes during fs freeze and
> writeprotect all pages that are mapped. But that's going to be slow...

fsync() has the same problem - we have no record of the pages that
need to be committed and then write protected when fsync() is called
after write()...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255625 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Mon, Oct 26, 2015 at 6:22 AM, Dave Chinner <david@fromorbit.com> wrote:
> On Thu, Oct 22, 2015 at 11:08:18PM +0200, Jan Kara wrote:
>> Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
>> filesystems since there's nothing which writeprotects pages that are
>> writeably mapped. In normal path, page writeback does this but that doesn't
>> happen for DAX. I remember we once talked about this but it got lost.
>> We need something like walk all filesystem inodes during fs freeze and
>> writeprotect all pages that are mapped. But that's going to be slow...
>
> fsync() has the same problem - we have no record of the pages that
> need to be committed and then write protected when fsync() is called
> after write()...

I know Ross is still working on that implementation.  However, I had a
thought on the flight to ksummit that maybe we shouldn't worry about
tracking dirty state on a per-page basis.  For small / frequent
synchronizations an application really should be using the nvml
library [1] to issue cache flushes and pcommit from userspace on a
per-cacheline basis.  That leaves unmodified apps that want to be
correct in the presence of dax mappings.  Two things we can do to
mitigate that case:

1/ Make DAX mappings opt-in with a new MMAP_DAX (page-cache bypass)
flag.  Applications shouldn't silently become incorrect simply because
the fs is mounted with -o dax.  If an app doesn't understand DAX
mappings it should get page-cache semantics.  This also protects apps
that are not expecting DAX semantics on raw block device mappings.

2/ Even if we get a new flag that lets the kernel know the app
understands DAX mappings, we shouldn't leave fsync broken.  Can we
instead get by with a simple / big hammer solution?  I.e.

    on_each_cpu(sync_cache, ...);

...where sync_cache is something like:

    cache_disable();
    wbinvd();
    pcommit();
    cache_enable();

Disruptive, yes, but if an app cares about efficient persistent memory
synchronization fsync is already the wrong api.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255688 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Mon, Oct 26, 2015 at 11:48:06AM +0900, Dan Williams wrote:
> On Mon, Oct 26, 2015 at 6:22 AM, Dave Chinner <david@fromorbit.com> wrote:
> > On Thu, Oct 22, 2015 at 11:08:18PM +0200, Jan Kara wrote:
> >> Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
> >> filesystems since there's nothing which writeprotects pages that are
> >> writeably mapped. In normal path, page writeback does this but that doesn't
> >> happen for DAX. I remember we once talked about this but it got lost.
> >> We need something like walk all filesystem inodes during fs freeze and
> >> writeprotect all pages that are mapped. But that's going to be slow...
> >
> > fsync() has the same problem - we have no record of the pages that
> > need to be committed and then write protected when fsync() is called
> > after write()...
> 
> I know Ross is still working on that implementation.  However, I had a
> thought on the flight to ksummit that maybe we shouldn't worry about
> tracking dirty state on a per-page basis.  For small / frequent
> synchronizations an application really should be using the nvml
> library [1] to issue cache flushes and pcommit from userspace on a
> per-cacheline basis.  That leaves unmodified apps that want to be
> correct in the presence of dax mappings.  Two things we can do to
> mitigate that case:
> 
> 1/ Make DAX mappings opt-in with a new MMAP_DAX (page-cache bypass)
> flag.  Applications shouldn't silently become incorrect simply because
> the fs is mounted with -o dax.  If an app doesn't understand DAX
> mappings it should get page-cache semantics.  This also protects apps
> that are not expecting DAX semantics on raw block device mappings.

Which is the complete opposite of what we are trying to acehive with
DAX. i.e. that existing applications "just work" with DAX without
modification. So this is a non-starter.

Also, DAX access isn't a property of mmap - it's a property
of the inode. We cannot do DAX access via mmap while mixing page
cache based access through file descriptor based interfaces. This
I why I'm adding an inode attribute (on disk) to enable per-file DAX
capabilities - either everything is via the DAX paths, or nothing
is.

> 2/ Even if we get a new flag that lets the kernel know the app
> understands DAX mappings, we shouldn't leave fsync broken.  Can we
> instead get by with a simple / big hammer solution?  I.e.

Because we don't physically have to write back data the problem is
both simpler and more complex. The simplest solution is for the
underlying block device to implement blkdev_issue_flush() correctly.

i.e. if blkdev_issue_flush() behaves according to it's required
semantics - that all volatile cached data is flushed to stable
storage - then fsync-on-DAX will work appropriately. As it is, this is
needed for journal based filesystems to work correctly, as they are
assuming that their journal writes are being treated correctly as
REQ_FLUSH | REQ_FUA to ensure correct data/metadata/journal
ordering is maintained....

So, to begin with, this problem needs to be solved at the block
device level. That's the simple, brute-force, big hammer solution to
the problem, and it requires no changes at the filesystem level at
all.

However, to avoid having to flush the entire block device range on
fsync we need a much more complex solution that tracks the dirty
ranges of the file and hence what needs committing when fsync is
run....

> Disruptive, yes, but if an app cares about efficient persistent memory
> synchronization fsync is already the wrong api.

I don't really care about efficiency right now - correctness comes
first. Fundamentally, the app should not care whether it is writing to
persistent memory or spinning rust - the filesystem needs to
provide the application with exactly the same integrity guarantees
regardless of the underlying storage.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255709 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Mon 26-10-15 17:23:19, Dave Chinner wrote:
> On Mon, Oct 26, 2015 at 11:48:06AM +0900, Dan Williams wrote:
> > 2/ Even if we get a new flag that lets the kernel know the app
> > understands DAX mappings, we shouldn't leave fsync broken.  Can we
> > instead get by with a simple / big hammer solution?  I.e.
> 
> Because we don't physically have to write back data the problem is
> both simpler and more complex. The simplest solution is for the
> underlying block device to implement blkdev_issue_flush() correctly.
> 
> i.e. if blkdev_issue_flush() behaves according to it's required
> semantics - that all volatile cached data is flushed to stable
> storage - then fsync-on-DAX will work appropriately. As it is, this is
> needed for journal based filesystems to work correctly, as they are
> assuming that their journal writes are being treated correctly as
> REQ_FLUSH | REQ_FUA to ensure correct data/metadata/journal
> ordering is maintained....
> 
> So, to begin with, this problem needs to be solved at the block
> device level. That's the simple, brute-force, big hammer solution to
> the problem, and it requires no changes at the filesystem level at
> all.

Completely agreed. Just make sure REQ_FLUSH, REQ_FUA works correctly for
pmem and fsync(2) / sync(2) issues go away. Fs freezing stuff is a
different story, that will likely need some coordination from the
filesystem layer (although with some luck we could keep it hidden in
fs/super.c and fs/block_dev.c). I can have a look at that once ext4 dax
support works unless someone beats me to it...

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1255756 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Mon, Oct 26, 2015 at 3:23 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Mon, Oct 26, 2015 at 11:48:06AM +0900, Dan Williams wrote:
>> On Mon, Oct 26, 2015 at 6:22 AM, Dave Chinner <david@fromorbit.com> wrote:
>> > On Thu, Oct 22, 2015 at 11:08:18PM +0200, Jan Kara wrote:
>> >> Ugh2: Now I realized that DAX mmap isn't safe wrt fs freezing even for
>> >> filesystems since there's nothing which writeprotects pages that are
>> >> writeably mapped. In normal path, page writeback does this but that doesn't
>> >> happen for DAX. I remember we once talked about this but it got lost.
>> >> We need something like walk all filesystem inodes during fs freeze and
>> >> writeprotect all pages that are mapped. But that's going to be slow...
>> >
>> > fsync() has the same problem - we have no record of the pages that
>> > need to be committed and then write protected when fsync() is called
>> > after write()...
>>
>> I know Ross is still working on that implementation.  However, I had a
>> thought on the flight to ksummit that maybe we shouldn't worry about
>> tracking dirty state on a per-page basis.  For small / frequent
>> synchronizations an application really should be using the nvml
>> library [1] to issue cache flushes and pcommit from userspace on a
>> per-cacheline basis.  That leaves unmodified apps that want to be
>> correct in the presence of dax mappings.  Two things we can do to
>> mitigate that case:
>>
>> 1/ Make DAX mappings opt-in with a new MMAP_DAX (page-cache bypass)
>> flag.  Applications shouldn't silently become incorrect simply because
>> the fs is mounted with -o dax.  If an app doesn't understand DAX
>> mappings it should get page-cache semantics.  This also protects apps
>> that are not expecting DAX semantics on raw block device mappings.
>
> Which is the complete opposite of what we are trying to acehive with
> DAX. i.e. that existing applications "just work" with DAX without
> modification. So this is a non-starter.

The list of things DAX breaks is getting shorter, but certainly the
page-less paths will not be without surprises for quite a while yet...

> Also, DAX access isn't a property of mmap - it's a property
> of the inode. We cannot do DAX access via mmap while mixing page
> cache based access through file descriptor based interfaces. This
> I why I'm adding an inode attribute (on disk) to enable per-file DAX
> capabilities - either everything is via the DAX paths, or nothing
> is.
>

Per-inode control sounds very useful, I'll look at a similar mechanism
for the raw block case.

However, still not quite convinced page-cache control is an inode-only
property, especially when direct-i/o is not an inode-property.  That
said, I agree the complexity of handling mixed mappings of the same
file is prohibitive.

>> 2/ Even if we get a new flag that lets the kernel know the app
>> understands DAX mappings, we shouldn't leave fsync broken.  Can we
>> instead get by with a simple / big hammer solution?  I.e.
>
> Because we don't physically have to write back data the problem is
> both simpler and more complex. The simplest solution is for the
> underlying block device to implement blkdev_issue_flush() correctly.
>
> i.e. if blkdev_issue_flush() behaves according to it's required
> semantics - that all volatile cached data is flushed to stable
> storage - then fsync-on-DAX will work appropriately. As it is, this is
> needed for journal based filesystems to work correctly, as they are
> assuming that their journal writes are being treated correctly as
> REQ_FLUSH | REQ_FUA to ensure correct data/metadata/journal
> ordering is maintained....
>
> So, to begin with, this problem needs to be solved at the block
> device level. That's the simple, brute-force, big hammer solution to
> the problem, and it requires no changes at the filesystem level at
> all.
>
> However, to avoid having to flush the entire block device range on
> fsync we need a much more complex solution that tracks the dirty
> ranges of the file and hence what needs committing when fsync is
> run....
>
>> Disruptive, yes, but if an app cares about efficient persistent memory
>> synchronization fsync is already the wrong api.
>
> I don't really care about efficiency right now - correctness comes
> first. Fundamentally, the app should not care whether it is writing to
> persistent memory or spinning rust - the filesystem needs to
> provide the application with exactly the same integrity guarantees
> regardless of the underlying storage.
>

Sounds good, get blkdev_issue_flush() functional first and then worry
about building a more efficient solution on top.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1256327 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Mon, Oct 26, 2015 at 05:56:30PM +0900, Dan Williams wrote:
> On Mon, Oct 26, 2015 at 3:23 PM, Dave Chinner <david@fromorbit.com> wrote:
> > Also, DAX access isn't a property of mmap - it's a property
> > of the inode. We cannot do DAX access via mmap while mixing page
> > cache based access through file descriptor based interfaces. This
> > I why I'm adding an inode attribute (on disk) to enable per-file DAX
> > capabilities - either everything is via the DAX paths, or nothing
> > is.
> >
> 
> Per-inode control sounds very useful, I'll look at a similar mechanism
> for the raw block case.
> 
> However, still not quite convinced page-cache control is an inode-only
> property, especially when direct-i/o is not an inode-property.  That
> said, I agree the complexity of handling mixed mappings of the same
> file is prohibitive.

We didn't get that choice with direct IO - support via O_DIRECT was
kinda inherited from other OS's(*). We still have all sorts of
coherency problems between buffered/mmap/direct IO on the same file,
and I'd really, really like to avoid making that same mistake again
with DAX.

i.e. We have a choice with DAX right now that will allow us to avoid
coherency problems that we know existi and can't solve right now.
Making DAX and inode property rather than a application context
property avoids those coherence problems as all access will play by
the same rules....

(*)That said, some other OS's did O_DIRECT as an inode property (e.g.
solaris) where O_DIRECT was only done if no other cached operations
were required (e.g. mmap), and so the fd would transparently shift
between buffered and O_DIRECT depending on external accesses to the
inode. This was not liked because of it's unpredictable effect on
CPU usage and IO latency....

> Sounds good, get blkdev_issue_flush() functional first and then worry
> about building a more efficient solution on top.

*nod*

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257454 — Re: [PATCH 5/5] block: enable dax for raw block devices

On Tue, Oct 27, 2015 at 09:19:30AM +1100, Dave Chinner wrote:
> On Mon, Oct 26, 2015 at 05:56:30PM +0900, Dan Williams wrote:
> > On Mon, Oct 26, 2015 at 3:23 PM, Dave Chinner <david@fromorbit.com> wrote:
> > > Also, DAX access isn't a property of mmap - it's a property
> > > of the inode. We cannot do DAX access via mmap while mixing page
> > > cache based access through file descriptor based interfaces. This
> > > I why I'm adding an inode attribute (on disk) to enable per-file DAX
> > > capabilities - either everything is via the DAX paths, or nothing
> > > is.
> > >
> > 
> > Per-inode control sounds very useful, I'll look at a similar mechanism
> > for the raw block case.
> > 
> > However, still not quite convinced page-cache control is an inode-only
> > property, especially when direct-i/o is not an inode-property.  That
> > said, I agree the complexity of handling mixed mappings of the same
> > file is prohibitive.
> 
> We didn't get that choice with direct IO - support via O_DIRECT was
> kinda inherited from other OS's(*). We still have all sorts of
> coherency problems between buffered/mmap/direct IO on the same file,
> and I'd really, really like to avoid making that same mistake again
> with DAX.
> 
> i.e. We have a choice with DAX right now that will allow us to avoid
> coherency problems that we know existi and can't solve right now.
> Making DAX and inode property rather than a application context
> property avoids those coherence problems as all access will play by
> the same rules....
> 
> (*)That said, some other OS's did O_DIRECT as an inode property (e.g.
> solaris) where O_DIRECT was only done if no other cached operations
> were required (e.g. mmap), and so the fd would transparently shift
> between buffered and O_DIRECT depending on external accesses to the
> inode. This was not liked because of it's unpredictable effect on
> CPU usage and IO latency....
> 
> > Sounds good, get blkdev_issue_flush() functional first and then worry
> > about building a more efficient solution on top.
> 
> *nod*

Okay, I'll get this sent out this week.  I've been working furiously on the
fsync/msync solution which tracks dirty pages via the radix tree - I guess
I'll send out an RFC version of those patches tomorrow so that we can begin
the review process and any glaring issues can be addressed soon. 

That set has grown rather large, though, and I do worry that making it into
v4.4 would be a stretch, although I guess I'm still holding out hope.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web