Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1649970 > unrolled thread
| Started by | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| First post | 2017-05-24 23:50 +0200 |
| Last post | 2017-05-24 23:50 +0200 |
| Articles | 8 — 1 participant |
Back to article view | Back to linux.kernel
[RFC PATCH 00/16] Unmappable memory in SGLs for p2p transfers Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 04/16] um: add dummy ioremap and iounmap functions Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 15/16] dma-mapping: introduce and use unmappable safe sg_virt call Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 02/16] staging: ccree: Cleanup: remove references to page_link Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 09/16] bvec: introduce bvec_page and bvec_set_page accessors Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 16/16] nvmet: use unmappable sgl in rdma target Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 11/16] bvec: convert to using pfn_t internally Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
[RFC PATCH 03/16] kfifo: Cleanup example to not use page_link Logan Gunthorpe <logang@deltatee.com> - 2017-05-24 23:50 +0200
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 00/16] Unmappable memory in SGLs for p2p transfers |
| Message-ID | <tKMkh-7HS-3@gated-at.bofh.it> |
Hi, This RFC patchset continues my work attempting to enforce iomem safety within scatterlists. This takes a bit of a different tack from my last series [1] which tried to introduce a common scatterlist mapping function. Instead, this series takes the approach of marking SGLs that may contain unmappable memory and fencing off only the marked instances from calling sg_page and sg_virt, etc. This means we'd no longer need to clean up every SGL user in the kernel and try to deal with cases that assume every SGL always contains mappable memory with no error path. Patches 1 through 5 are cleanup/prep patches that I'll likely be submitting to their respective maintainers in short order. Patch 6 converts SGLs to use pfn_t instead of the existing page_link as suggested (and seemingly planned for) by Dan. Patches 7 and 8 then add unmappable or io memory support to SGLs Patches 9 through 14, similarly convert the bvec layer to use pfn_t Patches 15 and 16 test the above work by applying it to the nvme-fabrics code such that unmappable SGLs are being used without any BUG_ONS in somewhat limitted testing cases. Seeing this work is still incomplete and experimental, I'm looking to get feedback on people's opinions of whether this is an acceptable approach. If it is favourable, would people be open to seeing cleaned up versions of patches 6, and 9 through 14 be submitted upstream? (ie. just the parts for converting to pfn_t and continuing the unmappable/iomem/p2pmem work out-of-tree for a time). This work also opens up the possibility of having p2pmem not use ZONE_DEVICE struct pages and instead just sticking with pfn_ts with a specific p2p radix tree for looking up the backing device. Presently, I'm ambivalent towards this and would like to hear other's opinions. This series is based on v4.12-rc2 and a git tree is available here: https://github.com/sbates130272/linux-p2pmem.git io_pfn_t Thanks for your time, Logan [1] https://lkml.org/lkml/2017/4/25/738 Logan Gunthorpe (16): dmaengine: ste_dma40, imx-dma: Cleanup scatterlist layering violations staging: ccree: Cleanup: remove references to page_link kfifo: Cleanup example to not use page_link um: add dummy ioremap and iounmap functions tile: provide default ioremap declaration scatterlist: convert page_link to pfn_t scatterlist: support unmappable memory in the scatterlist scatterlist: add iomem support to sg_miter and sg_copy_* bvec: introduce bvec_page and bvec_set_page accessors bvec: massive conversion of all bv_page users bvec: convert to using pfn_t internally bvec: use sg_set_pfn when mapping a bio to an sgl block: bio: introduce bio_add_pfn block: bio: go straight from pfn_t to phys instead of through page dma-mapping: introduce and use unmappable safe sg_virt call nvmet: use unmappable sgl in rdma target arch/powerpc/sysdev/axonram.c | 2 +- arch/tile/mm/pgtable.c | 13 ++ arch/um/include/asm/io.h | 17 ++ block/bio-integrity.c | 8 +- block/bio.c | 58 +++---- block/blk-core.c | 2 +- block/blk-integrity.c | 6 +- block/blk-lib.c | 2 +- block/blk-merge.c | 10 +- block/blk-zoned.c | 6 +- block/bounce.c | 27 +-- drivers/block/aoe/aoecmd.c | 4 +- drivers/block/brd.c | 3 +- drivers/block/drbd/drbd_bitmap.c | 6 +- drivers/block/drbd/drbd_main.c | 4 +- drivers/block/drbd/drbd_receiver.c | 4 +- drivers/block/drbd/drbd_worker.c | 2 +- drivers/block/floppy.c | 4 +- drivers/block/loop.c | 12 +- drivers/block/ps3disk.c | 2 +- drivers/block/ps3vram.c | 2 +- drivers/block/rbd.c | 2 +- drivers/block/rsxx/dma.c | 3 +- drivers/block/umem.c | 2 +- drivers/block/zram/zram_drv.c | 14 +- drivers/dma/imx-dma.c | 7 +- drivers/dma/ste_dma40.c | 5 +- drivers/lightnvm/pblk-core.c | 2 +- drivers/lightnvm/pblk-read.c | 6 +- drivers/md/bcache/btree.c | 2 +- drivers/md/bcache/debug.c | 4 +- drivers/md/bcache/request.c | 4 +- drivers/md/bcache/super.c | 10 +- drivers/md/bcache/util.c | 6 +- drivers/md/dm-crypt.c | 16 +- drivers/md/dm-integrity.c | 18 +- drivers/md/dm-io.c | 2 +- drivers/md/dm-log-writes.c | 12 +- drivers/md/dm-verity-target.c | 4 +- drivers/md/raid5.c | 10 +- drivers/memstick/core/mspro_block.c | 2 +- drivers/nvdimm/blk.c | 4 +- drivers/nvdimm/btt.c | 5 +- drivers/nvdimm/pmem.c | 2 +- drivers/nvme/host/core.c | 2 +- drivers/nvme/host/nvme.h | 2 +- drivers/nvme/host/pci.c | 5 +- drivers/nvme/target/Kconfig | 12 ++ drivers/nvme/target/io-cmd.c | 2 +- drivers/nvme/target/rdma.c | 29 +++- drivers/s390/block/dasd_diag.c | 2 +- drivers/s390/block/dasd_eckd.c | 14 +- drivers/s390/block/dasd_fba.c | 6 +- drivers/s390/block/dcssblk.c | 2 +- drivers/s390/block/scm_blk.c | 2 +- drivers/s390/block/scm_blk_cluster.c | 2 +- drivers/s390/block/xpram.c | 2 +- drivers/scsi/mpt3sas/mpt3sas_transport.c | 6 +- drivers/scsi/sd.c | 16 +- drivers/scsi/sd_dif.c | 4 +- drivers/staging/ccree/ssi_buffer_mgr.c | 17 +- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 +- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 10 +- drivers/staging/lustre/lnet/lnet/lib-move.c | 4 +- drivers/staging/lustre/lnet/lnet/router.c | 6 +- drivers/staging/lustre/lnet/selftest/brw_test.c | 4 +- drivers/staging/lustre/lnet/selftest/conrpc.c | 10 +- drivers/staging/lustre/lnet/selftest/framework.c | 2 +- drivers/staging/lustre/lnet/selftest/rpc.c | 4 +- drivers/staging/lustre/lustre/include/lustre_net.h | 2 +- drivers/staging/lustre/lustre/osc/osc_page.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 6 +- drivers/staging/lustre/lustre/ptlrpc/sec_plain.c | 4 +- drivers/target/target_core_file.c | 4 +- drivers/xen/biomerge.c | 4 +- fs/9p/vfs_addr.c | 6 +- fs/afs/rxrpc.c | 4 +- fs/block_dev.c | 8 +- fs/btrfs/check-integrity.c | 4 +- fs/btrfs/compression.c | 14 +- fs/btrfs/disk-io.c | 4 +- fs/btrfs/extent_io.c | 8 +- fs/btrfs/file-item.c | 8 +- fs/btrfs/inode.c | 14 +- fs/btrfs/raid56.c | 4 +- fs/buffer.c | 2 +- fs/cifs/connect.c | 3 +- fs/cifs/file.c | 6 +- fs/cifs/misc.c | 2 +- fs/cifs/smb2ops.c | 2 +- fs/cifs/transport.c | 3 +- fs/crypto/bio.c | 2 +- fs/direct-io.c | 2 +- fs/exofs/ore.c | 4 +- fs/exofs/ore_raid.c | 2 +- fs/ext4/page-io.c | 2 +- fs/ext4/readpage.c | 2 +- fs/f2fs/data.c | 12 +- fs/gfs2/lops.c | 4 +- fs/gfs2/meta_io.c | 2 +- fs/iomap.c | 2 +- fs/mpage.c | 2 +- fs/orangefs/inode.c | 3 +- fs/splice.c | 2 +- fs/xfs/xfs_aops.c | 2 +- include/linux/bio.h | 34 +++- include/linux/bvec.h | 21 ++- include/linux/dma-mapping.h | 9 +- include/linux/pfn_t.h | 42 ++++- include/linux/scatterlist.h | 189 +++++++++++++++------ kernel/power/swap.c | 2 +- lib/Kconfig | 11 ++ lib/iov_iter.c | 24 +-- lib/scatterlist.c | 65 ++++++- mm/page_io.c | 8 +- net/ceph/messenger.c | 6 +- samples/kfifo/dma-example.c | 8 +- 118 files changed, 690 insertions(+), 393 deletions(-) create mode 100644 arch/um/include/asm/io.h -- 2.1.4
[toc] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 04/16] um: add dummy ioremap and iounmap functions |
| Message-ID | <tKMkj-7HS-31@gated-at.bofh.it> |
| In reply to | #1649970 |
The user mode architecture does not provide ioremap or iounmap, and
because of this, the arch won't build when the functions are used in some
core libraries.
I have designs to use these functions in scatterlist.c where they'd
almost certainly never be called on the um architecture but it does need
to compile.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
arch/um/include/asm/io.h | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
create mode 100644 arch/um/include/asm/io.h
diff --git a/arch/um/include/asm/io.h b/arch/um/include/asm/io.h
new file mode 100644
index 0000000..8f35d57
--- /dev/null
+++ b/arch/um/include/asm/io.h
@@ -0,0 +1,17 @@
+#ifndef _ASM_UM_IO_H
+#define _ASM_UM_IO_H
+
+#define ioremap ioremap
+static inline void __iomem *ioremap(phys_addr_t offset, size_t size)
+{
+ return (void __iomem *)(unsigned long)offset;
+}
+
+#define iounmap iounmap
+static inline void iounmap(void __iomem *addr)
+{
+}
+
+#include <asm-generic/io.h>
+
+#endif
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 15/16] dma-mapping: introduce and use unmappable safe sg_virt call |
| Message-ID | <tKMkj-7HS-33@gated-at.bofh.it> |
| In reply to | #1649970 |
Introduce sg_try_virt which is safe to call with a potentially
unmappable sgl. sg_try_virt returns NULL in cases that the sgl doesn't
have an accessible virtual address to return.
Then, in dma_map_sg_attrs, we use the new function instead of sg_virt
when marking memory as initialized.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
include/linux/dma-mapping.h | 9 +++++++--
include/linux/scatterlist.h | 16 ++++++++++++++++
2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..5ef1ab5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -241,8 +241,13 @@ static inline int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg,
int i, ents;
struct scatterlist *s;
- for_each_sg(sg, s, nents, i)
- kmemcheck_mark_initialized(sg_virt(s), s->length);
+ for_each_sg(sg, s, nents, i) {
+ void *addr = sg_try_virt(sg);
+
+ if (addr)
+ kmemcheck_mark_initialized(addr, s->length);
+ }
+
BUG_ON(!valid_dma_direction(dir));
ents = ops->map_sg(dev, sg, nents, dir, attrs);
BUG_ON(ents < 0);
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2c54c6c..1e11af9 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -298,6 +298,22 @@ static inline void *sg_virt(struct scatterlist *sg)
return page_address(sg_page(sg)) + sg->offset;
}
+/**
+ * sg_try_virt - Return virtual address of an sg entry or NULL if it is
+ * unmappable
+ * @sg: SG entry
+ *
+ * Description:
+ * This is the same as sg_virt but is safe to call with unmappable
+ * memory. This function will return NULL in case the sg is is
+ * unmappable.
+ *
+ **/
+static inline void *sg_try_virt(struct scatterlist *sg)
+{
+ return pfn_t_to_virt(sg->__pfn);
+}
+
int sg_nents(struct scatterlist *sg);
int sg_nents_for_len(struct scatterlist *sg, u64 len);
struct scatterlist *sg_next(struct scatterlist *);
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 02/16] staging: ccree: Cleanup: remove references to page_link |
| Message-ID | <tKMkj-7HS-37@gated-at.bofh.it> |
| In reply to | #1649970 |
This is a layering violation so we replace it with calls to
sg_page. This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
drivers/staging/ccree/ssi_buffer_mgr.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/ccree/ssi_buffer_mgr.c b/drivers/staging/ccree/ssi_buffer_mgr.c
index 038e2ff..07bbf8f 100644
--- a/drivers/staging/ccree/ssi_buffer_mgr.c
+++ b/drivers/staging/ccree/ssi_buffer_mgr.c
@@ -43,8 +43,8 @@
#ifdef CC_DEBUG
#define DUMP_SGL(sg) \
while (sg) { \
- SSI_LOG_DEBUG("page=%lu offset=%u length=%u (dma_len=%u) " \
- "dma_addr=%08x\n", (sg)->page_link, (sg)->offset, \
+ SSI_LOG_DEBUG("page=%p offset=%u length=%u (dma_len=%u) " \
+ "dma_addr=%08x\n", sg_page(sg), (sg)->offset, \
(sg)->length, sg_dma_len(sg), (sg)->dma_address); \
(sg) = sg_next(sg); \
}
@@ -442,10 +442,10 @@ static int ssi_buffer_mgr_map_scatterlist(
return -ENOMEM;
}
SSI_LOG_DEBUG("Mapped sg: dma_address=0x%llX "
- "page_link=0x%08lX addr=%pK offset=%u "
+ "page=%p addr=%pK offset=%u "
"length=%u\n",
(unsigned long long)sg_dma_address(sg),
- sg->page_link,
+ sg_page(sg),
sg_virt(sg),
sg->offset, sg->length);
*lbytes = nbytes;
@@ -505,10 +505,10 @@ ssi_aead_handle_config_buf(struct device *dev,
return -ENOMEM;
}
SSI_LOG_DEBUG("Mapped curr_buff: dma_address=0x%llX "
- "page_link=0x%08lX addr=%pK "
+ "page=%p addr=%pK "
"offset=%u length=%u\n",
(unsigned long long)sg_dma_address(&areq_ctx->ccm_adata_sg),
- areq_ctx->ccm_adata_sg.page_link,
+ sg_page(&areq_ctx->ccm_adata_sg),
sg_virt(&areq_ctx->ccm_adata_sg),
areq_ctx->ccm_adata_sg.offset,
areq_ctx->ccm_adata_sg.length);
@@ -540,10 +540,10 @@ static inline int ssi_ahash_handle_curr_buf(struct device *dev,
return -ENOMEM;
}
SSI_LOG_DEBUG("Mapped curr_buff: dma_address=0x%llX "
- "page_link=0x%08lX addr=%pK "
+ "page=%p addr=%pK "
"offset=%u length=%u\n",
(unsigned long long)sg_dma_address(areq_ctx->buff_sg),
- areq_ctx->buff_sg->page_link,
+ sg_page(areq_ctx->buff_sg),
sg_virt(areq_ctx->buff_sg),
areq_ctx->buff_sg->offset,
areq_ctx->buff_sg->length);
@@ -1870,4 +1870,3 @@ int ssi_buffer_mgr_fini(struct ssi_drvdata *drvdata)
}
return 0;
}
-
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 09/16] bvec: introduce bvec_page and bvec_set_page accessors |
| Message-ID | <tKMkj-7HS-45@gated-at.bofh.it> |
| In reply to | #1649970 |
Introduce two accessor functions for bv_page: bvec_page to return the
page and bvec_set_page. A follow on patch will mechanically convert all the
individual uses within the kernel.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
include/linux/bvec.h | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 89b65b8..b5369a0 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -43,6 +43,16 @@ struct bvec_iter {
current bvec */
};
+static inline struct page *bvec_page(const struct bio_vec *bvec)
+{
+ return bvec->bv_page;
+}
+
+static inline void bvec_set_page(struct bio_vec *bvec, struct page *page)
+{
+ bvec->bv_page = page;
+}
+
/*
* various member access, note that bio_data should of course not be used
* on highmem page vectors
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 16/16] nvmet: use unmappable sgl in rdma target |
| Message-ID | <tKMkj-7HS-43@gated-at.bofh.it> |
| In reply to | #1649970 |
This is incomplete but is given to test the unmappable sg page code.
Further work would actually use p2p memory in the rdma target. (One
should imagine something vaguely resembling our original p2pmem RFC[1]
being on top of this.)
We convert to using an unmappable sgl in the rdma nvme target driver and
nvme pci driver. We also fix up any remaining uses that would have caused
a BUG_ON.
This still needs a fair bit of work before it's fully safe. No bugs are hit
with nvme or nullb and Intel swiotlb but more testing (and likely fixes)
need to be done for other block drivers and dma_map arches.
It may make sense to add a flag in the request_queue which indicates
the driver is not going to BUG_ON with an unmappable SGL and it may also
make sense to have CONFIG_SG_UNMAPPABLE depend on architectures that
are known have unmappable safe dma mapping code.
[1] https://lwn.net/Articles/718593/
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
drivers/nvme/host/pci.c | 3 ++-
drivers/nvme/target/Kconfig | 12 ++++++++++++
drivers/nvme/target/io-cmd.c | 2 +-
drivers/nvme/target/rdma.c | 29 +++++++++++++++++++++++++----
4 files changed, 40 insertions(+), 6 deletions(-)
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index adf4133..56becec 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -625,7 +625,8 @@ static int nvme_map_data(struct nvme_dev *dev, struct request *req,
DMA_TO_DEVICE : DMA_FROM_DEVICE;
int ret = BLK_MQ_RQ_QUEUE_ERROR;
- sg_init_table(iod->sg, blk_rq_nr_phys_segments(req));
+ sg_init_unmappable_table(iod->sg, blk_rq_nr_phys_segments(req));
+
iod->nents = blk_rq_map_sg(q, req, iod->sg);
if (!iod->nents)
goto out;
diff --git a/drivers/nvme/target/Kconfig b/drivers/nvme/target/Kconfig
index 03e4ab6..bc71cb1 100644
--- a/drivers/nvme/target/Kconfig
+++ b/drivers/nvme/target/Kconfig
@@ -35,6 +35,18 @@ config NVME_TARGET_RDMA
If unsure, say N.
+config NVME_TARGET_RDMA_P2PMEM
+ bool "Support Peer-2-Peer memory (Experimental)"
+ depends on NVME_TARGET_RDMA
+ select SG_UNMAPPABLE
+ help
+ This enable experimental support for using Peer 2 Peer PCI
+ memory in the NVME RDMA target driver. Enabling this could trigger
+ BUG_ONs when using the target with architectures or block devices
+ that do not currently support DMAing to unmappable memory.
+
+ If unsure, say N.
+
config NVME_TARGET_FC
tristate "NVMe over Fabrics FC target driver"
depends on NVME_TARGET
diff --git a/drivers/nvme/target/io-cmd.c b/drivers/nvme/target/io-cmd.c
index c77940d..84f1804 100644
--- a/drivers/nvme/target/io-cmd.c
+++ b/drivers/nvme/target/io-cmd.c
@@ -75,7 +75,7 @@ static void nvmet_execute_rw(struct nvmet_req *req)
bio_set_op_attrs(bio, op, op_flags);
for_each_sg(req->sg, sg, req->sg_cnt, i) {
- while (bio_add_page(bio, sg_page(sg), sg->length, sg->offset)
+ while (bio_add_pfn(bio, sg_pfn_t(sg), sg->length, sg->offset)
!= sg->length) {
struct bio *prev = bio;
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index 9e45cde..6f926da 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -193,11 +193,30 @@ static void nvmet_rdma_free_sgl(struct scatterlist *sgl, unsigned int nents)
if (!sgl || !nents)
return;
- for_each_sg(sgl, sg, nents, count)
- __free_page(sg_page(sg));
+ for_each_sg(sgl, sg, nents, count) {
+ struct page *pg = pfn_t_to_page(sg_pfn_t(sg));
+
+ if (pg)
+ __free_page(pg);
+ }
+
kfree(sgl);
}
+#ifdef CONFIG_NVME_TARGET_RDMA_P2PMEM
+static void nvmet_rdma_init_sg(struct scatterlist *sg,
+ unsigned int nent)
+{
+ sg_init_unmappable_table(sg, nent);
+}
+#else
+static void nvmet_rdma_init_sg(struct scatterlist *sg,
+ unsigned int nent)
+{
+ sg_init_table(sg, nent);
+}
+#endif
+
static int nvmet_rdma_alloc_sgl(struct scatterlist **sgl, unsigned int *nents,
u32 length)
{
@@ -211,7 +230,7 @@ static int nvmet_rdma_alloc_sgl(struct scatterlist **sgl, unsigned int *nents,
if (!sg)
goto out;
- sg_init_table(sg, nent);
+ nvmet_rdma_init_sg(sg, nent);
while (length) {
u32 page_len = min_t(u32, length, PAGE_SIZE);
@@ -231,7 +250,9 @@ static int nvmet_rdma_alloc_sgl(struct scatterlist **sgl, unsigned int *nents,
out_free_pages:
while (i > 0) {
i--;
- __free_page(sg_page(&sg[i]));
+ page = pfn_t_to_page(sg_pfn_t(&sg[i]));
+ if (page)
+ __free_page(page);
}
kfree(sg);
out:
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 11/16] bvec: convert to using pfn_t internally |
| Message-ID | <tKMkj-7HS-47@gated-at.bofh.it> |
| In reply to | #1649970 |
With bv_page fenced off, we can now convert it to a pfn_t fairly simply.
Everything should work the same with the exception of a BUG_ON
for potentially unmappable pfns.
A couple of initializers that assign {NULL} to a bio_vec had to also be
changed to {} to prevent a warning seeing pfn_t is actually a struct.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
block/blk-integrity.c | 4 ++--
block/blk-merge.c | 6 +++---
include/linux/bvec.h | 13 +++++++++----
3 files changed, 14 insertions(+), 9 deletions(-)
diff --git a/block/blk-integrity.c b/block/blk-integrity.c
index 857138a..2ba66f7 100644
--- a/block/blk-integrity.c
+++ b/block/blk-integrity.c
@@ -40,7 +40,7 @@
*/
int blk_rq_count_integrity_sg(struct request_queue *q, struct bio *bio)
{
- struct bio_vec iv, ivprv = { NULL };
+ struct bio_vec iv, ivprv = {};
unsigned int segments = 0;
unsigned int seg_size = 0;
struct bvec_iter iter;
@@ -86,7 +86,7 @@ EXPORT_SYMBOL(blk_rq_count_integrity_sg);
int blk_rq_map_integrity_sg(struct request_queue *q, struct bio *bio,
struct scatterlist *sglist)
{
- struct bio_vec iv, ivprv = { NULL };
+ struct bio_vec iv, ivprv = {};
struct scatterlist *sg = NULL;
unsigned int segments = 0;
struct bvec_iter iter;
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 65dcee9..93e87fe 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -245,7 +245,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q,
struct bio *bio,
bool no_sg_merge)
{
- struct bio_vec bv, bvprv = { NULL };
+ struct bio_vec bv, bvprv = {};
int cluster, prev = 0;
unsigned int seg_size, nr_phys_segs;
struct bio *fbio, *bbio;
@@ -347,7 +347,7 @@ EXPORT_SYMBOL(blk_recount_segments);
static int blk_phys_contig_segment(struct request_queue *q, struct bio *bio,
struct bio *nxt)
{
- struct bio_vec end_bv = { NULL }, nxt_bv;
+ struct bio_vec end_bv = {}, nxt_bv;
if (!blk_queue_cluster(q))
return 0;
@@ -430,7 +430,7 @@ static int __blk_bios_map_sg(struct request_queue *q, struct bio *bio,
struct scatterlist *sglist,
struct scatterlist **sg)
{
- struct bio_vec bvec, bvprv = { NULL };
+ struct bio_vec bvec, bvprv = {};
struct bvec_iter iter;
int cluster = blk_queue_cluster(q), nsegs = 0;
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index b53b879..d950288 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -22,12 +22,13 @@
#include <linux/kernel.h>
#include <linux/bug.h>
+#include <linux/pfn_t.h>
/*
* was unsigned short, but we might as well be ready for > 64kB I/O pages
*/
struct bio_vec {
- struct page *bv_page;
+ pfn_t bv_pfn;
unsigned int bv_len;
unsigned int bv_offset;
};
@@ -45,12 +46,13 @@ struct bvec_iter {
static inline struct page *bvec_page(const struct bio_vec *bvec)
{
- return bvec->bv_page;
+ BUG_ON(!pfn_t_is_always_mappable(bvec->bv_pfn));
+ return pfn_t_to_page(bvec->bv_pfn);
}
static inline void bvec_set_page(struct bio_vec *bvec, struct page *page)
{
- bvec->bv_page = page;
+ bvec->bv_pfn = page_to_pfn_t(page);
}
/*
@@ -59,6 +61,9 @@ static inline void bvec_set_page(struct bio_vec *bvec, struct page *page)
*/
#define __bvec_iter_bvec(bvec, iter) (&(bvec)[(iter).bi_idx])
+#define bvec_iter_pfn_t(bvec, iter) \
+ (__bvec_iter_bvec((bvec), (iter))->bv_pfn)
+
#define bvec_iter_page(bvec, iter) \
(bvec_page(__bvec_iter_bvec((bvec), (iter))))
@@ -71,7 +76,7 @@ static inline void bvec_set_page(struct bio_vec *bvec, struct page *page)
#define bvec_iter_bvec(bvec, iter) \
((struct bio_vec) { \
- .bv_page = bvec_iter_page((bvec), (iter)), \
+ .bv_pfn = bvec_iter_pfn_t((bvec), (iter)), \
.bv_len = bvec_iter_len((bvec), (iter)), \
.bv_offset = bvec_iter_offset((bvec), (iter)), \
})
--
2.1.4
[toc] | [prev] | [next] | [standalone]
| From | Logan Gunthorpe <logang@deltatee.com> |
|---|---|
| Date | 2017-05-24 23:50 +0200 |
| Subject | [RFC PATCH 03/16] kfifo: Cleanup example to not use page_link |
| Message-ID | <tKMkj-7HS-49@gated-at.bofh.it> |
| In reply to | #1649970 |
This is a layering violation so we replace the uses with calls to
sg_page(). This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
---
samples/kfifo/dma-example.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/samples/kfifo/dma-example.c b/samples/kfifo/dma-example.c
index aa243db..be0d4a5 100644
--- a/samples/kfifo/dma-example.c
+++ b/samples/kfifo/dma-example.c
@@ -75,8 +75,8 @@ static int __init example_init(void)
for (i = 0; i < nents; i++) {
printk(KERN_INFO
"sg[%d] -> "
- "page_link 0x%.8lx offset 0x%.8x length 0x%.8x\n",
- i, sg[i].page_link, sg[i].offset, sg[i].length);
+ "page %p offset 0x%.8x length 0x%.8x\n",
+ i, sg_page(&sg[i]), sg[i].offset, sg[i].length);
if (sg_is_last(&sg[i]))
break;
@@ -104,8 +104,8 @@ static int __init example_init(void)
for (i = 0; i < nents; i++) {
printk(KERN_INFO
"sg[%d] -> "
- "page_link 0x%.8lx offset 0x%.8x length 0x%.8x\n",
- i, sg[i].page_link, sg[i].offset, sg[i].length);
+ "page %p offset 0x%.8x length 0x%.8x\n",
+ i, sg_page(&sg[i]), sg[i].offset, sg[i].length);
if (sg_is_last(&sg[i]))
break;
--
2.1.4
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web