Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1256831 > unrolled thread

[PATCH v12 0/5] Support for Open-Channel SSDs

Back to article view | Back to linux.kernel

Contents

  [PATCH v12 0/5] Support for Open-Channel SSDs Matias Bjørling <m@bjorling.me> - 2015-10-27 16:00 +0100
    [PATCH v12 2/5] gennvm: Generic NVM manager Matias Bjørling <m@bjorling.me> - 2015-10-27 16:00 +0100
    [PATCH v12 4/5] null_nvm: LightNVM test driver Matias Bjørling <m@bjorling.me> - 2015-10-27 16:10 +0100

#1256831 — [PATCH v12 0/5] Support for Open-Channel SSDs

This patchset implements support for Open-Channel SSDs.

Applies against Jens' for-4.4/drivers and is available through the lkml_v12
branch at:

  https://github.com/OpenChannelSSD/linux

Changes since v11:
 - Add support for group identifiers.
 - Fixed kbuild error on nvme_nvm_unregister.

Changes since v10:
 - Fix sparse errors.
 - Enable flash command support for SLC mode, suspend/resume, plane hints.
 - A couple of optimizations for the rq to nvme conversion.
 - static function fixes (Fengguang Wu)

Changes since v9:
 - Fix compilation of the kernel without LightNVM selected.
 - Make static completion_queues variable unique so it doesn't collide with
   the null_blk module.

Changes since v8:
 - Specification was changed to define a SSD configuration using a single entry.
   This replaces the per-lun specification with a single configuration. Easing
   the access to global geometry values and simplifies address calculations.
 - Implement PPA addressing modes. Encode NAND flash structure into physical
   page addresses. An address now has bits reserved for each part of the flash
   units (sectors, page, block, plane, lun, channels), replacing the linear
   address space previously found in the LightNVM specification.
 - Rename bm_hb to gennvm (generic nvm manager). This part now covers any type
   of non-volatile memory media. Thus, it is not only a block manager, but
   manages several pieces of core information.
 - Add detection for CNEX Westlake controller.
 - Completion of I/Os was previously directly completed in the target. Push
   completion through the media manager first and then target.
 - Rebased on top of latest changes to the NVMe device driver architecture.
 - Various fixes and refactoring.

Changes since v7:
 - The IOCTL management interface have been created. Devices can now be
   managed through the lightnvm-adm tool (ladm). This is available from:
     https://github.com/OpenChannelSSD/lightnvm-adm.git
 - The debugging management interface is now only available if debugging
   is enabled. (/sys/module/lnvm/parameters/configure_debug)
 - GCC compatibility fixes (Dongsheng Yang)
 - Fix for BM initialization (Dongsheng Yang)
 - Fix for error during rrpc init (Javier Gonzalez)
 - Refactor NVMe driver to not expect non-standard behavior. Added PCI
   id identification for selected devices (qemu-nvme and cnex).
 - Enable NVMe metadata to be passed from target.
 - Create ioctl interface for management.
 - Add NVM_DEBUG flag to explicit enable/diskable parameter
   configure_debug access. The interface is only available if debugging
   is enabled.
 - Add support for physical block address commands.
 - Added uapi/lightnvm.h header to expose ioctl interface.
 - Fix qemu-nvme initialization extensions regarding L2P translation
   support.
 - Clean up of dead code.
 - Updated e-mails, copyrights, etc.

Changes since v6:
 - Multipage support (Javier Gonzalez)
 - General code cleanups
 - Fixed memleak on register failure

Changes since v5:
Feedback from Christoph Hellwig.
 - Created new null_nvm from null_blk to register itself as a lightnvm
   device.
 - Changed the interface of register/unregister to only take disk_name.
   The gendisk alloc in nvme is kept. Most instantiations will
   involve the device gendisk, therefore wait with refactoring to a
   later time.
 - Renamed global parameters in core.c and rrpc.c

Changes since v4:
 - Remove gendisk->nvm dependency
 - Remove device driver rq private field dependency.
 - Update submission and completion. The flow is now
     Target -> Block Manager -> Device Driver, replacing callbacks in
     device driver.
 - Abstracted out the block manager into its own module. Other block
   managers can now be implemented. For example to support fully
   host-based SSDs.
 - No longer exposes the device driver gendisk to user-space.
 - Management is moved into /sys/modules/lnvm/parameters/configure_debug

Changes since v3:

 - Remove dependency on REQ_NVM_GC
 - Refactor nvme integration to use nvme_submit_sync_cmd for
   internal commands.
 - Fix race condition bug on multiple threads on RRPC target.
 - Rename sysfs entry under the block device from nvm to lightnvm.
   The configuration is found in /sys/block/*/lightnvm/

Changes since v2:

 Feedback from Paul Bolle:
 - Fix license to GPLv2, documentation, compilation.
 Feedback from Keith Busch:
 - nvme: Move lightnvm out and into nvme-lightnvm.c.
 - nvme: Set controller css on lightnvm command set.
 - nvme: Remove OACS.
 Feedback from Christoph Hellwig:
 - lightnvm: Move out of block layer into /drivers/lightnvm/core.c
 - lightnvm: refactor request->phys_sector into device drivers.
 - lightnvm: refactor prep/unprep into device drivers.
 - lightnvm: move nvm_dev from request_queue to gendisk.

 New
 - Bad block table support (From Javier).
 - Update maintainers file.

Changes since v1:

 - Splitted LightNVM into two parts. A get/put interface for flash
   blocks and the respective targets that implement flash translation
   layer logic.
 - Updated the patches according to the LightNVM specification changes.
 - Added interface to add/remove targets for a block device.

Thanks to Jens Axboe, Christoph Hellwig, Keith Busch, Paul Bolle,
Javier Gonzalez and Jesper Madsen for discussions and contributions.

Matias Bjørling (5):
  lightnvm: Support for Open-Channel SSDs
  gennvm: Generic NVM manager
  rrpc: Round-robin sector target with cost-based gc
  null_nvm: LightNVM test driver
  nvme: LightNVM support

 Documentation/ioctl/ioctl-number.txt |    1 +
 MAINTAINERS                          |    8 +
 drivers/Kconfig                      |    2 +
 drivers/Makefile                     |    1 +
 drivers/lightnvm/Kconfig             |   48 ++
 drivers/lightnvm/Makefile            |    8 +
 drivers/lightnvm/core.c              |  829 +++++++++++++++++++++
 drivers/lightnvm/gennvm.c            |  475 ++++++++++++
 drivers/lightnvm/gennvm.h            |   46 ++
 drivers/lightnvm/null_nvm.c          |  457 ++++++++++++
 drivers/lightnvm/rrpc.c              | 1316 ++++++++++++++++++++++++++++++++++
 drivers/lightnvm/rrpc.h              |  239 ++++++
 drivers/nvme/host/Makefile           |    2 +-
 drivers/nvme/host/lightnvm.c         |  524 ++++++++++++++
 drivers/nvme/host/nvme.h             |   10 +
 drivers/nvme/host/pci.c              |   39 +-
 include/linux/lightnvm.h             |  536 ++++++++++++++
 include/uapi/linux/lightnvm.h        |  130 ++++
 18 files changed, 4659 insertions(+), 12 deletions(-)
 create mode 100644 drivers/lightnvm/Kconfig
 create mode 100644 drivers/lightnvm/Makefile
 create mode 100644 drivers/lightnvm/core.c
 create mode 100644 drivers/lightnvm/gennvm.c
 create mode 100644 drivers/lightnvm/gennvm.h
 create mode 100644 drivers/lightnvm/null_nvm.c
 create mode 100644 drivers/lightnvm/rrpc.c
 create mode 100644 drivers/lightnvm/rrpc.h
 create mode 100644 drivers/nvme/host/lightnvm.c
 create mode 100644 include/linux/lightnvm.h
 create mode 100644 include/uapi/linux/lightnvm.h

-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1256835 — [PATCH v12 2/5] gennvm: Generic NVM manager

The implementation for Open-Channel SSDs is divided into media
management and targets. This patch implements a generic media manager
for open-channel SSDs. After a media manager has been initialized,
single or multiple targets can be instantiated with the media managed as
the backend.

Signed-off-by: Matias Bjørling <m@bjorling.me>
---
 drivers/lightnvm/Kconfig  |   7 +
 drivers/lightnvm/Makefile |   1 +
 drivers/lightnvm/gennvm.c | 475 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/lightnvm/gennvm.h |  46 +++++
 4 files changed, 529 insertions(+)
 create mode 100644 drivers/lightnvm/gennvm.c
 create mode 100644 drivers/lightnvm/gennvm.h

diff --git a/drivers/lightnvm/Kconfig b/drivers/lightnvm/Kconfig
index d4f309f..2a27971 100644
--- a/drivers/lightnvm/Kconfig
+++ b/drivers/lightnvm/Kconfig
@@ -25,4 +25,11 @@ config NVM_DEBUG
 
 	It is required to create/remove targets without IOCTLs.
 
+config NVM_GENNVM
+	tristate "Generic NVM manager for Open-Channel SSDs"
+	---help---
+	NVM media manager for Open-Channel SSDs that offload management
+	functionality to device, while keeping data placement and garbage
+	collection decisions on the host.
+
 endif # NVM
diff --git a/drivers/lightnvm/Makefile b/drivers/lightnvm/Makefile
index 38185e9..e2428e0 100644
--- a/drivers/lightnvm/Makefile
+++ b/drivers/lightnvm/Makefile
@@ -3,3 +3,4 @@
 #
 
 obj-$(CONFIG_NVM)		:= core.o
+obj-$(CONFIG_NVM_GENNVM) 	+= gennvm.o
diff --git a/drivers/lightnvm/gennvm.c b/drivers/lightnvm/gennvm.c
new file mode 100644
index 0000000..ba4d962
--- /dev/null
+++ b/drivers/lightnvm/gennvm.c
@@ -0,0 +1,475 @@
+/*
+ * Copyright (C) 2015 Matias Bjorling <m@bjorling.me>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING.  If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139,
+ * USA.
+ *
+ * Implementation of a generic nvm manager for Open-Channel SSDs.
+ */
+
+#include "gennvm.h"
+
+static void gennvm_blocks_free(struct nvm_dev *dev)
+{
+	struct gen_nvm *gn = dev->mp;
+	struct gen_lun *lun;
+	int i;
+
+	gennvm_for_each_lun(gn, lun, i) {
+		if (!lun->vlun.blocks)
+			break;
+		vfree(lun->vlun.blocks);
+	}
+}
+
+static void gennvm_luns_free(struct nvm_dev *dev)
+{
+	struct gen_nvm *gn = dev->mp;
+
+	kfree(gn->luns);
+}
+
+static int gennvm_luns_init(struct nvm_dev *dev, struct gen_nvm *gn)
+{
+	struct gen_lun *lun;
+	int i;
+
+	gn->luns = kcalloc(dev->nr_luns, sizeof(struct gen_lun), GFP_KERNEL);
+	if (!gn->luns)
+		return -ENOMEM;
+
+	gennvm_for_each_lun(gn, lun, i) {
+		spin_lock_init(&lun->vlun.lock);
+		INIT_LIST_HEAD(&lun->free_list);
+		INIT_LIST_HEAD(&lun->used_list);
+		INIT_LIST_HEAD(&lun->bb_list);
+
+		lun->reserved_blocks = 2; /* for GC only */
+		lun->vlun.id = i;
+		lun->vlun.lun_id = i % dev->luns_per_chnl;
+		lun->vlun.chnl_id = i / dev->luns_per_chnl;
+		lun->vlun.nr_free_blocks = dev->blks_per_lun;
+	}
+	return 0;
+}
+
+static int gennvm_block_bb(u32 lun_id, void *bb_bitmap, unsigned int nr_blocks,
+								void *private)
+{
+	struct gen_nvm *gn = private;
+	struct gen_lun *lun = &gn->luns[lun_id];
+	struct nvm_block *block;
+	int i;
+
+	if (unlikely(bitmap_empty(bb_bitmap, nr_blocks)))
+		return 0;
+
+	i = -1;
+	while ((i = find_next_bit(bb_bitmap, nr_blocks, i + 1)) <
+			nr_blocks) {
+		block = &lun->vlun.blocks[i];
+		if (!block) {
+			pr_err("gen_nvm: BB data is out of bounds.\n");
+			return -EINVAL;
+		}
+		list_move_tail(&block->list, &lun->bb_list);
+	}
+
+	return 0;
+}
+
+static int gennvm_block_map(u64 slba, u32 nlb, __le64 *entries, void *private)
+{
+	struct nvm_dev *dev = private;
+	struct gen_nvm *gn = dev->mp;
+	sector_t max_pages = dev->total_pages * (dev->sec_size >> 9);
+	u64 elba = slba + nlb;
+	struct gen_lun *lun;
+	struct nvm_block *blk;
+	u64 i;
+	int lun_id;
+
+	if (unlikely(elba > dev->total_pages)) {
+		pr_err("gen_nvm: L2P data from device is out of bounds!\n");
+		return -EINVAL;
+	}
+
+	for (i = 0; i < nlb; i++) {
+		u64 pba = le64_to_cpu(entries[i]);
+
+		if (unlikely(pba >= max_pages && pba != U64_MAX)) {
+			pr_err("gen_nvm: L2P data entry is out of bounds!\n");
+			return -EINVAL;
+		}
+
+		/* Address zero is a special one. The first page on a disk is
+		 * protected. It often holds internal device boot
+		 * information.
+		 */
+		if (!pba)
+			continue;
+
+		/* resolve block from physical address */
+		lun_id = pba / dev->sec_per_lun;
+		lun = &gn->luns[lun_id];
+
+		/* Calculate block offset into lun */
+		pba = pba - (dev->sec_per_lun * lun_id);
+		blk = &lun->vlun.blocks[pba / dev->sec_per_blk];
+
+		if (!blk->type) {
+			/* at this point, we don't know anything about the
+			 * block. It's up to the FTL on top to re-etablish the
+			 * block state
+			 */
+			list_move_tail(&blk->list, &lun->used_list);
+			blk->type = 1;
+			lun->vlun.nr_free_blocks--;
+		}
+	}
+
+	return 0;
+}
+
+static int gennvm_blocks_init(struct nvm_dev *dev, struct gen_nvm *gn)
+{
+	struct gen_lun *lun;
+	struct nvm_block *block;
+	sector_t lun_iter, blk_iter, cur_block_id = 0;
+	int ret;
+
+	gennvm_for_each_lun(gn, lun, lun_iter) {
+		lun->vlun.blocks = vzalloc(sizeof(struct nvm_block) *
+							dev->blks_per_lun);
+		if (!lun->vlun.blocks)
+			return -ENOMEM;
+
+		for (blk_iter = 0; blk_iter < dev->blks_per_lun; blk_iter++) {
+			block = &lun->vlun.blocks[blk_iter];
+
+			INIT_LIST_HEAD(&block->list);
+
+			block->lun = &lun->vlun;
+			block->id = cur_block_id++;
+
+			/* First block is reserved for device */
+			if (unlikely(lun_iter == 0 && blk_iter == 0))
+				continue;
+
+			list_add_tail(&block->list, &lun->free_list);
+		}
+
+		if (dev->ops->get_bb_tbl) {
+			ret = dev->ops->get_bb_tbl(dev->q, lun->vlun.id,
+					dev->blks_per_lun, gennvm_block_bb, gn);
+			if (ret)
+				pr_err("gen_nvm: could not read BB table\n");
+		}
+	}
+
+	if (dev->ops->get_l2p_tbl) {
+		ret = dev->ops->get_l2p_tbl(dev->q, 0, dev->total_pages,
+							gennvm_block_map, dev);
+		if (ret) {
+			pr_err("gen_nvm: could not read L2P table.\n");
+			pr_warn("gen_nvm: default block initialization");
+		}
+	}
+
+	return 0;
+}
+
+static int gennvm_register(struct nvm_dev *dev)
+{
+	struct gen_nvm *gn;
+	int ret;
+
+	gn = kzalloc(sizeof(struct gen_nvm), GFP_KERNEL);
+	if (!gn)
+		return -ENOMEM;
+
+	gn->nr_luns = dev->nr_luns;
+	dev->mp = gn;
+
+	ret = gennvm_luns_init(dev, gn);
+	if (ret) {
+		pr_err("gen_nvm: could not initialize luns\n");
+		goto err;
+	}
+
+	ret = gennvm_blocks_init(dev, gn);
+	if (ret) {
+		pr_err("gen_nvm: could not initialize blocks\n");
+		goto err;
+	}
+
+	return 1;
+err:
+	kfree(gn);
+	return ret;
+}
+
+static void gennvm_unregister(struct nvm_dev *dev)
+{
+	gennvm_blocks_free(dev);
+	gennvm_luns_free(dev);
+	kfree(dev->mp);
+	dev->mp = NULL;
+}
+
+static struct nvm_block *gennvm_get_blk(struct nvm_dev *dev,
+				struct nvm_lun *vlun, unsigned long flags)
+{
+	struct gen_lun *lun = container_of(vlun, struct gen_lun, vlun);
+	struct nvm_block *blk = NULL;
+	int is_gc = flags & NVM_IOTYPE_GC;
+
+	BUG_ON(!lun);
+
+	spin_lock(&vlun->lock);
+
+	if (list_empty(&lun->free_list)) {
+		pr_err_ratelimited("gen_nvm: lun %u have no free pages available",
+								lun->vlun.id);
+		spin_unlock(&vlun->lock);
+		goto out;
+	}
+
+	while (!is_gc && lun->vlun.nr_free_blocks < lun->reserved_blocks) {
+		spin_unlock(&vlun->lock);
+		goto out;
+	}
+
+	blk = list_first_entry(&lun->free_list, struct nvm_block, list);
+	list_move_tail(&blk->list, &lun->used_list);
+	blk->type = 1;
+
+	lun->vlun.nr_free_blocks--;
+
+	spin_unlock(&vlun->lock);
+out:
+	return blk;
+}
+
+static void gennvm_put_blk(struct nvm_dev *dev, struct nvm_block *blk)
+{
+	struct nvm_lun *vlun = blk->lun;
+	struct gen_lun *lun = container_of(vlun, struct gen_lun, vlun);
+
+	spin_lock(&vlun->lock);
+
+	switch (blk->type) {
+	case 1:
+		list_move_tail(&blk->list, &lun->free_list);
+		lun->vlun.nr_free_blocks++;
+		blk->type = 0;
+		break;
+	case 2:
+		list_move_tail(&blk->list, &lun->bb_list);
+		break;
+	default:
+		BUG();
+	}
+
+	spin_unlock(&vlun->lock);
+}
+
+static void gennvm_addr_to_generic_mode(struct nvm_dev *dev, struct nvm_rq *rqd)
+{
+	int i;
+
+	if (rqd->nr_pages > 1)
+		for (i = 0; i < rqd->nr_pages; i++)
+			rqd->ppa_list[i] = addr_to_generic_mode(dev,
+							rqd->ppa_list[i]);
+	else
+		rqd->ppa_addr = addr_to_generic_mode(dev, rqd->ppa_addr);
+}
+
+static void gennvm_generic_to_addr_mode(struct nvm_dev *dev, struct nvm_rq *rqd)
+{
+	int i;
+
+	if (rqd->nr_pages > 1)
+		for (i = 0; i < rqd->nr_pages; i++)
+			rqd->ppa_list[i] = generic_to_addr_mode(dev,
+							rqd->ppa_list[i]);
+	else
+		rqd->ppa_addr = generic_to_addr_mode(dev, rqd->ppa_addr);
+}
+
+static int gennvm_submit_io(struct nvm_dev *dev, struct nvm_rq *rqd)
+{
+	if (!dev->ops->submit_io)
+		return 0;
+
+	/* Convert address space */
+	gennvm_generic_to_addr_mode(dev, rqd);
+
+	rqd->dev = dev;
+	return dev->ops->submit_io(dev->q, rqd);
+}
+
+static void gennvm_blk_set_type(struct nvm_dev *dev, struct ppa_addr *ppa,
+								int type)
+{
+	struct gen_nvm *gn = dev->mp;
+	struct gen_lun *lun;
+	struct nvm_block *blk;
+
+	BUG_ON(ppa->g.ch > dev->nr_chnls);
+	BUG_ON(ppa->g.lun > dev->luns_per_chnl);
+	BUG_ON(ppa->g.blk > dev->blks_per_lun);
+
+	lun = &gn->luns[ppa->g.lun * ppa->g.ch];
+	blk = &lun->vlun.blocks[ppa->g.blk];
+
+	/* will be moved to bb list on put_blk from target */
+	blk->type = type;
+}
+
+/* mark block bad. It is expected the target recover from the error. */
+static void gennvm_mark_blk_bad(struct nvm_dev *dev, struct nvm_rq *rqd)
+{
+	int i;
+
+	if (!dev->ops->set_bb)
+		return;
+
+	if (dev->ops->set_bb(dev->q, rqd, 1))
+		return;
+
+	gennvm_addr_to_generic_mode(dev, rqd);
+
+	/* look up blocks and mark them as bad */
+	if (rqd->nr_pages > 1)
+		for (i = 0; i < rqd->nr_pages; i++)
+			gennvm_blk_set_type(dev, &rqd->ppa_list[i], 2);
+	else
+		gennvm_blk_set_type(dev, &rqd->ppa_addr, 2);
+}
+
+static int gennvm_end_io(struct nvm_rq *rqd, int error)
+{
+	struct nvm_tgt_instance *ins = rqd->ins;
+	int ret = 0;
+
+	switch (error) {
+	case NVM_RSP_SUCCESS:
+		break;
+	case NVM_RSP_ERR_EMPTYPAGE:
+		break;
+	case NVM_RSP_ERR_FAILWRITE:
+		gennvm_mark_blk_bad(rqd->dev, rqd);
+	default:
+		ret++;
+	}
+
+	ret += ins->tt->end_io(rqd, error);
+
+	return ret;
+}
+
+static int gennvm_erase_blk(struct nvm_dev *dev, struct nvm_block *blk,
+							unsigned long flags)
+{
+	int plane_cnt = 0, pl_idx, ret;
+	struct ppa_addr addr;
+	struct nvm_rq rqd;
+
+	if (!dev->ops->erase_block)
+		return 0;
+
+	addr = block_to_ppa(dev, blk);
+
+	if (dev->plane_mode == NVM_PLANE_SINGLE) {
+		rqd.nr_pages = 1;
+		rqd.ppa_addr = addr;
+	} else {
+		plane_cnt = (1 << dev->plane_mode);
+		rqd.nr_pages = plane_cnt;
+
+		rqd.ppa_list = nvm_dev_dma_alloc(dev, GFP_KERNEL,
+							&rqd.dma_ppa_list);
+		if (!rqd.ppa_list) {
+			pr_err("gen_nvm: failed to allocate dma memory\n");
+			return -ENOMEM;
+		}
+
+		for (pl_idx = 0; pl_idx < plane_cnt; pl_idx++) {
+			addr.g.pl = pl_idx;
+			rqd.ppa_list[pl_idx] = addr;
+		}
+	}
+
+	gennvm_generic_to_addr_mode(dev, &rqd);
+
+	ret = dev->ops->erase_block(dev->q, &rqd);
+
+	if (plane_cnt)
+		nvm_dev_dma_free(dev, rqd.ppa_list, rqd.dma_ppa_list);
+
+	return ret;
+}
+
+static struct nvm_lun *gennvm_get_lun(struct nvm_dev *dev, int lunid)
+{
+	struct gen_nvm *gn = dev->mp;
+
+	return &gn->luns[lunid].vlun;
+}
+
+static void gennvm_free_blocks_print(struct nvm_dev *dev)
+{
+	struct gen_nvm *gn = dev->mp;
+	struct gen_lun *lun;
+	unsigned int i;
+
+	gennvm_for_each_lun(gn, lun, i)
+		pr_info("%s: lun%8u\t%u\n",
+					dev->name, i, lun->vlun.nr_free_blocks);
+}
+
+static struct nvmm_type gennvm = {
+	.name		= "gennvm",
+	.version	= {0, 1, 0},
+
+	.register_mgr	= gennvm_register,
+	.unregister_mgr	= gennvm_unregister,
+
+	.get_blk	= gennvm_get_blk,
+	.put_blk	= gennvm_put_blk,
+
+	.submit_io	= gennvm_submit_io,
+	.end_io		= gennvm_end_io,
+	.erase_blk	= gennvm_erase_blk,
+
+	.get_lun	= gennvm_get_lun,
+	.free_blocks_print = gennvm_free_blocks_print,
+};
+
+static int __init gennvm_module_init(void)
+{
+	return nvm_register_mgr(&gennvm);
+}
+
+static void gennvm_module_exit(void)
+{
+	nvm_unregister_mgr(&gennvm);
+}
+
+module_init(gennvm_module_init);
+module_exit(gennvm_module_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Block manager for Hybrid Open-Channel SSDs");
diff --git a/drivers/lightnvm/gennvm.h b/drivers/lightnvm/gennvm.h
new file mode 100644
index 0000000..d23bd35
--- /dev/null
+++ b/drivers/lightnvm/gennvm.h
@@ -0,0 +1,46 @@
+/*
+ * Copyright: Matias Bjorling <mb@bjorling.me>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ */
+
+#ifndef GENNVM_H_
+#define GENNVM_H_
+
+#include <linux/module.h>
+#include <linux/vmalloc.h>
+
+#include <linux/lightnvm.h>
+
+struct gen_lun {
+	struct nvm_lun vlun;
+
+	int reserved_blocks;
+	/* lun block lists */
+	struct list_head used_list;	/* In-use blocks */
+	struct list_head free_list;	/* Not used blocks i.e. released
+					 * and ready for use
+					 */
+	struct list_head bb_list;	/* Bad blocks. Mutually exclusive with
+					 * free_list and used_list
+					 */
+};
+
+struct gen_nvm {
+	int nr_luns;
+	struct gen_lun *luns;
+};
+
+#define gennvm_for_each_lun(bm, lun, i) \
+		for ((i) = 0, lun = &(bm)->luns[0]; \
+			(i) < (bm)->nr_luns; (i)++, lun = &(bm)->luns[(i)])
+
+#endif /* GENNVM_H_ */
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1256836 — [PATCH v12 4/5] null_nvm: LightNVM test driver

This driver implements the I/O flow for a LightNVM device driver. It
does no transfers. It can be used to test setup/teardown of devices and
evaluating performance of media managers and targets.

The framework of the driver is derived from the null_blk module.

Signed-off-by: Matias Bjørling <m@bjorling.me>
---
 drivers/lightnvm/Kconfig    |   6 +
 drivers/lightnvm/Makefile   |   1 +
 drivers/lightnvm/null_nvm.c | 457 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 464 insertions(+)
 create mode 100644 drivers/lightnvm/null_nvm.c

diff --git a/drivers/lightnvm/Kconfig b/drivers/lightnvm/Kconfig
index a16bf56..9111170 100644
--- a/drivers/lightnvm/Kconfig
+++ b/drivers/lightnvm/Kconfig
@@ -39,4 +39,10 @@ config NVM_RRPC
 	host. The target is implemented using a linear mapping table and
 	cost-based garbage collection. It is optimized for 4K IO sizes.
 
+config NVM_NULL_NVM
+	tristate "Null test LightNVM driver"
+	---help---
+	The null test driver can be used to evaluate targets without an
+	underlying device.
+
 endif # NVM
diff --git a/drivers/lightnvm/Makefile b/drivers/lightnvm/Makefile
index 7e0f42a..2f1287b 100644
--- a/drivers/lightnvm/Makefile
+++ b/drivers/lightnvm/Makefile
@@ -5,3 +5,4 @@
 obj-$(CONFIG_NVM)		:= core.o
 obj-$(CONFIG_NVM_GENNVM) 	+= gennvm.o
 obj-$(CONFIG_NVM_RRPC)		+= rrpc.o
+obj-$(CONFIG_NVM_NULL_NVM)	+= null_nvm.o
diff --git a/drivers/lightnvm/null_nvm.c b/drivers/lightnvm/null_nvm.c
new file mode 100644
index 0000000..f191d98
--- /dev/null
+++ b/drivers/lightnvm/null_nvm.c
@@ -0,0 +1,457 @@
+/*
+ * derived from Jens Axboe's block/null_blk.c
+ */
+
+#include <linux/module.h>
+
+#include <linux/moduleparam.h>
+#include <linux/sched.h>
+#include <linux/blkdev.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/blk-mq.h>
+#include <linux/hrtimer.h>
+#include <linux/lightnvm.h>
+
+static struct kmem_cache *ppa_cache;
+struct nulln_cmd {
+	struct llist_node ll_list;
+	struct request *rq;
+};
+
+struct nulln {
+	struct list_head list;
+	unsigned int index;
+	struct request_queue *q;
+	struct blk_mq_tag_set tag_set;
+	struct hrtimer timer;
+	char disk_name[DISK_NAME_LEN];
+};
+
+static LIST_HEAD(nulln_list);
+static struct mutex nulln_lock;
+static int nulln_indexes;
+
+struct completion_queue {
+	struct llist_head list;
+	struct hrtimer timer;
+};
+
+/*
+ * These are per-cpu for now, they will need to be configured by the
+ * complete_queues parameter and appropriately mapped.
+ */
+static DEFINE_PER_CPU(struct completion_queue, null_comp_queues);
+
+enum {
+	NULL_IRQ_NONE		= 0,
+	NULL_IRQ_SOFTIRQ	= 1,
+	NULL_IRQ_TIMER		= 2,
+};
+
+static int submit_queues;
+module_param(submit_queues, int, S_IRUGO);
+MODULE_PARM_DESC(submit_queues, "Number of submission queues");
+
+static int home_node = NUMA_NO_NODE;
+module_param(home_node, int, S_IRUGO);
+MODULE_PARM_DESC(home_node, "Home node for the device");
+
+static int null_param_store_val(const char *str, int *val, int min, int max)
+{
+	int ret, new_val;
+
+	ret = kstrtoint(str, 10, &new_val);
+	if (ret)
+		return -EINVAL;
+
+	if (new_val < min || new_val > max)
+		return -EINVAL;
+
+	*val = new_val;
+	return 0;
+}
+
+static int gb = 250;
+module_param(gb, int, S_IRUGO);
+MODULE_PARM_DESC(gb, "Size in GB");
+
+static int bs = 4096;
+module_param(bs, int, S_IRUGO);
+MODULE_PARM_DESC(bs, "Block size (in bytes)");
+
+static int nr_devices = 1;
+module_param(nr_devices, int, S_IRUGO);
+MODULE_PARM_DESC(nr_devices, "Number of devices to register");
+
+static int irqmode = NULL_IRQ_SOFTIRQ;
+
+static int null_set_irqmode(const char *str, const struct kernel_param *kp)
+{
+	return null_param_store_val(str, &irqmode, NULL_IRQ_NONE,
+					NULL_IRQ_TIMER);
+}
+
+static const struct kernel_param_ops null_irqmode_param_ops = {
+	.set	= null_set_irqmode,
+	.get	= param_get_int,
+};
+
+device_param_cb(irqmode, &null_irqmode_param_ops, &irqmode, S_IRUGO);
+MODULE_PARM_DESC(irqmode, "IRQ completion handler. 0-none, 1-softirq, 2-timer");
+
+static int completion_nsec = 10000;
+module_param(completion_nsec, int, S_IRUGO);
+MODULE_PARM_DESC(completion_nsec, "Time in ns to complete a request in hardware. Default: 10,000ns");
+
+static int hw_queue_depth = 64;
+module_param(hw_queue_depth, int, S_IRUGO);
+MODULE_PARM_DESC(hw_queue_depth, "Queue depth for each hardware queue. Default: 64");
+
+static bool use_per_node_hctx;
+module_param(use_per_node_hctx, bool, S_IRUGO);
+MODULE_PARM_DESC(use_per_node_hctx, "Use per-node allocation for hardware context queues. Default: false");
+
+static int num_channels = 1;
+module_param(num_channels, int, S_IRUGO);
+MODULE_PARM_DESC(num_channels, "Number of channels to be exposed. Default: 1");
+
+static enum hrtimer_restart null_cmd_timer_expired(struct hrtimer *timer)
+{
+	struct completion_queue *cq;
+	struct llist_node *entry;
+	struct nulln_cmd *cmd;
+
+	cq = &per_cpu(null_comp_queues, smp_processor_id());
+
+	while ((entry = llist_del_all(&cq->list)) != NULL) {
+		entry = llist_reverse_order(entry);
+		do {
+			cmd = container_of(entry, struct nulln_cmd, ll_list);
+			entry = entry->next;
+			blk_mq_end_request(cmd->rq, 0);
+
+			if (cmd->rq) {
+				struct request_queue *q = cmd->rq->q;
+
+				if (!q->mq_ops && blk_queue_stopped(q)) {
+					spin_lock(q->queue_lock);
+					if (blk_queue_stopped(q))
+						blk_start_queue(q);
+					spin_unlock(q->queue_lock);
+				}
+			}
+		} while (entry);
+	}
+
+	return HRTIMER_NORESTART;
+}
+
+static void null_cmd_end_timer(struct nulln_cmd *cmd)
+{
+	struct completion_queue *cq = &per_cpu(null_comp_queues, get_cpu());
+
+	cmd->ll_list.next = NULL;
+	if (llist_add(&cmd->ll_list, &cq->list)) {
+		ktime_t kt = ktime_set(0, completion_nsec);
+
+		hrtimer_start(&cq->timer, kt, HRTIMER_MODE_REL_PINNED);
+	}
+
+	put_cpu();
+}
+
+static void null_softirq_done_fn(struct request *rq)
+{
+	blk_mq_end_request(rq, 0);
+}
+
+static inline void null_handle_cmd(struct nulln_cmd *cmd)
+{
+	/* Complete IO by inline, softirq or timer */
+	switch (irqmode) {
+	case NULL_IRQ_SOFTIRQ:
+	case NULL_IRQ_NONE:
+		blk_mq_complete_request(cmd->rq, cmd->rq->errors);
+		break;
+	case NULL_IRQ_TIMER:
+		null_cmd_end_timer(cmd);
+		break;
+	}
+}
+
+static int null_id(struct request_queue *q, struct nvm_id *id)
+{
+	sector_t size = gb * 1024 * 1024 * 1024ULL;
+	unsigned long per_chnl_size =
+				size / bs / num_channels;
+	struct nvm_id_group *grp;
+
+	id->ver_id = 0x1;
+	id->vmnt = 0;
+	id->cgrps = 1;
+	id->cap = 0x3;
+	id->dom = 0x1;
+	id->ppat = NVM_ADDRMODE_LINEAR;
+
+	grp = &id->groups[0];
+	grp->mtype = 0;
+	grp->fmtype = 1;
+	grp->num_ch = 1;
+	grp->num_lun = num_channels;
+	grp->num_pln = 1;
+	grp->num_blk = per_chnl_size / 256;
+	grp->num_pg = 256;
+	grp->fpg_sz = bs;
+	grp->csecs = bs;
+	grp->trdt = 25000;
+	grp->trdm = 25000;
+	grp->tprt = 500000;
+	grp->tprm = 500000;
+	grp->tbet = 1500000;
+	grp->tbem = 1500000;
+	grp->mpos = 0x010101; /* single plane rwe */
+	grp->cpar = hw_queue_depth;
+
+	return 0;
+}
+
+static void null_end_io(struct request *rq, int error)
+{
+	struct nvm_rq *rqd = rq->end_io_data;
+	struct nvm_dev *dev = rqd->dev;
+
+	dev->mt->end_io(rqd, error);
+
+	blk_put_request(rq);
+}
+
+static int null_submit_io(struct request_queue *q, struct nvm_rq *rqd)
+{
+	struct request *rq;
+	struct bio *bio = rqd->bio;
+
+	rq = blk_mq_alloc_request(q, bio_rw(bio), GFP_KERNEL, 0);
+	if (IS_ERR(rq))
+		return -ENOMEM;
+
+	rq->cmd_type = REQ_TYPE_DRV_PRIV;
+	rq->__sector = bio->bi_iter.bi_sector;
+	rq->ioprio = bio_prio(bio);
+
+	if (bio_has_data(bio))
+		rq->nr_phys_segments = bio_phys_segments(q, bio);
+
+	rq->__data_len = bio->bi_iter.bi_size;
+	rq->bio = rq->biotail = bio;
+
+	rq->end_io_data = rqd;
+
+	blk_execute_rq_nowait(q, NULL, rq, 0, null_end_io);
+
+	return 0;
+}
+
+static void *null_create_dma_pool(struct request_queue *q, char *name)
+{
+	mempool_t *virtmem_pool;
+
+	ppa_cache = kmem_cache_create(name, PAGE_SIZE, 0, 0, NULL);
+	if (!ppa_cache) {
+		pr_err("null_nvm: Unable to create kmem cache\n");
+		return NULL;
+	}
+
+	virtmem_pool = mempool_create_slab_pool(64, ppa_cache);
+	if (!virtmem_pool) {
+		pr_err("null_nvm: Unable to create virtual memory pool\n");
+		return NULL;
+	}
+
+	return virtmem_pool;
+}
+
+static void null_destroy_dma_pool(void *pool)
+{
+	mempool_t *virtmem_pool = pool;
+
+	mempool_destroy(virtmem_pool);
+}
+
+static void *null_dev_dma_alloc(struct request_queue *q, void *pool,
+				gfp_t mem_flags, dma_addr_t *dma_handler)
+{
+	return mempool_alloc(pool, mem_flags);
+}
+
+static void null_dev_dma_free(void *pool, void *entry, dma_addr_t dma_handler)
+{
+	mempool_free(entry, pool);
+}
+
+static struct nvm_dev_ops nulln_dev_ops = {
+	.identity		= null_id,
+	.submit_io		= null_submit_io,
+
+	.create_dma_pool	= null_create_dma_pool,
+	.destroy_dma_pool	= null_destroy_dma_pool,
+	.dev_dma_alloc		= null_dev_dma_alloc,
+	.dev_dma_free		= null_dev_dma_free,
+
+	/* Emulate nvme protocol */
+	.max_phys_sect		= 64,
+};
+
+static int null_queue_rq(struct blk_mq_hw_ctx *hctx,
+			 const struct blk_mq_queue_data *bd)
+{
+	struct nulln_cmd *cmd = blk_mq_rq_to_pdu(bd->rq);
+
+	cmd->rq = bd->rq;
+
+	blk_mq_start_request(bd->rq);
+
+	null_handle_cmd(cmd);
+	return BLK_MQ_RQ_QUEUE_OK;
+}
+
+static struct blk_mq_ops null_mq_ops = {
+	.queue_rq	= null_queue_rq,
+	.map_queue	= blk_mq_map_queue,
+	.complete	= null_softirq_done_fn,
+};
+
+static void null_del_dev(struct nulln *nulln)
+{
+	list_del_init(&nulln->list);
+
+	nvm_unregister(nulln->disk_name);
+
+	blk_cleanup_queue(nulln->q);
+	blk_mq_free_tag_set(&nulln->tag_set);
+	kfree(nulln);
+}
+
+static int null_add_dev(void)
+{
+	struct nulln *nulln;
+	int rv;
+
+	nulln = kzalloc_node(sizeof(*nulln), GFP_KERNEL, home_node);
+	if (!nulln) {
+		rv = -ENOMEM;
+		goto out;
+	}
+
+	if (use_per_node_hctx)
+		submit_queues = nr_online_nodes;
+
+	nulln->tag_set.ops = &null_mq_ops;
+	nulln->tag_set.nr_hw_queues = submit_queues;
+	nulln->tag_set.queue_depth = hw_queue_depth;
+	nulln->tag_set.numa_node = home_node;
+	nulln->tag_set.cmd_size = sizeof(struct nulln_cmd);
+	nulln->tag_set.driver_data = nulln;
+
+	rv = blk_mq_alloc_tag_set(&nulln->tag_set);
+	if (rv)
+		goto out_free_nulln;
+
+	nulln->q = blk_mq_init_queue(&nulln->tag_set);
+	if (IS_ERR(nulln->q)) {
+		rv = -ENOMEM;
+		goto out_cleanup_tags;
+	}
+
+	nulln->q->queuedata = nulln;
+	queue_flag_set_unlocked(QUEUE_FLAG_NONROT, nulln->q);
+	queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, nulln->q);
+
+	mutex_lock(&nulln_lock);
+	list_add_tail(&nulln->list, &nulln_list);
+	nulln->index = nulln_indexes++;
+	mutex_unlock(&nulln_lock);
+
+	blk_queue_logical_block_size(nulln->q, bs);
+	blk_queue_physical_block_size(nulln->q, bs);
+
+	sprintf(nulln->disk_name, "nulln%d", nulln->index);
+
+	rv = nvm_register(nulln->q, nulln->disk_name, &nulln_dev_ops);
+	if (rv)
+		goto out_cleanup_blk_queue;
+
+	return 0;
+
+out_cleanup_blk_queue:
+	blk_cleanup_queue(nulln->q);
+out_cleanup_tags:
+	blk_mq_free_tag_set(&nulln->tag_set);
+out_free_nulln:
+	kfree(nulln);
+out:
+	return rv;
+}
+
+static int __init null_init(void)
+{
+	unsigned int i;
+
+	if (bs > PAGE_SIZE) {
+		pr_warn("null_nvm: invalid block size\n");
+		pr_warn("null_nvm: defaults block size to %lu\n", PAGE_SIZE);
+		bs = PAGE_SIZE;
+	}
+
+	if (use_per_node_hctx) {
+		if (submit_queues < nr_online_nodes) {
+			pr_warn("null_nvm: submit_queues param is set to %u.",
+							nr_online_nodes);
+			submit_queues = nr_online_nodes;
+		}
+	} else if (submit_queues > nr_cpu_ids)
+		submit_queues = nr_cpu_ids;
+	else if (!submit_queues)
+		submit_queues = 1;
+
+	mutex_init(&nulln_lock);
+
+	/* Initialize a separate list for each CPU for issuing softirqs */
+	for_each_possible_cpu(i) {
+		struct completion_queue *cq = &per_cpu(null_comp_queues, i);
+
+		init_llist_head(&cq->list);
+
+		if (irqmode != NULL_IRQ_TIMER)
+			continue;
+
+		hrtimer_init(&cq->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+		cq->timer.function = null_cmd_timer_expired;
+	}
+
+	for (i = 0; i < nr_devices; i++) {
+		if (null_add_dev())
+			return -EINVAL;
+	}
+
+	pr_info("null_nvm: module loaded\n");
+	return 0;
+}
+
+static void __exit null_exit(void)
+{
+	struct nulln *nulln;
+
+	mutex_lock(&nulln_lock);
+	while (!list_empty(&nulln_list)) {
+		nulln = list_entry(nulln_list.next, struct nulln, list);
+		null_del_dev(nulln);
+	}
+	mutex_unlock(&nulln_lock);
+}
+
+module_init(null_init);
+module_exit(null_exit);
+
+MODULE_AUTHOR("Matias Bjorling <mb@lightnvm.io>");
+MODULE_LICENSE("GPL");
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web