Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1238434

[PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power

Show all headers | View raw

We received a bug report recently when DDW (64-bit direct DMA on Power)
is not enabled for NVMe devices. In that case, we fall back to 32-bit
DMA via the IOMMU, which is always done via 4K TCEs (Translation Control
Entries).

The NVMe device driver, though, assumes that the DMA alignment for the
PRP entries will match the device's page size, and that the DMA aligment
matches the kernel's page aligment. On Power, the the IOMMU page size,
as mentioned above, can be 4K, while the device can have a page size of
8K, while the kernel has a page size of 64K. This eventually trips the
BUG_ON in nvme_setup_prps(), as we have a 'dma_len' that is a multiple
of 4K but not 8K (e.g., 0xF000).

In this particular case, and generally, we want to use the IOMMU's page
size for the default device page size, rather than the kernel's page
size.

With this patch, a NVMe device survives our internal hardware
exerciser; the kernel BUGs within a few seconds without the patch.

Signed-off-by: Nishanth Aravamudan <nacc@linux.vnet.ibm.com>

diff --git a/drivers/block/nvme-core.c b/drivers/block/nvme-core.c
index 7920c27..969a95e 100644
--- a/drivers/block/nvme-core.c
+++ b/drivers/block/nvme-core.c
@@ -42,6 +42,7 @@
 #include <linux/types.h>
 #include <scsi/sg.h>
 #include <asm-generic/io-64-nonatomic-lo-hi.h>
+#include <asm/iommu.h>
 
 #define NVME_MINORS		(1U << MINORBITS)
 #define NVME_Q_DEPTH		1024
@@ -1680,6 +1681,11 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 	unsigned page_shift = PAGE_SHIFT;
 	unsigned dev_page_min = NVME_CAP_MPSMIN(cap) + 12;
 	unsigned dev_page_max = NVME_CAP_MPSMAX(cap) + 12;
+#ifdef CONFIG_PPC64
+	struct iommu_table *tbl = get_iommu_table_base(dev->dev);
+	if (tbl)
+		page_shift = IOMMU_PAGE_SHIFT(tbl);
+#endif
 
 	if (page_shift < dev_page_min) {
 		dev_err(dev->dev,

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 0/2] Fix NVMe driver support on Power with 32-bit DMA Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 19:20 +0200
  [PATCH 1/2] powerpc/iommu: expose IOMMU page shift Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 19:20 +0200
    [PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 19:30 +0200
      Re: [PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power Christoph Hellwig <hch@infradead.org> - 2015-10-02 19:30 +0200
        Re: [PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 19:50 +0200
          Re: [PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power Christoph Hellwig <hch@infradead.org> - 2015-10-02 19:50 +0200
  [PATCH 2/5 v2] powerpc/dma-mapping: override dma_get_page_shift Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:20 +0200
    [PATCH 3/5 v2] powerpc/dma: implement per-platform dma_get_page_shift Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:20 +0200
      [PATCH 4/5 v2] pseries/iommu: implement DDW-aware dma_get_page_shift Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:30 +0200
        [PATCH 5/5 v2] drivers/nvme: default to the IOMMU page size Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:40 +0200
        Re: [PATCH 4/5 v2] pseries/iommu: implement DDW-aware  dma_get_page_shift kbuild test robot <lkp@intel.com> - 2015-10-02 22:40 +0200
  [PATCH 1/5 v2] dma-mapping: add generic dma_get_page_shift API Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:20 +0200
  [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 22:20 +0200
    Re: [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-02 23:10 +0200
      Re: [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Benjamin Herrenschmidt <benh@kernel.crashing.org> - 2015-10-02 23:40 +0200
        Re: [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Nishanth Aravamudan <nacc@linux.vnet.ibm.com> - 2015-10-03 00:00 +0200
    Re: [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Benjamin Herrenschmidt <benh@kernel.crashing.org> - 2015-10-02 23:30 +0200
      Re: [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Christoph Hellwig <hch@infradead.org> - 2015-10-03 10:30 +0200

csiph-web