Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1242969

[PATCH 3.2 055/107] IB/qib: Change lkey table allocation to support more MRs

From Ben Hutchings <ben@decadent.org.uk>
Newsgroups linux.kernel
Subject [PATCH 3.2 055/107] IB/qib: Change lkey table allocation to support more MRs
Date 2015-10-09 03:00 +0200
Message-ID <qhucq-3d3-21@gated-at.bofh.it> (permalink)
References <qhtzH-2tg-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

3.2.72-rc1 review patch.  If anyone has any objections, please let me know.

------------------

From: Mike Marciniszyn <mike.marciniszyn@intel.com>

commit d6f1c17e162b2a11e708f28fa93f2f79c164b442 upstream.

The lkey table is allocated with with a get_user_pages() with an
order based on a number of index bits from a module parameter.

The underlying kernel code cannot allocate that many contiguous pages.

There is no reason the underlying memory needs to be physically
contiguous.

This patch:
- switches the allocation/deallocation to vmalloc/vfree
- caps the number of bits to 23 to insure at least 1 generation bit
  o this matches the module parameter description

Reviewed-by: Vinit Agnihotri <vinit.abhay.agnihotri@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
[bwh: Backported to 3.2:
 - Adjust context
 - Add definition of qib_dev_warn(), added upstream by commit ddb887658970
   ("IB/qib: Convert opcode counters to per-context")]
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
---
 drivers/infiniband/hw/qib/qib_keys.c  |  4 ++++
 drivers/infiniband/hw/qib/qib_verbs.c | 14 ++++++++++----
 drivers/infiniband/hw/qib/qib_verbs.h |  2 ++
 3 files changed, 16 insertions(+), 4 deletions(-)

--- a/drivers/infiniband/hw/qib/qib_keys.c
+++ b/drivers/infiniband/hw/qib/qib_keys.c
@@ -69,6 +69,10 @@ int qib_alloc_lkey(struct qib_lkey_table
 	 * unrestricted LKEY.
 	 */
 	rkt->gen++;
+	/*
+	 * bits are capped in qib_verbs.c to insure enough bits
+	 * for generation number
+	 */
 	mr->lkey = (r << (32 - ib_qib_lkey_table_size)) |
 		((((1 << (24 - ib_qib_lkey_table_size)) - 1) & rkt->gen)
 		 << 8);
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -40,6 +40,7 @@
 #include <linux/rculist.h>
 #include <linux/mm.h>
 #include <linux/random.h>
+#include <linux/vmalloc.h>
 
 #include "qib.h"
 #include "qib_common.h"
@@ -2035,10 +2036,16 @@ int qib_register_ib_device(struct qib_de
 	 * the LKEY).  The remaining bits act as a generation number or tag.
 	 */
 	spin_lock_init(&dev->lk_table.lock);
+	/* insure generation is at least 4 bits see keys.c */
+	if (ib_qib_lkey_table_size > MAX_LKEY_TABLE_BITS) {
+		qib_dev_warn(dd, "lkey bits %u too large, reduced to %u\n",
+			ib_qib_lkey_table_size, MAX_LKEY_TABLE_BITS);
+		ib_qib_lkey_table_size = MAX_LKEY_TABLE_BITS;
+	}
 	dev->lk_table.max = 1 << ib_qib_lkey_table_size;
 	lk_tab_size = dev->lk_table.max * sizeof(*dev->lk_table.table);
 	dev->lk_table.table = (struct qib_mregion **)
-		__get_free_pages(GFP_KERNEL, get_order(lk_tab_size));
+		vmalloc(lk_tab_size);
 	if (dev->lk_table.table == NULL) {
 		ret = -ENOMEM;
 		goto err_lk;
@@ -2208,7 +2215,7 @@ err_tx:
 					sizeof(struct qib_pio_header),
 				  dev->pio_hdrs, dev->pio_hdrs_phys);
 err_hdrs:
-	free_pages((unsigned long) dev->lk_table.table, get_order(lk_tab_size));
+	vfree(dev->lk_table.table);
 err_lk:
 	kfree(dev->qp_table);
 err_qpt:
@@ -2262,7 +2269,6 @@ void qib_unregister_ib_device(struct qib
 					sizeof(struct qib_pio_header),
 				  dev->pio_hdrs, dev->pio_hdrs_phys);
 	lk_tab_size = dev->lk_table.max * sizeof(*dev->lk_table.table);
-	free_pages((unsigned long) dev->lk_table.table,
-		   get_order(lk_tab_size));
+	vfree(dev->lk_table.table);
 	kfree(dev->qp_table);
 }
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -622,6 +622,8 @@ struct qib_qpn_table {
 	struct qpn_map map[QPNMAP_ENTRIES];
 };
 
+#define MAX_LKEY_TABLE_BITS 23
+
 struct qib_lkey_table {
 	spinlock_t lock; /* protect changes in this struct */
 	u32 next;               /* next unused index (speeds search) */
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1421,6 +1421,10 @@ extern struct mutex qib_mutex;
 			qib_get_unit_name((dd)->unit), ##__VA_ARGS__); \
 	} while (0)
 
+#define qib_dev_warn(dd, fmt, ...) \
+	dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \
+		qib_get_unit_name((dd)->unit), ##__VA_ARGS__)
+
 #define qib_dev_porterr(dd, port, fmt, ...) \
 	do { \
 		dev_err(&(dd)->pcidev->dev, "%s: IB%u:%u " fmt, \

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 3.2 000/107] 3.2.72-rc1 review Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 072/107] USB: option: add ZTE PIDs Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 069/107] hfs,hfsplus: cache pages correctly between  bnode_create and bnode_free Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 015/107] ocfs2: fix BUG in ocfs2_downconvert_thread_do_work() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 049/107] DRM - radeon: Don't link train DisplayPort on  HPD until we get the dpcd Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 098/107] ipv6: lock socket in ip6_datagram_connect() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 024/107] libfc: Fix fc_fcp_cleanup_each_cmd() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:30 +0200
  [PATCH 3.2 094/107] Initialize msg/shm IPC objects before doing  ipc_addid() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 060/107] drm/i915: Always mark the object as dirty  when used by the GPU Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 047/107] eCryptfs: Invalidate dcache entries when  lower i_nlink is zero Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 009/107] target: REPORT LUNS should return LUN 0 even  for dynamic ACLs Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 095/107] net/tipc: initialize security state for new  connection  socket Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 018/107] x86/ldt: Make modify_ldt synchronous Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 036/107] PCI: Add VPD function 0 quirk for Intel  Ethernet devices Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 016/107] net: Clone skb before setting peeked flag Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 026/107] x86/ldt: Further fix FPU emulation Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 029/107] sparc64: Fix userspace FPU register corruptions. Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 023/107] libiscsi: Fix host busy blocking during  connection teardown Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 019/107] x86/ldt: Correct LDT access in single  stepping logic Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 017/107] net: Fix skb_set_peeked use-after-free bug Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 005/107] crypto: ixp4xx - Remove bogus BUG_ON on  scattered dst buffer Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 033/107] PCI: Fix TI816X class code quirk Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 051/107] rtlwifi: rtl8192cu: Add new device ID Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 032/107] [media] rc-core: fix remove uevent generation Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 068/107] powerpc/MSI: Fix race condition in tearing  down MSI interrupts Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 037/107] usb: gadget: m66592-udc: forever loop in  set_feature() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 054/107] xfs: return errors from partial I/O failures  to files Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 103/107] ipv6: prevent fib6_run_gc() contention Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 044/107] serial: 8250: bind to ALi Fast Infrared  Controller (ALI5123) Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 070/107] hfs: fix B-tree corruption after insertion at  position 0 Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 050/107] rtlwifi: rtl8192cu: Add new device ID Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 048/107] xfs: Fix xfs_attr_leafblock definition Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 025/107] ipc,sem: fix use after free on IPC_RMID after  a task using same semaphore set exits Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 007/107] target/iscsi: Fix double free of a TUR  followed by a solicited NOPOUT Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:40 +0200
  [PATCH 3.2 081/107] usb: Use the USB_SS_MULT() macro to get the  burst multiplier. Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 079/107] s390/compat: correct uc_sigmask of the compat  signal frame Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 091/107] virtio-net: drop NETIF_F_FRAGLIST Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 053/107] drivercore: Fix unregistration path of  platform devices Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 006/107] USB: sierra: add 1199:68AB device ID Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 059/107] spi: spi-pxa2xx: Check status register to  determine if SSSR_TINT is disabled Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 002/107] pktgen: Require CONFIG_INET due to use of  IPv4 checksum function Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 085/107] cifs: use server timestamp for ntlmv2  authentication Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 066/107] ARM: 8429/1: disable GCC SRA optimization Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 061/107] Add radeon suspend/resume quirk for HP Compaq  dc5750. Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 046/107] USB: ftdi_sio: Added custom PID for  CustomWare products Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 013/107] perf: Fix fasync handling on inherited events Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 074/107] btrfs: skip waiting on ordered range for  special files Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 052/107] of/address: Don't loop forever in  of_find_matching_node_by_address(). Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 042/107] NFSv4: don't set SETATTR for O_RDONLY|O_EXCL Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 093/107] ipc/sem.c: fully initialize sem_array before  making it visible Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 039/107] auxdisplay: ks0108: fix refcount Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 083/107] usb: xhci: Clear XHCI_STATE_DYING on start Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 062/107] IB/uverbs: reject invalid or unknown opcodes Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 058/107] IB/uverbs: Fix race between ib_uverbs_open  and remove_one Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 102/107] perf tools: Fix build with perl 5.18 Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 075/107] ARM: 7880/1: Clear the IT state independent  of the Thumb-2 mode Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 105/107] parisc: Filter out spurious interrupts in  PA-RISC irq handler Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 100/107] net/ipv6: Correct PIM6 mrt_lock handling Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 030/107] dcache: Handle escaped paths in prepend_path Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 041/107] windfarm: decrement client count when  unregistering Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 076/107] ARM: fix Thumb2 signal handling when ARMv6 is  enabled Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 057/107] IB/mlx4: Use correct SL on AH query under RoCE Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 090/107] ipv6: addrconf: validate new MTU before  applying it Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 010/107] MIPS: Fix sched_getaffinity with MT FPAFF enabled Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 02:50 +0200
  [PATCH 3.2 063/107] Input: evdev - do not report errors form flush() Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 078/107] ASoC: fix broken pxa SoC support Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 064/107] crypto: ghash-clmulni: specify context size  for ghash async algorithm Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 065/107] fs: create and use seq_show_option for escaping Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 084/107] xhci: change xhci 1.0 only restrictions to  support xhci 1.1 Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 071/107] perf header: Fixup reading of HEADER_NRCPUS  feature Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 104/107] ipv6: update ip6_rt_last_gc every time GC is run Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 055/107] IB/qib: Change lkey table allocation to  support more MRs Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 073/107] Btrfs: fix read corruption of compressed and  shared extents Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  Re: [PATCH 3.2 000/107] 3.2.72-rc1 review Guenter Roeck <linux@roeck-us.net> - 2015-10-09 03:00 +0200
    Re: [PATCH 3.2 000/107] 3.2.72-rc1 review Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:20 +0200
  [PATCH 3.2 077/107] x86/platform: Fix Geode LX timekeeping in the  generic x86 build Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 087/107] ocfs2/dlm: fix deadlock when dispatch assert  master Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 056/107] SUNRPC: xs_reset_transport must mark the  connection as disconnected Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200
  [PATCH 3.2 082/107] xhci: give command abortion one more chance  before killing xhci Ben Hutchings <ben@decadent.org.uk> - 2015-10-09 03:00 +0200

csiph-web