Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1340035

[PATCH 18/24] staging: lustre: remove messages from lazy portal on NI shutdown

Show all headers | View raw

From: Amir Shehata <amir.shehata@intel.com>

When shutting down an NI in a busy system, some messages received
on this NI, might be on the lazy portal.  They would have grabbed
a ref count on the NI.  Therefore NI will not be removed until
messages are processed.

In order to avoid this scenario, when an NI is shutdown go through
all messages queued on the lazy portal and drop messages for the
NI being shutdown

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6040
Reviewed-on: http://review.whamcloud.com/13836
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
---
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |    1 +
 drivers/staging/lustre/lnet/lnet/api-ni.c          |    6 ++
 drivers/staging/lustre/lnet/lnet/lib-ptl.c         |   54 +++++++++++++-------
 3 files changed, 43 insertions(+), 18 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 0928bc9..a5f1aec 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -482,6 +482,7 @@ int lnet_dyn_add_ni(lnet_pid_t requested_pid, char *nets,
 		    __s32 peer_timeout, __s32 peer_cr, __s32 peer_buf_cr,
 		    __s32 credits);
 int lnet_dyn_del_ni(__u32 net);
+int lnet_clear_lazy_portal(struct lnet_ni *ni, int portal, char *reason);
 
 int lnet_islocalnid(lnet_nid_t nid);
 int lnet_islocalnet(__u32 net);
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 0c7db19..3ecc96a 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1196,10 +1196,16 @@ lnet_shutdown_lndnis(void)
 static void
 lnet_shutdown_lndni(struct lnet_ni *ni)
 {
+	int i;
+
 	lnet_net_lock(LNET_LOCK_EX);
 	lnet_ni_unlink_locked(ni);
 	lnet_net_unlock(LNET_LOCK_EX);
 
+	/* clear messages for this NI on the lazy portal */
+	for (i = 0; i < the_lnet.ln_nportals; i++)
+		lnet_clear_lazy_portal(ni, i, "Shutting down NI");
+
 	/* Do peer table cleanup for this ni */
 	lnet_peer_tables_cleanup(ni);
 
diff --git a/drivers/staging/lustre/lnet/lnet/lib-ptl.c b/drivers/staging/lustre/lnet/lnet/lib-ptl.c
index 0cdeea9..5a9ab87 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-ptl.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-ptl.c
@@ -902,17 +902,8 @@ LNetSetLazyPortal(int portal)
 }
 EXPORT_SYMBOL(LNetSetLazyPortal);
 
-/**
- * Turn off the lazy portal attribute. Delayed requests on the portal,
- * if any, will be all dropped when this function returns.
- *
- * \param portal Index of the portal to disable the lazy attribute on.
- *
- * \retval 0       On success.
- * \retval -EINVAL If \a portal is not a valid index.
- */
 int
-LNetClearLazyPortal(int portal)
+lnet_clear_lazy_portal(struct lnet_ni *ni, int portal, char *reason)
 {
 	struct lnet_portal *ptl;
 	LIST_HEAD(zombies);
@@ -931,21 +922,48 @@ LNetClearLazyPortal(int portal)
 		return 0;
 	}
 
-	if (the_lnet.ln_shutdown)
-		CWARN("Active lazy portal %d on exit\n", portal);
-	else
-		CDEBUG(D_NET, "clearing portal %d lazy\n", portal);
+	if (ni) {
+		struct lnet_msg *msg, *tmp;
 
-	/* grab all the blocked messages atomically */
-	list_splice_init(&ptl->ptl_msg_delayed, &zombies);
+		/* grab all messages which are on the NI passed in */
+		list_for_each_entry_safe(msg, tmp, &ptl->ptl_msg_delayed,
+					 msg_list) {
+			if (msg->msg_rxpeer->lp_ni == ni)
+				list_move(&msg->msg_list, &zombies);
+		}
+	} else {
+		if (the_lnet.ln_shutdown)
+			CWARN("Active lazy portal %d on exit\n", portal);
+		else
+			CDEBUG(D_NET, "clearing portal %d lazy\n", portal);
+
+		/* grab all the blocked messages atomically */
+		list_splice_init(&ptl->ptl_msg_delayed, &zombies);
 
-	lnet_ptl_unsetopt(ptl, LNET_PTL_LAZY);
+		lnet_ptl_unsetopt(ptl, LNET_PTL_LAZY);
+	}
 
 	lnet_ptl_unlock(ptl);
 	lnet_res_unlock(LNET_LOCK_EX);
 
-	lnet_drop_delayed_msg_list(&zombies, "Clearing lazy portal attr");
+	lnet_drop_delayed_msg_list(&zombies, reason);
 
 	return 0;
 }
+
+/**
+ * Turn off the lazy portal attribute. Delayed requests on the portal,
+ * if any, will be all dropped when this function returns.
+ *
+ * \param portal Index of the portal to disable the lazy attribute on.
+ *
+ * \retval 0       On success.
+ * \retval -EINVAL If \a portal is not a valid index.
+ */
+int
+LNetClearLazyPortal(int portal)
+{
+	return lnet_clear_lazy_portal(NULL, portal,
+				      "Clearing lazy portal attr");
+}
 EXPORT_SYMBOL(LNetClearLazyPortal);
-- 
1.7.1

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 00/24] Second batch of LNet updates James Simmons <jsimmons@infradead.org> - 2016-02-22 23:30 +0100
  [PATCH 17/24] staging: lustre: prevent assert on LNet module unload James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 24/24] staging: lustre: Remove LASSERTS from router checker James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 19/24] staging: lustre: remove unnecessary EXPORT_SYMBOL from lnet layer James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 01/24] staging: lustre: Dynamic LNet Configuration (DLC) IOCTL changes James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 07/24] staging: lustre: improve LNet clean up code and API James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 23/24] staging: lustre: Use lnet_is_route_alive for router aliveness James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 22/24] staging: lustre: Allocate the correct number of rtr buffers James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 20/24] staging: lustre: avoid race during lnet acceptor thread termination James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 11/24] staging: lustre: startup lnet acceptor thread dynamically James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 09/24] staging: lustre: make some lnet functions static James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 04/24] staging: lustre: DLC user/kernel space glue code James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 13/24] staging: lustre: return -EEXIST if NI is not unique James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 03/24] staging: lustre: fix crash due to NULL networks string James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 21/24] staging: lustre: use sock.h in only acceptor.c James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 18/24] staging: lustre: remove messages from lazy portal on NI shutdown James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 05/24] staging: lustre: make local functions static for LNet ni James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 06/24] staging: lustre: remove LUSTRE_{,SRV_}LNET_PID James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 16/24] staging: lustre: assume a kernel build James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 10/24] staging: lustre: missed a few cases of using NULL instead of 0 James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 14/24] staging: lustre: handle lnet_check_routes() errors James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 08/24] staging: lustre: return appropriate errno when adding route James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100
  [PATCH 12/24] staging: lustre: reject invalid net configuration for lnet James Simmons <jsimmons@infradead.org> - 2016-02-22 23:40 +0100

csiph-web