Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1448386

[PATCH 58/58] staging: lustre: ptlrpc: Early replies need to honor at_max

From James Simmons <jsimmons@infradead.org>
Newsgroups linux.kernel
Subject [PATCH 58/58] staging: lustre: ptlrpc: Early replies need to honor at_max
Date 2016-07-22 06:20 +0200
Message-ID <rXA6m-Ky-7@gated-at.bofh.it> (permalink)
References <rXyQV-88M-3@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

From: Chris Horn <hornc@cray.com>

When determining whether an early reply can be sent the server will
calculate the new deadline based on an offset from the request
arrival time. However, when actually setting the new deadline
the server offsets the current time. This can result in deadlines
being extended more than at_max seconds past the request arrival
time. Instead, the server should offset the arrival time when updating
its request timeout.

When a client receives an early reply it doesn't know the server side
arrival time so we use the original sent time as an approximation.

Signed-off-by: Chris Horn <hornc@cray.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4578
Reviewed-on: http://review.whamcloud.com/9100
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/ptlrpc/client.c  |    8 +++++---
 drivers/staging/lustre/lustre/ptlrpc/import.c  |   11 +++++++----
 drivers/staging/lustre/lustre/ptlrpc/service.c |   18 +++++++++++++-----
 3 files changed, 25 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index d4463d7..ddb08ea 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -385,10 +385,12 @@ static int ptlrpc_at_recv_early_reply(struct ptlrpc_request *req)
 	spin_lock(&req->rq_lock);
 	olddl = req->rq_deadline;
 	/*
-	 * server assumes it now has rq_timeout from when it sent the
-	 * early reply, so client should give it at least that long.
+	 * server assumes it now has rq_timeout from when the request
+	 * arrived, so the client should give it at least that long.
+	 * since we don't know the arrival time we'll use the original
+	 * sent time
 	 */
-	req->rq_deadline = ktime_get_real_seconds() + req->rq_timeout +
+	req->rq_deadline = req->rq_sent + req->rq_timeout +
 			   ptlrpc_at_get_net_latency(req);
 
 	DEBUG_REQ(D_ADAPTTO, req,
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index 3292e6e..af8ffbc 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -1497,10 +1497,13 @@ EXPORT_SYMBOL(ptlrpc_disconnect_import);
 /* Adaptive Timeout utils */
 extern unsigned int at_min, at_max, at_history;
 
-/* Bin into timeslices using AT_BINS bins.
- * This gives us a max of the last binlimit*AT_BINS secs without the storage,
- * but still smoothing out a return to normalcy from a slow response.
- * (E.g. remember the maximum latency in each minute of the last 4 minutes.)
+/*
+ *Update at_current with the specified value (bounded by at_min and at_max),
+ * as well as the AT history "bins".
+ *  - Bin into timeslices using AT_BINS bins.
+ *  - This gives us a max of the last at_history seconds without the storage,
+ *    but still smoothing out a return to normalcy from a slow response.
+ *  - (E.g. remember the maximum latency in each minute of the last 4 minutes.)
  */
 int at_measured(struct adaptive_timeout *at, unsigned int val)
 {
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 4788c49..30d8b72 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -1005,13 +1005,16 @@ ptlrpc_at_remove_timed(struct ptlrpc_request *req)
 	array->paa_count--;
 }
 
+/*
+ * Attempt to extend the request deadline by sending an early reply to the
+ * client.
+ */
 static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
 {
 	struct ptlrpc_service_part *svcpt = req->rq_rqbd->rqbd_svcpt;
 	struct ptlrpc_request *reqcopy;
 	struct lustre_msg *reqmsg;
 	long olddl = req->rq_deadline - ktime_get_real_seconds();
-	time64_t newdl;
 	int rc;
 
 	/* deadline is when the client expects us to reply, margin is the
@@ -1039,8 +1042,13 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
 		return -ENOSYS;
 	}
 
-	/* Fake our processing time into the future to ask the clients
-	 * for some extra amount of time
+	/*
+	 * We want to extend the request deadline by at_extra seconds,
+	 * so we set our service estimate to reflect how much time has
+	 * passed since this request arrived plus an additional
+	 * at_extra seconds. The client will calculate the new deadline
+	 * based on this service estimate (plus some additional time to
+	 * account for network latency). See ptlrpc_at_recv_early_reply
 	 */
 	at_measured(&svcpt->scp_at_estimate, at_extra +
 		    ktime_get_real_seconds() - req->rq_arrival_time.tv_sec);
@@ -1056,7 +1064,6 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
 			  ktime_get_real_seconds());
 		return -ETIMEDOUT;
 	}
-	newdl = ktime_get_real_seconds() + at_get(&svcpt->scp_at_estimate);
 
 	reqcopy = ptlrpc_request_cache_alloc(GFP_NOFS);
 	if (!reqcopy)
@@ -1110,7 +1117,8 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
 
 	if (!rc) {
 		/* Adjust our own deadline to what we told the client */
-		req->rq_deadline = newdl;
+		req->rq_deadline = req->rq_arrival_time.tv_sec +
+				   at_get(&svcpt->scp_at_estimate);
 		req->rq_early_count++; /* number sent, server side */
 	} else {
 		DEBUG_REQ(D_ERROR, req, "Early reply send failed %d", rc);
-- 
1.7.1

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH 00/58] staging: lustre: bug fixes from lustre 2.5.[56-58] James Simmons <jsimmons@infradead.org> - 2016-07-22 05:00 +0200
  [PATCH 07/58] staging: lustre: llite: label the debug info James Simmons <jsimmons@infradead.org> - 2016-07-22 05:00 +0200
  [PATCH 02/58] staging: lustre: obd: rename struct lmv_stripe_md field mea to lmv James Simmons <jsimmons@infradead.org> - 2016-07-22 05:00 +0200
  [PATCH 05/58] staging: lustre: lmv: add new lmv structures James Simmons <jsimmons@infradead.org> - 2016-07-22 05:00 +0200
  [PATCH 10/58] staging: lustre: llite: reduce indent in ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 06/58] staging: lustre: mdc: handle IT_READDIR operations James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 08/58] staging: lustre: llite: pass struct md_op_data to ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 03/58] staging: lustre: ptlrpc: remove wirecheck for struct lmv_stripe_md James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 15/58] staging: lustre: llite: clarify some debug messages for statahead James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 04/58] staging: lustre: llite: cache directory striping information James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 14/58] staging: lustre: llite: rename some variables for ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:10 +0200
  [PATCH 16/58] staging: lustre: llite: remove code never called James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 23/58] staging: lustre: llite: change remove parameter to bool James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 24/58] staging: lustre: mdc: don't take rpc lock for readdir case James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 17/58] staging: lustre: llite: pass in __u64 pos for ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 21/58] staging: lustre: llite: style cleanup for llite_internal.h James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 11/58] staging: lustre: llite: set next only when needed in ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 13/58] staging: lustre: llite: change done flag in ll_dir_read to bool James Simmons <jsimmons@infradead.org> - 2016-07-22 05:20 +0200
  [PATCH 25/58] staging: lustre: lmv: remove unused lmv_get_mea function James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 20/58] staging: lustre: llite: remove comment from ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 19/58] staging: lustre: llite: add md_op_data parameter to ll_get_dir_page James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 26/58] staging: lustre: lmv: remove duplicate MAX_HASH_* James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 22/58] staging: lustre: llite: pass inode to ll_release_page James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 12/58] staging: lustre: llite: handle done flags differently in ll_dir_read James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 28/58] staging: lustre: lmv: remove lmv_get_easize James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 27/58] staging: lustre: lmv: change handling of lmv striping information James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 18/58] staging: lustre: llite: do post work for statahead in readdir case James Simmons <jsimmons@infradead.org> - 2016-07-22 05:30 +0200
  [PATCH 36/58] staging: lustre: lmv: fix issue found by Klocwork Insight tool James Simmons <jsimmons@infradead.org> - 2016-07-22 05:40 +0200
  [PATCH 29/58] staging: lustre: lmv: replace obd_free_memmd with lmv_free_memmd James Simmons <jsimmons@infradead.org> - 2016-07-22 05:40 +0200
  [PATCH 34/58] staging: lustre: obdclass: bug fixes for lu_device_type handling James Simmons <jsimmons@infradead.org> - 2016-07-22 05:40 +0200
  [PATCH 31/58] staging: lustre: llite: fix "getdirstripe" to show stripe info James Simmons <jsimmons@infradead.org> - 2016-07-22 05:40 +0200
  [PATCH 32/58] staging: lustre: delete striped directory James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 33/58] staging: lustre: obdclass: fix lmd_parse() to handle comma-separated NIDs James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 41/58] staging: lustre: mdc: fixup MDS_SWAP_LAYOUTS ELC handling James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 38/58] staging: lustre: llite: enable clients to inject error for lfsck James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 43/58] staging: lustre: const correct FID/OSTID/... helpers James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 40/58] staging: lustre: llite: a few fixes for migration. James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 35/58] staging: lustre: add ability to migrate inodes. James Simmons <jsimmons@infradead.org> - 2016-07-22 05:50 +0200
  [PATCH 39/58] staging: lustre: osc: allow to call brw_commit() multiple times James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 50/58] staging: lustre: lov: handle the case of stripe size is not power 2 James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 37/58] staging: lustre: libcfs: Only dump log once per sec. to avoid EEXIST James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 49/58] staging: lustre: llite: avoid a deadlock in page write James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 46/58] staging: lustre: lmv: access lum_stripe_offset as little endian James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 44/58] staging: lustre: use bool for several function in lustre_idl.h/lustre_fid.h James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 42/58] staging: lustre: don't need to const __u64 parameters for lustre_idl.h James Simmons <jsimmons@infradead.org> - 2016-07-22 06:00 +0200
  [PATCH 45/58] staging: lustre: simplify inline functions in lustre_fid.h James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 55/58] staging: lustre: ldlm: flock completion fixes. James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 47/58] staging: lustre: lmv: lookup remote migrating object in LMV James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 52/58] staging: lustre: lmv: rename request to preq in lmv_getattr_name() James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 51/58] staging: lustre: lmv: cleanup req in lmv_getattr_name() James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 48/58] staging: lustre: lmv: Ensure lmv_intent_lookup cleans up reqp James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 53/58] staging: lustre: obdclass: unified flow control interfaces James Simmons <jsimmons@infradead.org> - 2016-07-22 06:10 +0200
  [PATCH 57/58] staging: lustre: llite: add error handler in inode prepare phase James Simmons <jsimmons@infradead.org> - 2016-07-22 06:20 +0200
  [PATCH 58/58] staging: lustre: ptlrpc: Early replies need to honor at_max James Simmons <jsimmons@infradead.org> - 2016-07-22 06:20 +0200
  [PATCH 54/58] staging: lustre: reorder LOV_MAGIC_* definition James Simmons <jsimmons@infradead.org> - 2016-07-22 06:20 +0200
  [PATCH 56/58] staging: lustre: move ioctls to lustre_ioctl.h James Simmons <jsimmons@infradead.org> - 2016-07-22 06:20 +0200

csiph-web