[PATCH 4.4 060/112] sunrpc: fix write space race causing stalls

From	Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Newsgroups	linux.kernel
Subject	[PATCH 4.4 060/112] sunrpc: fix write space race causing stalls
Date	2016-10-26 15:00 +0200
Message-ID	<swvYg-3Qk-87@gated-at.bofh.it> (permalink)
References	<swvER-3Eq-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

4.4-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Vrabel <david.vrabel@citrix.com>

commit d48f9ce73c997573e1b512893fa6eddf353a6f69 upstream.

Write space becoming available may race with putting the task to sleep
in xprt_wait_for_buffer_space().  The existing mechanism to avoid the
race does not work.

This (edited) partial trace illustrates the problem:

   [1] rpc_task_run_action: task:43546@5 ... action=call_transmit
   [2] xs_write_space <-xs_tcp_write_space
   [3] xprt_write_space <-xs_write_space
   [4] rpc_task_sleep: task:43546@5 ...
   [5] xs_write_space <-xs_tcp_write_space

[1] Task 43546 runs but is out of write space.

[2] Space becomes available, xs_write_space() clears the
    SOCKWQ_ASYNC_NOSPACE bit.

[3] xprt_write_space() attemts to wake xprt->snd_task (== 43546), but
    this has not yet been queued and the wake up is lost.

[4] xs_nospace() is called which calls xprt_wait_for_buffer_space()
    which queues task 43546.

[5] The call to sk->sk_write_space() at the end of xs_nospace() (which
    is supposed to handle the above race) does not call
    xprt_write_space() as the SOCKWQ_ASYNC_NOSPACE bit is clear and
    thus the task is not woken.

Fix the race by resetting the SOCKWQ_ASYNC_NOSPACE bit in xs_nospace()
so the second call to sk->sk_write_space() calls xprt_write_space().

Suggested-by: Trond Myklebust <trondmy@primarydata.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 net/sunrpc/xprtsock.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -474,7 +474,16 @@ static int xs_nospace(struct rpc_task *t
 	spin_unlock_bh(&xprt->transport_lock);
 
 	/* Race breaker in case memory is freed before above code is called */
-	sk->sk_write_space(sk);
+	if (ret == -EAGAIN) {
+		struct socket_wq *wq;
+
+		rcu_read_lock();
+		wq = rcu_dereference(sk->sk_wq);
+		set_bit(SOCKWQ_ASYNC_NOSPACE, &wq->flags);
+		rcu_read_unlock();
+
+		sk->sk_write_space(sk);
+	}
 	return ret;
 }

Thread

[PATCH 4.4 000/112] 4.4.28-stable review Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 14:50 +0200
  [PATCH 4.4 061/112] NFSv4: Dont report revoked delegations as valid in nfs_have_delegation() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 084/112] irqchip/gicv3: Handle loop timeout proper Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 028/112] powerpc/powernv: Use CPU-endian hub diag-data type in pnv_eeh_get_and_dump_hub_diag() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 051/112] ALSA: hda - Fix a failure of micmute led when having multi adcs Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 074/112] Clarify locking of cifs file and tcon structures and make more granular Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 027/112] powerpc/powernv: Pass CPU-endian PE number to opal_pci_eeh_freeze_clear() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 071/112] fs/super.c: fix race between freeze_super() and thaw_super() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 077/112] SMB3: GUIDs should be constructed as random but valid uuids Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 019/112] perf intel-pt: Fix snapshot overlap detection decoder errors Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 024/112] dm mpath: check if paths request_queue is dying in activate_path() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 072/112] cifs: Limit the overall credit acquired Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 070/112] arc: dont leak bits of kernel stack into coredump Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 057/112] Input: i8042 - skip selftest on ASUS laptops Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 054/112] lib: move strtobool() to kstrtobool() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 037/112] zfcp: restore: Dont use 0 to indicate invalid LUN in rec trace Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 058/112] Input: elantech - force needed quirks on Fujitsu H760 Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 055/112] lib: update single-char callers of strtobool() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 060/112] sunrpc: fix write space race causing stalls Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 053/112] MIPS: ptrace: Fix regs_return_value for kernel context Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 056/112] lib: add "on"/"off" support to kstrtobool Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:00 +0200
  [PATCH 4.4 010/112] platform: dont return 0 from platform_get_irq[_byname]() on error Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 047/112] [media] mb86a20s: fix the locking logic Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 022/112] dm: mark request_queue dead before destroying the DM device Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 021/112] perf intel-pt: Fix MTC timestamp calculation for large MTC periods Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 043/112] scsi: zfcp: spin_lock_irqsave() is not nestable Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 017/112] pstore/ram: Use memcpy_toio instead of memcpy Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 041/112] zfcp: fix payload trace length for SAN request&response Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 050/112] [media] cx231xx: fix GPIOs for Pixelview SBTVD hybrid Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 049/112] [media] cx231xx: dont return error on success Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 044/112] fbdev/efifb: Fix 16 color palette entry calculation Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 048/112] [media] mb86a20s: fix demod settings Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 015/112] pstore/ramoops: fixup driver removal Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 032/112] ubi: Deal with interrupted erasures in WL Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 046/112] ovl: copy_up_xattr(): use strnlen Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 036/112] zfcp: retain trace level for SCSI and HBA FSF response records Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 013/112] parisc: Fix kernel memory layout regarding position of __gp Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 011/112] cpufreq: intel_pstate: Fix unsafe HWP MSR access Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 038/112] zfcp: trace on request for open and close of WKA port Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 035/112] zfcp: close window with unblocked rport during rport gone Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 034/112] zfcp: fix ELS/GS request&response length for hardware data router Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 042/112] zfcp: trace full payload of all SAN records (req,resp,iels) Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 040/112] zfcp: fix D_ID field with actual value on tracing SAN responses Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 045/112] ovl: Fix info leak in ovl_lookup_temp() Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  [PATCH 4.4 033/112] zfcp: fix fc_host port_type with NPIV Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-10-26 15:10 +0200
  Re: [PATCH 4.4 000/112] 4.4.28-stable review Shuah Khan <shuah.kh@samsung.com> - 2016-10-26 20:50 +0200
  Re: [PATCH 4.4 000/112] 4.4.28-stable review Guenter Roeck <linux@roeck-us.net> - 2016-10-26 23:50 +0200

csiph-web