Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1236304 > unrolled thread

[PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune Sowmini Varadhan <sowmini.varadhan@oracle.com> - 2015-09-30 15:50 +0200
    Re: [PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in  rds_tcp_tune santosh shilimkar <santosh.shilimkar@oracle.com> - 2015-09-30 18:00 +0200

#1236304 — [PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune

Using the value of RDS_TCP_DEFAULT_BUFSIZE (128K)
clobbers efficient use of TSO because it inflates the size_goal
that is computed in tcp_sendmsg/tcp_sendpage and skews packet
latency, and the default values for these parameters actually
results in significantly better performance.

In request-response tests using rds-stress with a packet size of
100K with 16 threads (test parameters -q 100000 -a 256 -t16 -d16)
between a single pair of IP addresses achieves a throughput of
6-8 Gbps. Without this patch, throughput maxes at 2-3 Gbps under
equivalent conditions on these platforms.

Signed-off-by: Sowmini Varadhan <sowmini.varadhan@oracle.com>
---
 net/rds/tcp.c |   16 ++++------------
 1 files changed, 4 insertions(+), 12 deletions(-)

diff --git a/net/rds/tcp.c b/net/rds/tcp.c
index c42b60b..9d6ddba 100644
--- a/net/rds/tcp.c
+++ b/net/rds/tcp.c
@@ -67,21 +67,13 @@ void rds_tcp_nonagle(struct socket *sock)
 	set_fs(oldfs);
 }
 
+/* All module specific customizations to the RDS-TCP socket should be done in
+ * rds_tcp_tune() and applied after socket creation. In general these
+ * customizations should be tunable via module_param()
+ */
 void rds_tcp_tune(struct socket *sock)
 {
-	struct sock *sk = sock->sk;
-
 	rds_tcp_nonagle(sock);
-
-	/*
-	 * We're trying to saturate gigabit with the default,
-	 * see svc_sock_setbufsize().
-	 */
-	lock_sock(sk);
-	sk->sk_sndbuf = RDS_TCP_DEFAULT_BUFSIZE;
-	sk->sk_rcvbuf = RDS_TCP_DEFAULT_BUFSIZE;
-	sk->sk_userlocks |= SOCK_SNDBUF_LOCK|SOCK_RCVBUF_LOCK;
-	release_sock(sk);
 }
 
 u32 rds_tcp_snd_nxt(struct rds_tcp_connection *tc)
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1236466 — Re: [PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune

On 9/30/2015 6:45 AM, Sowmini Varadhan wrote:
> Using the value of RDS_TCP_DEFAULT_BUFSIZE (128K)
> clobbers efficient use of TSO because it inflates the size_goal
> that is computed in tcp_sendmsg/tcp_sendpage and skews packet
> latency, and the default values for these parameters actually
> results in significantly better performance.
>
> In request-response tests using rds-stress with a packet size of
> 100K with 16 threads (test parameters -q 100000 -a 256 -t16 -d16)
> between a single pair of IP addresses achieves a throughput of
> 6-8 Gbps. Without this patch, throughput maxes at 2-3 Gbps under
> equivalent conditions on these platforms.
>
> Signed-off-by: Sowmini Varadhan <sowmini.varadhan@oracle.com>
> ---
>   net/rds/tcp.c |   16 ++++------------
>   1 files changed, 4 insertions(+), 12 deletions(-)
>
> diff --git a/net/rds/tcp.c b/net/rds/tcp.c
> index c42b60b..9d6ddba 100644
> --- a/net/rds/tcp.c
> +++ b/net/rds/tcp.c
> @@ -67,21 +67,13 @@ void rds_tcp_nonagle(struct socket *sock)
>   	set_fs(oldfs);
>   }
>
> +/* All module specific customizations to the RDS-TCP socket should be done in
> + * rds_tcp_tune() and applied after socket creation. In general these
> + * customizations should be tunable via module_param()
> + */
>   void rds_tcp_tune(struct socket *sock)
>   {
> -	struct sock *sk = sock->sk;
> -
>   	rds_tcp_nonagle(sock);
> -
> -	/*
> -	 * We're trying to saturate gigabit with the default,
> -	 * see svc_sock_setbufsize().
> -	 */
> -	lock_sock(sk);
> -	sk->sk_sndbuf = RDS_TCP_DEFAULT_BUFSIZE;
> -	sk->sk_rcvbuf = RDS_TCP_DEFAULT_BUFSIZE;
> -	sk->sk_userlocks |= SOCK_SNDBUF_LOCK|SOCK_RCVBUF_LOCK;
> -	release_sock(sk);
>   }
>
>   u32 rds_tcp_snd_nxt(struct rds_tcp_connection *tc)
>
We should at least start with sndbuf/rcvbuf parameters.
Nice work. Almost ~3X lift in RDS TCP performance.

Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web