Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1642952 > unrolled thread

[PATCH net-next V5 0/9] vhost_net rx batch dequeuing

Back to article view | Back to linux.kernel

Contents

  [PATCH net-next V5 0/9] vhost_net rx batch dequeuing Jason Wang <jasowang@redhat.com> - 2017-05-17 06:20 +0200
    [PATCH net-next V5 7/9] tun: support receiving skb through msg_control Jason Wang <jasowang@redhat.com> - 2017-05-17 06:20 +0200
    [PATCH net-next V5 1/9] ptr_ring: add ptr_ring_unconsume Jason Wang <jasowang@redhat.com> - 2017-05-17 06:20 +0200
    [PATCH net-next V5 8/9] tap: support receiving skb from msg_control Jason Wang <jasowang@redhat.com> - 2017-05-17 06:20 +0200
    [PATCH net-next V5 2/9] skb_array: introduce skb_array_unconsume Jason Wang <jasowang@redhat.com> - 2017-05-17 06:20 +0200
    Re: [PATCH net-next V5 0/9] vhost_net rx batch dequeuing "Michael S. Tsirkin" <mst@redhat.com> - 2017-05-17 23:10 +0200
      Re: [PATCH net-next V5 0/9] vhost_net rx batch dequeuing Jason Wang <jasowang@redhat.com> - 2017-05-19 08:30 +0200
        Re: [PATCH net-next V5 0/9] vhost_net rx batch dequeuing "Michael S. Tsirkin" <mst@redhat.com> - 2017-05-19 18:40 +0200
    Re: [PATCH net-next V5 0/9] vhost_net rx batch dequeuing David Miller <davem@davemloft.net> - 2017-05-18 16:10 +0200

#1642952 — [PATCH net-next V5 0/9] vhost_net rx batch dequeuing

This series tries to implement rx batching for vhost-net. This is done
by batching the dequeuing from skb_array which was exported by
underlayer socket and pass the sbk back through msg_control to finish
userspace copying. This is also the requirement for more batching
implemention on rx path.

Tests shows at most 7.56% improvment bon rx pps on top of batch
zeroing and no obvious changes for TCP_STREAM/TCP_RR result.

Please review.

Thanks

Changes from V4:
- drop batch zeroing patch
- renew the performance numbers
- move skb pointer array out of vhost_net structure

Changes from V3:
- add batch zeroing patch to fix the build warnings

Changes from V2:
- rebase to net-next HEAD
- use unconsume helpers to put skb back on releasing
- introduce and use vhost_net internal buffer helpers
- renew performance numbers on top of batch zeroing

Changes from V1:
- switch to use for() in __ptr_ring_consume_batched()
- rename peek_head_len_batched() to fetch_skbs()
- use skb_array_consume_batched() instead of
  skb_array_consume_batched_bh() since no consumer run in bh
- drop the lockless peeking patch since skb_array could be resized, so
  it's not safe to call lockless one

Jason Wang (8):
  skb_array: introduce skb_array_unconsume
  ptr_ring: introduce batch dequeuing
  skb_array: introduce batch dequeuing
  tun: export skb_array
  tap: export skb_array
  tun: support receiving skb through msg_control
  tap: support receiving skb from msg_control
  vhost_net: try batch dequing from skb array

Michael S. Tsirkin (1):
  ptr_ring: add ptr_ring_unconsume

 drivers/net/tap.c         |  25 +++++++--
 drivers/net/tun.c         |  31 ++++++++---
 drivers/vhost/net.c       | 128 +++++++++++++++++++++++++++++++++++++++++++---
 include/linux/if_tap.h    |   5 ++
 include/linux/if_tun.h    |   5 ++
 include/linux/ptr_ring.h  | 120 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/skb_array.h |  31 +++++++++++
 7 files changed, 327 insertions(+), 18 deletions(-)

-- 
2.7.4

[toc] | [next] | [standalone]

#1642953 — [PATCH net-next V5 7/9] tun: support receiving skb through msg_control

This patch makes tun_recvmsg() can receive from skb from its caller
through msg_control. Vhost_net will be the first user.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/tun.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 3cbfc5c..f8041f9c 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1510,9 +1510,8 @@ static struct sk_buff *tun_ring_recv(struct tun_file *tfile, int noblock,
 
 static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile,
 			   struct iov_iter *to,
-			   int noblock)
+			   int noblock, struct sk_buff *skb)
 {
-	struct sk_buff *skb;
 	ssize_t ret;
 	int err;
 
@@ -1521,10 +1520,12 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile,
 	if (!iov_iter_count(to))
 		return 0;
 
-	/* Read frames from ring */
-	skb = tun_ring_recv(tfile, noblock, &err);
-	if (!skb)
-		return err;
+	if (!skb) {
+		/* Read frames from ring */
+		skb = tun_ring_recv(tfile, noblock, &err);
+		if (!skb)
+			return err;
+	}
 
 	ret = tun_put_user(tun, tfile, skb, to);
 	if (unlikely(ret < 0))
@@ -1544,7 +1545,7 @@ static ssize_t tun_chr_read_iter(struct kiocb *iocb, struct iov_iter *to)
 
 	if (!tun)
 		return -EBADFD;
-	ret = tun_do_read(tun, tfile, to, file->f_flags & O_NONBLOCK);
+	ret = tun_do_read(tun, tfile, to, file->f_flags & O_NONBLOCK, NULL);
 	ret = min_t(ssize_t, ret, len);
 	if (ret > 0)
 		iocb->ki_pos = ret;
@@ -1646,7 +1647,8 @@ static int tun_recvmsg(struct socket *sock, struct msghdr *m, size_t total_len,
 					 SOL_PACKET, TUN_TX_TIMESTAMP);
 		goto out;
 	}
-	ret = tun_do_read(tun, tfile, &m->msg_iter, flags & MSG_DONTWAIT);
+	ret = tun_do_read(tun, tfile, &m->msg_iter, flags & MSG_DONTWAIT,
+			  m->msg_control);
 	if (ret > (ssize_t)total_len) {
 		m->msg_flags |= MSG_TRUNC;
 		ret = flags & MSG_TRUNC ? ret : total_len;
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1642955 — [PATCH net-next V5 1/9] ptr_ring: add ptr_ring_unconsume

From: "Michael S. Tsirkin" <mst@redhat.com>

Applications that consume a batch of entries in one go
can benefit from ability to return some of them back
into the ring.

Add an API for that - assuming there's space. If there's no space
naturally can't do this and have to drop entries, but this implies ring
is full so we'd likely drop some anyway.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 include/linux/ptr_ring.h | 55 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/include/linux/ptr_ring.h b/include/linux/ptr_ring.h
index 6b2e0dd..796b90f 100644
--- a/include/linux/ptr_ring.h
+++ b/include/linux/ptr_ring.h
@@ -403,6 +403,61 @@ static inline int ptr_ring_init(struct ptr_ring *r, int size, gfp_t gfp)
 	return 0;
 }
 
+/*
+ * Return entries into ring. Destroy entries that don't fit.
+ *
+ * Note: this is expected to be a rare slow path operation.
+ *
+ * Note: producer lock is nested within consumer lock, so if you
+ * resize you must make sure all uses nest correctly.
+ * In particular if you consume ring in interrupt or BH context, you must
+ * disable interrupts/BH when doing so.
+ */
+static inline void ptr_ring_unconsume(struct ptr_ring *r, void **batch, int n,
+				      void (*destroy)(void *))
+{
+	unsigned long flags;
+	int head;
+
+	spin_lock_irqsave(&r->consumer_lock, flags);
+	spin_lock(&r->producer_lock);
+
+	if (!r->size)
+		goto done;
+
+	/*
+	 * Clean out buffered entries (for simplicity). This way following code
+	 * can test entries for NULL and if not assume they are valid.
+	 */
+	head = r->consumer_head - 1;
+	while (likely(head >= r->consumer_tail))
+		r->queue[head--] = NULL;
+	r->consumer_tail = r->consumer_head;
+
+	/*
+	 * Go over entries in batch, start moving head back and copy entries.
+	 * Stop when we run into previously unconsumed entries.
+	 */
+	while (n) {
+		head = r->consumer_head - 1;
+		if (head < 0)
+			head = r->size - 1;
+		if (r->queue[head]) {
+			/* This batch entry will have to be destroyed. */
+			goto done;
+		}
+		r->queue[head] = batch[--n];
+		r->consumer_tail = r->consumer_head = head;
+	}
+
+done:
+	/* Destroy all entries left in the batch. */
+	while (n)
+		destroy(batch[--n]);
+	spin_unlock(&r->producer_lock);
+	spin_unlock_irqrestore(&r->consumer_lock, flags);
+}
+
 static inline void **__ptr_ring_swap_queue(struct ptr_ring *r, void **queue,
 					   int size, gfp_t gfp,
 					   void (*destroy)(void *))
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1642956 — [PATCH net-next V5 8/9] tap: support receiving skb from msg_control

This patch makes tap_recvmsg() can receive from skb from its caller
through msg_control. Vhost_net will be the first user.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 drivers/net/tap.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/tap.c b/drivers/net/tap.c
index abdaf86..9af3239 100644
--- a/drivers/net/tap.c
+++ b/drivers/net/tap.c
@@ -824,15 +824,17 @@ static ssize_t tap_put_user(struct tap_queue *q,
 
 static ssize_t tap_do_read(struct tap_queue *q,
 			   struct iov_iter *to,
-			   int noblock)
+			   int noblock, struct sk_buff *skb)
 {
 	DEFINE_WAIT(wait);
-	struct sk_buff *skb;
 	ssize_t ret = 0;
 
 	if (!iov_iter_count(to))
 		return 0;
 
+	if (skb)
+		goto put;
+
 	while (1) {
 		if (!noblock)
 			prepare_to_wait(sk_sleep(&q->sk), &wait,
@@ -856,6 +858,7 @@ static ssize_t tap_do_read(struct tap_queue *q,
 	if (!noblock)
 		finish_wait(sk_sleep(&q->sk), &wait);
 
+put:
 	if (skb) {
 		ret = tap_put_user(q, skb, to);
 		if (unlikely(ret < 0))
@@ -872,7 +875,7 @@ static ssize_t tap_read_iter(struct kiocb *iocb, struct iov_iter *to)
 	struct tap_queue *q = file->private_data;
 	ssize_t len = iov_iter_count(to), ret;
 
-	ret = tap_do_read(q, to, file->f_flags & O_NONBLOCK);
+	ret = tap_do_read(q, to, file->f_flags & O_NONBLOCK, NULL);
 	ret = min_t(ssize_t, ret, len);
 	if (ret > 0)
 		iocb->ki_pos = ret;
@@ -1155,7 +1158,8 @@ static int tap_recvmsg(struct socket *sock, struct msghdr *m,
 	int ret;
 	if (flags & ~(MSG_DONTWAIT|MSG_TRUNC))
 		return -EINVAL;
-	ret = tap_do_read(q, &m->msg_iter, flags & MSG_DONTWAIT);
+	ret = tap_do_read(q, &m->msg_iter, flags & MSG_DONTWAIT,
+			  m->msg_control);
 	if (ret > total_len) {
 		m->msg_flags |= MSG_TRUNC;
 		ret = flags & MSG_TRUNC ? ret : total_len;
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1642957 — [PATCH net-next V5 2/9] skb_array: introduce skb_array_unconsume

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 include/linux/skb_array.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/skb_array.h b/include/linux/skb_array.h
index f4dfade..79850b6 100644
--- a/include/linux/skb_array.h
+++ b/include/linux/skb_array.h
@@ -156,6 +156,12 @@ static void __skb_array_destroy_skb(void *ptr)
 	kfree_skb(ptr);
 }
 
+static inline void skb_array_unconsume(struct skb_array *a,
+				       struct sk_buff **skbs, int n)
+{
+	ptr_ring_unconsume(&a->ring, (void **)skbs, n, __skb_array_destroy_skb);
+}
+
 static inline int skb_array_resize(struct skb_array *a, int size, gfp_t gfp)
 {
 	return ptr_ring_resize(&a->ring, size, gfp, __skb_array_destroy_skb);
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1643632

On Wed, May 17, 2017 at 12:14:36PM +0800, Jason Wang wrote:
> This series tries to implement rx batching for vhost-net. This is done
> by batching the dequeuing from skb_array which was exported by
> underlayer socket and pass the sbk back through msg_control to finish
> userspace copying. This is also the requirement for more batching
> implemention on rx path.
> 
> Tests shows at most 7.56% improvment bon rx pps on top of batch
> zeroing and no obvious changes for TCP_STREAM/TCP_RR result.
> 
> Please review.
> 
> Thanks

A surprisingly large gain for such as simple change.  It would be nice
to understand better why this helps - in particular, does the optimal
batch size change if ring is bigger or smaller? But let's merge it
meanwhile.

Series:

Acked-by: Michael S. Tsirkin <mst@redhat.com>



> Changes from V4:
> - drop batch zeroing patch
> - renew the performance numbers
> - move skb pointer array out of vhost_net structure
> 
> Changes from V3:
> - add batch zeroing patch to fix the build warnings
> 
> Changes from V2:
> - rebase to net-next HEAD
> - use unconsume helpers to put skb back on releasing
> - introduce and use vhost_net internal buffer helpers
> - renew performance numbers on top of batch zeroing
> 
> Changes from V1:
> - switch to use for() in __ptr_ring_consume_batched()
> - rename peek_head_len_batched() to fetch_skbs()
> - use skb_array_consume_batched() instead of
>   skb_array_consume_batched_bh() since no consumer run in bh
> - drop the lockless peeking patch since skb_array could be resized, so
>   it's not safe to call lockless one
> 
> Jason Wang (8):
>   skb_array: introduce skb_array_unconsume
>   ptr_ring: introduce batch dequeuing
>   skb_array: introduce batch dequeuing
>   tun: export skb_array
>   tap: export skb_array
>   tun: support receiving skb through msg_control
>   tap: support receiving skb from msg_control
>   vhost_net: try batch dequing from skb array
> 
> Michael S. Tsirkin (1):
>   ptr_ring: add ptr_ring_unconsume
> 
>  drivers/net/tap.c         |  25 +++++++--
>  drivers/net/tun.c         |  31 ++++++++---
>  drivers/vhost/net.c       | 128 +++++++++++++++++++++++++++++++++++++++++++---
>  include/linux/if_tap.h    |   5 ++
>  include/linux/if_tun.h    |   5 ++
>  include/linux/ptr_ring.h  | 120 +++++++++++++++++++++++++++++++++++++++++++
>  include/linux/skb_array.h |  31 +++++++++++
>  7 files changed, 327 insertions(+), 18 deletions(-)
> 
> -- 
> 2.7.4

[toc] | [prev] | [next] | [standalone]

#1645221

On 2017年05月18日 04:59, Michael S. Tsirkin wrote:
> On Wed, May 17, 2017 at 12:14:36PM +0800, Jason Wang wrote:
>> This series tries to implement rx batching for vhost-net. This is done
>> by batching the dequeuing from skb_array which was exported by
>> underlayer socket and pass the sbk back through msg_control to finish
>> userspace copying. This is also the requirement for more batching
>> implemention on rx path.
>>
>> Tests shows at most 7.56% improvment bon rx pps on top of batch
>> zeroing and no obvious changes for TCP_STREAM/TCP_RR result.
>>
>> Please review.
>>
>> Thanks
> A surprisingly large gain for such as simple change.  It would be nice
> to understand better why this helps - in particular, does the optimal
> batch size change if ring is bigger or smaller?

Will test, just want to confirm. You mean virtio ring not tx_queue_len here?

Thanks

> But let's merge it
> meanwhile.
>
> Series:
>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>
>
>
>> Changes from V4:
>> - drop batch zeroing patch
>> - renew the performance numbers
>> - move skb pointer array out of vhost_net structure
>>
>> Changes from V3:
>> - add batch zeroing patch to fix the build warnings
>>
>> Changes from V2:
>> - rebase to net-next HEAD
>> - use unconsume helpers to put skb back on releasing
>> - introduce and use vhost_net internal buffer helpers
>> - renew performance numbers on top of batch zeroing
>>
>> Changes from V1:
>> - switch to use for() in __ptr_ring_consume_batched()
>> - rename peek_head_len_batched() to fetch_skbs()
>> - use skb_array_consume_batched() instead of
>>    skb_array_consume_batched_bh() since no consumer run in bh
>> - drop the lockless peeking patch since skb_array could be resized, so
>>    it's not safe to call lockless one
>>
>> Jason Wang (8):
>>    skb_array: introduce skb_array_unconsume
>>    ptr_ring: introduce batch dequeuing
>>    skb_array: introduce batch dequeuing
>>    tun: export skb_array
>>    tap: export skb_array
>>    tun: support receiving skb through msg_control
>>    tap: support receiving skb from msg_control
>>    vhost_net: try batch dequing from skb array
>>
>> Michael S. Tsirkin (1):
>>    ptr_ring: add ptr_ring_unconsume
>>
>>   drivers/net/tap.c         |  25 +++++++--
>>   drivers/net/tun.c         |  31 ++++++++---
>>   drivers/vhost/net.c       | 128 +++++++++++++++++++++++++++++++++++++++++++---
>>   include/linux/if_tap.h    |   5 ++
>>   include/linux/if_tun.h    |   5 ++
>>   include/linux/ptr_ring.h  | 120 +++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/skb_array.h |  31 +++++++++++
>>   7 files changed, 327 insertions(+), 18 deletions(-)
>>
>> -- 
>> 2.7.4

[toc] | [prev] | [next] | [standalone]

#1645752

On Fri, May 19, 2017 at 02:27:16PM +0800, Jason Wang wrote:
> 
> 
> On 2017年05月18日 04:59, Michael S. Tsirkin wrote:
> > On Wed, May 17, 2017 at 12:14:36PM +0800, Jason Wang wrote:
> > > This series tries to implement rx batching for vhost-net. This is done
> > > by batching the dequeuing from skb_array which was exported by
> > > underlayer socket and pass the sbk back through msg_control to finish
> > > userspace copying. This is also the requirement for more batching
> > > implemention on rx path.
> > > 
> > > Tests shows at most 7.56% improvment bon rx pps on top of batch
> > > zeroing and no obvious changes for TCP_STREAM/TCP_RR result.
> > > 
> > > Please review.
> > > 
> > > Thanks
> > A surprisingly large gain for such as simple change.  It would be nice
> > to understand better why this helps - in particular, does the optimal
> > batch size change if ring is bigger or smaller?
> 
> Will test, just want to confirm. You mean virtio ring not tx_queue_len here?
> 
> Thanks

Exactly.
Thanks,
MST

[toc] | [prev] | [next] | [standalone]

#1644584

From: Jason Wang <jasowang@redhat.com>
Date: Wed, 17 May 2017 12:14:36 +0800

> This series tries to implement rx batching for vhost-net. This is done
> by batching the dequeuing from skb_array which was exported by
> underlayer socket and pass the sbk back through msg_control to finish
> userspace copying. This is also the requirement for more batching
> implemention on rx path.
> 
> Tests shows at most 7.56% improvment bon rx pps on top of batch
> zeroing and no obvious changes for TCP_STREAM/TCP_RR result.

Series applied, thanks Jason.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web