Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1283254 > unrolled thread

[PATCH 0/5] Threaded MSI interrupt for VFIO PCI device

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/5] Threaded MSI interrupt for VFIO PCI device Yunhong Jiang <yunhong.jiang@linux.intel.com> - 2015-12-03 19:40 +0100
    [PATCH 2/5] VIRT: Support runtime irq_bypass consumer Yunhong Jiang <yunhong.jiang@linux.intel.com> - 2015-12-03 19:40 +0100
    [PATCH 1/5] KVM: Extract the irqfd_wakeup_pollin/irqfd_wakeup_pollup Yunhong Jiang <yunhong.jiang@linux.intel.com> - 2015-12-03 19:40 +0100
    [PATCH 3/5] VFIO: Support threaded interrupt handling on VFIO Yunhong Jiang <yunhong.jiang@linux.intel.com> - 2015-12-03 19:40 +0100
    Re: [PATCH 0/5] Threaded MSI interrupt for VFIO PCI device Alex Williamson <alex.williamson@redhat.com> - 2015-12-03 20:00 +0100
      Re: [PATCH 0/5] Threaded MSI interrupt for VFIO PCI device Yunhong Jiang <yunhong.jiang@linux.intel.com> - 2015-12-03 23:50 +0100

#1283254 — [PATCH 0/5] Threaded MSI interrupt for VFIO PCI device

When assigning a VFIO device to a KVM guest with low latency requirement, it  
is better to handle the interrupt in the hard interrupt context, to reduce 
the context switch to/from the IRQ thread.

Based on discussion on https://lkml.org/lkml/2015/10/26/764, the VFIO msi 
interrupt is changed to use request_threaded_irq(). The primary interrupt 
handler tries to set the guest interrupt atomically. If it fails to achieve 
it, a threaded interrupt handler will be invoked.

The irq_bypass manager is extended for this purpose. The KVM eventfd will 
provide a irqbypass consumer to handle the interrupt at hard interrupt 
context. The producer will invoke the consumer's handler then.

Yunhong Jiang (5):
  Extract the irqfd_wakeup_pollin/irqfd_wakeup_pollup
  Support runtime irq_bypass consumer
  Support threaded interrupt handling on VFIO
  Add the irq handling consumer
  Expose x86 kvm_arch_set_irq_inatomic()

 arch/x86/kvm/Kconfig              |   1 +
 drivers/vfio/pci/vfio_pci_intrs.c |  39 ++++++++++--
 include/linux/irqbypass.h         |   8 +++
 include/linux/kvm_host.h          |  19 +++++-
 include/linux/kvm_irqfd.h         |   1 +
 virt/kvm/Kconfig                  |   3 +
 virt/kvm/eventfd.c                | 131 ++++++++++++++++++++++++++------------
 virt/lib/irqbypass.c              |  82 ++++++++++++++++++------
 8 files changed, 214 insertions(+), 70 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1283257 — [PATCH 2/5] VIRT: Support runtime irq_bypass consumer

Extend the irq_bypass manager to support runtime consumers. A runtime
irq_bypass consumer can handle interrupt when an interrupt triggered. A
runtime consumer has it's handle_irq() function set and passing a
irq_context for the irq handling.

A producer keep a link for the runtime consumers, so that it can invoke
each consumer's handle_irq() when irq invoked.

Currently the irq_bypass manager has several code path assuming there is
only one consumer/producer pair for each token. For example, when
register the producer, it exits the loop after finding one match
consumer.  This is updated to support both static consumer (like for
Posted Interrupt consumer) and runtime consumer.

Signed-off-by: Yunhong Jiang <yunhong.jiang@linux.intel.com>
---
 include/linux/irqbypass.h |  8 +++++
 virt/lib/irqbypass.c      | 82 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 69 insertions(+), 21 deletions(-)

diff --git a/include/linux/irqbypass.h b/include/linux/irqbypass.h
index 1551b5b2f4c2..d5bec0c7be3a 100644
--- a/include/linux/irqbypass.h
+++ b/include/linux/irqbypass.h
@@ -12,6 +12,7 @@
 #define IRQBYPASS_H
 
 #include <linux/list.h>
+#include <linux/srcu.h>
 
 struct irq_bypass_consumer;
 
@@ -47,6 +48,9 @@ struct irq_bypass_consumer;
  */
 struct irq_bypass_producer {
 	struct list_head node;
+	/* Update side is synchronized by the lock on irqbypass.c */
+	struct srcu_struct srcu;
+	struct list_head consumers;
 	void *token;
 	int irq;
 	int (*add_consumer)(struct irq_bypass_producer *,
@@ -61,6 +65,7 @@ struct irq_bypass_producer {
  * struct irq_bypass_consumer - IRQ bypass consumer definition
  * @node: IRQ bypass manager private list management
  * @token: opaque token to match between producer and consumer
+ * @sibling: consumers with same token list management
  * @add_producer: Connect the IRQ consumer to an IRQ producer
  * @del_producer: Disconnect the IRQ consumer from an IRQ producer
  * @stop: Perform any quiesce operations necessary prior to add/del (optional)
@@ -73,6 +78,7 @@ struct irq_bypass_producer {
  */
 struct irq_bypass_consumer {
 	struct list_head node;
+	struct list_head sibling;
 	void *token;
 	int (*add_producer)(struct irq_bypass_consumer *,
 			    struct irq_bypass_producer *);
@@ -80,6 +86,8 @@ struct irq_bypass_consumer {
 			     struct irq_bypass_producer *);
 	void (*stop)(struct irq_bypass_consumer *);
 	void (*start)(struct irq_bypass_consumer *);
+	int (*handle_irq)(void *arg);
+	void *irq_context;
 };
 
 int irq_bypass_register_producer(struct irq_bypass_producer *);
diff --git a/virt/lib/irqbypass.c b/virt/lib/irqbypass.c
index 09a03b5a21ff..43ef9e2c77dc 100644
--- a/virt/lib/irqbypass.c
+++ b/virt/lib/irqbypass.c
@@ -21,6 +21,7 @@
 #include <linux/list.h>
 #include <linux/module.h>
 #include <linux/mutex.h>
+#include <linux/rculist.h>
 
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("IRQ bypass manager utility module");
@@ -49,11 +50,8 @@ static int __connect(struct irq_bypass_producer *prod,
 			prod->del_consumer(prod, cons);
 	}
 
-	if (cons->start)
-		cons->start(cons);
-	if (prod->start)
-		prod->start(prod);
-
+	if (!ret && cons->handle_irq)
+		list_add_rcu(&cons->sibling, &prod->consumers);
 	return ret;
 }
 
@@ -71,6 +69,11 @@ static void __disconnect(struct irq_bypass_producer *prod,
 	if (prod->del_consumer)
 		prod->del_consumer(prod, cons);
 
+	if (cons->handle_irq) {
+		list_del_rcu(&cons->sibling);
+		synchronize_srcu(&prod->srcu);
+	}
+
 	if (cons->start)
 		cons->start(cons);
 	if (prod->start)
@@ -87,7 +90,8 @@ static void __disconnect(struct irq_bypass_producer *prod,
 int irq_bypass_register_producer(struct irq_bypass_producer *producer)
 {
 	struct irq_bypass_producer *tmp;
-	struct irq_bypass_consumer *consumer;
+	struct list_head *node, *next, siblings = LIST_HEAD_INIT(siblings);
+	int ret;
 
 	might_sleep();
 
@@ -96,6 +100,9 @@ int irq_bypass_register_producer(struct irq_bypass_producer *producer)
 
 	mutex_lock(&lock);
 
+	INIT_LIST_HEAD(&producer->consumers);
+	init_srcu_struct(&producer->srcu);
+
 	list_for_each_entry(tmp, &producers, node) {
 		if (tmp->token == producer->token) {
 			mutex_unlock(&lock);
@@ -104,23 +111,48 @@ int irq_bypass_register_producer(struct irq_bypass_producer *producer)
 		}
 	}
 
-	list_for_each_entry(consumer, &consumers, node) {
+	list_for_each_safe(node, next, &consumers) {
+		struct irq_bypass_consumer *consumer = container_of(
+			node, struct irq_bypass_consumer, node);
+
 		if (consumer->token == producer->token) {
-			int ret = __connect(producer, consumer);
-			if (ret) {
-				mutex_unlock(&lock);
-				module_put(THIS_MODULE);
-				return ret;
-			}
-			break;
+			ret = __connect(producer, consumer);
+			if (ret)
+				goto error;
+			/* Keep the connected consumers temply */
+			list_del(&consumer->node);
+			list_add_rcu(&consumer->node, &siblings);
 		}
 	}
 
+	list_for_each_safe(node, next, &siblings) {
+		struct irq_bypass_consumer *consumer = container_of(
+			node, struct irq_bypass_consumer, node);
+
+		list_del(&consumer->node);
+		list_add(&consumer->node, &consumers);
+		if (consumer->start)
+			consumer->start(consumer);
+	}
+
+	if (producer->start)
+		producer->start(producer);
 	list_add(&producer->node, &producers);
 
 	mutex_unlock(&lock);
-
 	return 0;
+
+error:
+	list_for_each_safe(node, next, &siblings) {
+		struct irq_bypass_consumer *consumer = container_of(
+			node, struct irq_bypass_consumer, node);
+
+		list_del(&consumer->node);
+		list_add(&consumer->node, &consumers);
+	}
+	mutex_unlock(&lock);
+	module_put(THIS_MODULE);
+	return ret;
 }
 EXPORT_SYMBOL_GPL(irq_bypass_register_producer);
 
@@ -133,7 +165,7 @@ EXPORT_SYMBOL_GPL(irq_bypass_register_producer);
  */
 void irq_bypass_unregister_producer(struct irq_bypass_producer *producer)
 {
-	struct irq_bypass_producer *tmp;
+	struct irq_bypass_producer *tmp, *n;
 	struct irq_bypass_consumer *consumer;
 
 	might_sleep();
@@ -143,7 +175,7 @@ void irq_bypass_unregister_producer(struct irq_bypass_producer *producer)
 
 	mutex_lock(&lock);
 
-	list_for_each_entry(tmp, &producers, node) {
+	list_for_each_entry_safe(tmp, n, &producers, node) {
 		if (tmp->token != producer->token)
 			continue;
 
@@ -159,6 +191,7 @@ void irq_bypass_unregister_producer(struct irq_bypass_producer *producer)
 		break;
 	}
 
+	cleanup_srcu_struct(&producer->srcu);
 	mutex_unlock(&lock);
 
 	module_put(THIS_MODULE);
@@ -180,6 +213,9 @@ int irq_bypass_register_consumer(struct irq_bypass_consumer *consumer)
 	if (!consumer->add_producer || !consumer->del_producer)
 		return -EINVAL;
 
+	if (consumer->handle_irq && !consumer->irq_context)
+		return -EINVAL;
+
 	might_sleep();
 
 	if (!try_module_get(THIS_MODULE))
@@ -188,7 +224,7 @@ int irq_bypass_register_consumer(struct irq_bypass_consumer *consumer)
 	mutex_lock(&lock);
 
 	list_for_each_entry(tmp, &consumers, node) {
-		if (tmp->token == consumer->token) {
+		if (tmp == consumer) {
 			mutex_unlock(&lock);
 			module_put(THIS_MODULE);
 			return -EBUSY;
@@ -203,6 +239,10 @@ int irq_bypass_register_consumer(struct irq_bypass_consumer *consumer)
 				module_put(THIS_MODULE);
 				return ret;
 			}
+			if (consumer->start)
+				consumer->start(consumer);
+			if (producer->start)
+				producer->start(producer);
 			break;
 		}
 	}
@@ -224,7 +264,7 @@ EXPORT_SYMBOL_GPL(irq_bypass_register_consumer);
  */
 void irq_bypass_unregister_consumer(struct irq_bypass_consumer *consumer)
 {
-	struct irq_bypass_consumer *tmp;
+	struct irq_bypass_consumer *tmp, *n;
 	struct irq_bypass_producer *producer;
 
 	might_sleep();
@@ -234,8 +274,8 @@ void irq_bypass_unregister_consumer(struct irq_bypass_consumer *consumer)
 
 	mutex_lock(&lock);
 
-	list_for_each_entry(tmp, &consumers, node) {
-		if (tmp->token != consumer->token)
+	list_for_each_entry_safe(tmp, n, &consumers, node) {
+		if (tmp != consumer)
 			continue;
 
 		list_for_each_entry(producer, &producers, node) {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283258 — [PATCH 1/5] KVM: Extract the irqfd_wakeup_pollin/irqfd_wakeup_pollup

Separate the irqfd_wakeup_pollin/irqfd_wakeup_pollup from the
irqfd_wakeup, so that we can reuse the logic for MSI fastpath injection.

Signed-off-by: Yunhong Jiang <yunhong.jiang@linux.intel.com>
---
 virt/kvm/eventfd.c | 86 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 51 insertions(+), 35 deletions(-)

diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 46dbc0a7dfc1..c31d43b762db 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -180,6 +180,53 @@ int __attribute__((weak)) kvm_arch_set_irq_inatomic(
 	return -EWOULDBLOCK;
 }
 
+static int
+irqfd_wakeup_pollin(struct kvm_kernel_irqfd *irqfd)
+{
+	struct kvm *kvm = irqfd->kvm;
+	struct kvm_kernel_irq_routing_entry irq;
+	unsigned seq;
+	int idx, ret;
+
+	idx = srcu_read_lock(&kvm->irq_srcu);
+	do {
+		seq = read_seqcount_begin(&irqfd->irq_entry_sc);
+		irq = irqfd->irq_entry;
+	} while (read_seqcount_retry(&irqfd->irq_entry_sc, seq));
+	/* An event has been signaled, inject an interrupt */
+	ret = kvm_arch_set_irq_inatomic(&irq, kvm,
+				KVM_USERSPACE_IRQ_SOURCE_ID, 1,
+				false);
+	srcu_read_unlock(&kvm->irq_srcu, idx);
+
+	return ret;
+}
+
+static int
+irqfd_wakeup_pollup(struct kvm_kernel_irqfd *irqfd)
+{
+	struct kvm *kvm = irqfd->kvm;
+	unsigned long flags;
+
+	spin_lock_irqsave(&kvm->irqfds.lock, flags);
+
+	/*
+	 * We must check if someone deactivated the irqfd before
+	 * we could acquire the irqfds.lock since the item is
+	 * deactivated from the KVM side before it is unhooked from
+	 * the wait-queue.  If it is already deactivated, we can
+	 * simply return knowing the other side will cleanup for us.
+	 * We cannot race against the irqfd going away since the
+	 * other side is required to acquire wqh->lock, which we hold
+	 */
+	if (irqfd_is_active(irqfd))
+		irqfd_deactivate(irqfd);
+
+	spin_unlock_irqrestore(&kvm->irqfds.lock, flags);
+
+	return 0;
+}
+
 /*
  * Called with wqh->lock held and interrupts disabled
  */
@@ -189,45 +236,14 @@ irqfd_wakeup(wait_queue_t *wait, unsigned mode, int sync, void *key)
 	struct kvm_kernel_irqfd *irqfd =
 		container_of(wait, struct kvm_kernel_irqfd, wait);
 	unsigned long flags = (unsigned long)key;
-	struct kvm_kernel_irq_routing_entry irq;
-	struct kvm *kvm = irqfd->kvm;
-	unsigned seq;
-	int idx;
 
-	if (flags & POLLIN) {
-		idx = srcu_read_lock(&kvm->irq_srcu);
-		do {
-			seq = read_seqcount_begin(&irqfd->irq_entry_sc);
-			irq = irqfd->irq_entry;
-		} while (read_seqcount_retry(&irqfd->irq_entry_sc, seq));
-		/* An event has been signaled, inject an interrupt */
-		if (kvm_arch_set_irq_inatomic(&irq, kvm,
-					      KVM_USERSPACE_IRQ_SOURCE_ID, 1,
-					      false) == -EWOULDBLOCK)
+	if (flags & POLLIN)
+		if (irqfd_wakeup_pollin(irqfd) == -EWOULDBLOCK)
 			schedule_work(&irqfd->inject);
-		srcu_read_unlock(&kvm->irq_srcu, idx);
-	}
 
-	if (flags & POLLHUP) {
+	if (flags & POLLHUP)
 		/* The eventfd is closing, detach from KVM */
-		unsigned long flags;
-
-		spin_lock_irqsave(&kvm->irqfds.lock, flags);
-
-		/*
-		 * We must check if someone deactivated the irqfd before
-		 * we could acquire the irqfds.lock since the item is
-		 * deactivated from the KVM side before it is unhooked from
-		 * the wait-queue.  If it is already deactivated, we can
-		 * simply return knowing the other side will cleanup for us.
-		 * We cannot race against the irqfd going away since the
-		 * other side is required to acquire wqh->lock, which we hold
-		 */
-		if (irqfd_is_active(irqfd))
-			irqfd_deactivate(irqfd);
-
-		spin_unlock_irqrestore(&kvm->irqfds.lock, flags);
-	}
+		irqfd_wakeup_pollup(irqfd);
 
 	return 0;
 }
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283259 — [PATCH 3/5] VFIO: Support threaded interrupt handling on VFIO

For VFIO device with MSI interrupt type, it's possible to handle the
interrupt on hard interrupt context without invoking the interrupt
thread. Handling the interrupt on hard interrupt context reduce the
interrupt latency.

Signed-off-by: Yunhong Jiang <yunhong.jiang@linux.intel.com>
---
 drivers/vfio/pci/vfio_pci_intrs.c | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c
index 3b3ba15558b7..108d335c5656 100644
--- a/drivers/vfio/pci/vfio_pci_intrs.c
+++ b/drivers/vfio/pci/vfio_pci_intrs.c
@@ -236,12 +236,35 @@ static void vfio_intx_disable(struct vfio_pci_device *vdev)
 	kfree(vdev->ctx);
 }
 
+static irqreturn_t vfio_msihandler(int irq, void *arg)
+{
+	struct vfio_pci_irq_ctx *ctx = arg;
+	struct irq_bypass_producer *producer = &ctx->producer;
+	struct irq_bypass_consumer *consumer;
+	int ret = IRQ_HANDLED, idx;
+
+	idx = srcu_read_lock(&producer->srcu);
+
+	list_for_each_entry_rcu(consumer, &producer->consumers, sibling) {
+		/*
+		 * Invoke the thread handler if any consumer would block, but
+		 * finish all consumes.
+		 */
+		if (consumer->handle_irq(consumer->irq_context) == -EWOULDBLOCK)
+			ret = IRQ_WAKE_THREAD;
+		continue;
+	}
+
+	srcu_read_unlock(&producer->srcu, idx);
+	return ret;
+}
+
 /*
  * MSI/MSI-X
  */
-static irqreturn_t vfio_msihandler(int irq, void *arg)
+static irqreturn_t vfio_msihandler_threaded(int irq, void *arg)
 {
-	struct eventfd_ctx *trigger = arg;
+	struct eventfd_ctx *trigger = ((struct vfio_pci_irq_ctx *)arg)->trigger;
 
 	eventfd_signal(trigger, 1);
 	return IRQ_HANDLED;
@@ -318,7 +341,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev,
 		return -EINVAL;
 
 	if (vdev->ctx[vector].trigger) {
-		free_irq(irq, vdev->ctx[vector].trigger);
+		free_irq(irq, &vdev->ctx[vector]);
 		irq_bypass_unregister_producer(&vdev->ctx[vector].producer);
 		kfree(vdev->ctx[vector].name);
 		eventfd_ctx_put(vdev->ctx[vector].trigger);
@@ -353,8 +376,14 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev,
 		pci_write_msi_msg(irq, &msg);
 	}
 
-	ret = request_irq(irq, vfio_msihandler, 0,
-			  vdev->ctx[vector].name, trigger);
+	/*
+	 * Currently the primary handler for the thread_irq will be invoked on
+	 * a thread, the IRQF_ONESHOT is a hack for it.
+	 */
+	ret = request_threaded_irq(irq, vfio_msihandler,
+				   vfio_msihandler_threaded,
+				   IRQF_ONESHOT, vdev->ctx[vector].name,
+				   &vdev->ctx[vector]);
 	if (ret) {
 		kfree(vdev->ctx[vector].name);
 		eventfd_ctx_put(trigger);
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283265

On Thu, 2015-12-03 at 10:22 -0800, Yunhong Jiang wrote:
> When assigning a VFIO device to a KVM guest with low latency requirement, it  
> is better to handle the interrupt in the hard interrupt context, to reduce 
> the context switch to/from the IRQ thread.
> 
> Based on discussion on https://lkml.org/lkml/2015/10/26/764, the VFIO msi 
> interrupt is changed to use request_threaded_irq(). The primary interrupt 
> handler tries to set the guest interrupt atomically. If it fails to achieve 
> it, a threaded interrupt handler will be invoked.
> 
> The irq_bypass manager is extended for this purpose. The KVM eventfd will 
> provide a irqbypass consumer to handle the interrupt at hard interrupt 
> context. The producer will invoke the consumer's handler then.

Do you have any performance data?  Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283433

On Thu, Dec 03, 2015 at 11:55:53AM -0700, Alex Williamson wrote:
> On Thu, 2015-12-03 at 10:22 -0800, Yunhong Jiang wrote:
> > When assigning a VFIO device to a KVM guest with low latency requirement, it  
> > is better to handle the interrupt in the hard interrupt context, to reduce 
> > the context switch to/from the IRQ thread.
> > 
> > Based on discussion on https://lkml.org/lkml/2015/10/26/764, the VFIO msi 
> > interrupt is changed to use request_threaded_irq(). The primary interrupt 
> > handler tries to set the guest interrupt atomically. If it fails to achieve 
> > it, a threaded interrupt handler will be invoked.
> > 
> > The irq_bypass manager is extended for this purpose. The KVM eventfd will 
> > provide a irqbypass consumer to handle the interrupt at hard interrupt 
> > context. The producer will invoke the consumer's handler then.
> 
> Do you have any performance data?  Thanks,

Sorry, I should include the data on the commit messages.

The test:
Launch a VM with a FPGA device, which triggers an interrpt every 1ms. The VM 
is launched on a isolated CPU with NONHZ_FULL enabled.
Two data are collected.  On the guest, a special program will check the 
latency from the time the interrupt been triggered to the time the interrupt 
received by guest IRS. On the host, I use the perf to collect the perf data.

The performance Data with the patch:

Host perf data:
[root@otcnfv02 test-tools]# perf kvm stat record -C 34 sleep 10
[root@otcnfv02 test-tools]# perf kvm stat report
Analyze events for all VMs, all VCPUs:
	     VM-EXIT    Samples  Samples%     Time%    Min Time    Max Time
Avg time
  EXTERNAL_INTERRUPT       9997    98.55%    99.31%      1.73us     17.07us
2.09us ( +-   0.21% )
   PAUSE_INSTRUCTION        127     1.25%     0.51%      0.69us      1.20us
0.84us ( +-   1.34% )
	   MSR_WRITE         20     0.20%     0.18%      1.62us      3.21us
1.93us ( +-   3.95% )

Guest data:
[nfv@rt-test-1 ~]$ ./run-int-test.sh
Latency is Min: 3.74us Max: 20.08us Avg: 4.49us
No of interrupts = 74995

The performance data without the patch:
Host perf data:
[root@otcnfv02 test-tools]# perf kvm stat record -C 34 sleep 10
[root@otcnfv02 test-tools]# perf kvm stat report
Analyze events for all VMs, all VCPUs:
	     VM-EXIT    Samples  Samples%     Time%    Min Time    Max Time
Avg time
   PAUSE_INSTRUCTION     141136    87.74%    50.39%      0.69us      8.51us      
0.77us ( +-   0.07% )
  EXTERNAL_INTERRUPT      19701    12.25%    49.59%      2.31us     15.93us      
5.46us ( +-   0.12% )
	   MSR_WRITE         24     0.01%     0.02%      1.51us      2.22us      
1.91us ( +-   1.91% )

Notice:
The EXTERNAL_INTERRUPT VMExit is different w/ the patch (9997) and w/o the 
patch (19701). It is because with threaded IRQ, the NOHZ_FULL it not working 
because two threads on the pCPU, thus we have both the FPGA device interrupt 
and the tick timer interrupt. After calculation, the average time for the 
FPGA device interrupt is 4.72 us.

Guest data:
[nfv@rt-test-1 ~]$ ./run-int-test.sh
Latency is Min: 6.70us Max: 50.38us Avg: 7.44us
No of interrupts = 42639

Thanks
--jyh

> 
> Alex
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web