Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1273195 > unrolled thread

[PATCH v4 00/13] ARM IRQ forward control based on IRQ bypass manager

Back to article view | Back to linux.kernel

Contents

  [PATCH v4 00/13] ARM IRQ forward control based on IRQ bypass manager Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 13/13] KVM: arm/arm64: implement IRQ bypass consumer functions Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 03/13] VFIO: platform: test forwarded state when selecting the IRQ handler Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 05/13] VFIO: platform: add vfio_platform_set_forwarded Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 06/13] VFIO: platform: add vfio_platform_irq_is_active Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 11/13] KVM: arm/arm64: vgic: implement clear active for non shared mapped IRQ Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:00 +0100
    [PATCH v4 01/13] KVM: arm/arm64: select IRQ_BYPASS_MANAGER Eric Auger <eric.auger@linaro.org> - 2015-11-19 16:10 +0100

#1273195 — [PATCH v4 00/13] ARM IRQ forward control based on IRQ bypass manager

This series allows to optimize the deactivation of virtual interrupts
associated to a vfio platform device IRQ. Let's call this optimization:
ARM IRQ forwarding.

Without that optimization the deactivation of the physical IRQ is done
by the host and the deactivation of the virtual IRQ, by the guest,
is trapped by KVM with a maintenance IRQ. With ARM IRQ forwarding, the
physical IRQ is automatically deactivated by the GIC interrupt controller
when the guest deactivates the virtual IRQ.

The series enables the auto-negotiation of this optimization, on ARM.
The auto-negotiation is coordinated by the IRQ bypass manager. The VFIO
platform driver and KVM/irqfd cooperate to turn that feature on.

The series implements the setup of IRQ forwarding and adapt the existing
VGIC state machine so that it works.

When the IRQ is forwarded, the VFIO platform driver does not need to
mask the physical IRQ anymore before signaling the eventfd. Indeed
genirq lowers the running priority, enabling other physical IRQ to hit
except that one.

The injection still is based on irqfd triggering. The only impact on
irqfd process is the resamplefd is not triggered anymore for level
sensitive IRQs since deactivation is not trapped by KVM.

More backgroung on ARM IRQ forwarding in the text below and at
http://www.linux-kvm.org/images/a/a8/01x04-ARMdevice.pdf.

This was tested on:
- Calxeda Midway, assigning the xgmac main IRQ.
- AMD Seattle, where all XGBE interrupts are assigned:
  - 2 level sensitive and 4 DMA edge sensitive IRQs

applies on top of:
KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access

All those pieces can be found at:
https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.4-rc1-forward-v4

History:

v3 -> v4:
- dependencies now are fixed (IRQ bypass manager, guest synchronous
  halt/resume, VGIC architected timer)
  - only applies on top of:
  KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access
- Rebase on vgic series related to forwarded shared interrupts, mainly
  [PATCH v2 0/8] Rework architected timer and forwarded IRQs handling &
  [PATCH 0/2] arm/arm64: KVM: Fix architected timer issues
- took into account Alex's comment on VFIO part:
  - vfio_platform_set_forwarded replacing vfio_platform_set_automasked
  - automasked flag not updated
  - vfio_platform_is_active renamed into vfio_platform_irq_is_active
  - handle returned value for vfio_platform_irq_is_active
  - remove WARN_ON on irq_bypass_register_producer failure
- took into account Christoffer's comments on vgic part
  - clarify comments
  - clarify edge-sensitive handling and validate_injection of edge
    mapped unshared IRQ with level == 0
  - use actual vcpu_id instead of VCPU 0
  - revisit unset_forward implement using unqueue
- moved KVM: arm/arm64: select IRQ_BYPASS_MANAGER in this series
- add support for clear active and clear pending

v2 (RFC) -> v3(PATCH):
- all dependencies now have a patch status
- we dropped the producer active boolean exchanged between the
  VFIO producer and irqfd arm consumer. As a consequence, on
  unforward, if the IRQ is active, this latter is deactivated
  without VFIO-masking it. So we do not exactly come back to the
  exact state where we would be in unforwarded state. A new
  physical IRQ can hit while the previous virtual IRQ is under
  treatment. Its injection in the guest may be rejected thanks
  to the VGIC state machine. This IRQ will be lost but I don't
  think this is a severe issue. In case no new IRQ hits, the
  guest deactivation of the virtual IRQ will trigger the resamplefd
  which will VFIO unmask the non-masked IRQ. This also has no
  consequence.
- VFIO platform driver consumer_add now can fail. It rejects the
  transition for forwarding state in case the IRQ is active
- the series is rebased on new irq_vcpu_affinity series
- no dependency anymore on "chip/vgic adaptations for forwarded irq"
  which was partially integrated into Marc's series. A fix is still
  needed through.
- Guest synchronous halt/resume patch re-integrated into this series
- integrate a new patch file coming mixing
  [PATCH v4 11/11] KVM: arm/arm64: vgic: Allow HW interrupts for
                   non-shared devices &
  [RFC v2 2/4] KVM: arm: vgic: fix state machine for forwarded IRQ

v1 -> v2:
- irq bypass manager and irqfd consumer moved in a separate patch
- kvm_arm_[halt,resume]_guest moved in a separate patch
- remove VFIO external functions since we do not need them anymore
- apply container_of strategy advised by Paolo. Only active field
  remains and discussions will tell whether we get rid of it.
- renamed kvm_arch functions

- kvm-vfio v6 -> RFC v1 based on IRQ bypass manager
  see previous history in https://lkml.org/lkml/2015/4/13/353).

Best Regards

Eric


Eric Auger (12):
  KVM: arm/arm64: select IRQ_BYPASS_MANAGER
  VFIO: platform: registration of a dummy IRQ bypass producer
  VFIO: platform: test forwarded state when selecting the IRQ handler
  VFIO: platform: single handler using function pointer
  VFIO: platform: add vfio_platform_set_forwarded
  VFIO: platform: add vfio_platform_irq_is_active
  VFIO: platform: add irq bypass producer management
  KVM: arm/arm64: vgic: support irqfd injection of a mapped IRQ
  KVM: arm/arm64: vgic: forwarding control
  KVM: arm/arm64: vgic: implement clear active for non shared mapped IRQ
  KVM: arm/arm64: vgic: implement clear pending for non shared mapped
    IRQ
  KVM: arm/arm64: implement IRQ bypass consumer functions

Marc Zyngier (1):
  KVM: arm/arm64: vgic: adapt state machine for non shared mapped
    interrupts

 arch/arm/kvm/Kconfig                          |   3 +
 arch/arm/kvm/arm.c                            |  37 +++++
 arch/arm64/kvm/Kconfig                        |   3 +
 drivers/vfio/platform/vfio_platform_irq.c     | 120 +++++++++++++-
 drivers/vfio/platform/vfio_platform_private.h |   4 +
 include/kvm/arm_vgic.h                        |   8 +-
 virt/kvm/arm/arch_timer.c                     |   2 +-
 virt/kvm/arm/vgic.c                           | 221 ++++++++++++++++++++++++--
 8 files changed, 376 insertions(+), 22 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1273197 — [PATCH v4 13/13] KVM: arm/arm64: implement IRQ bypass consumer functions

Implement IRQ bypass callbacks for arm/arm64 IRQ forwarding:
- kvm_arch_irq_bypass_add_producer: perform VGIC/irqchip
  settings for forwarding
- kvm_arch_irq_bypass_del_producer: same for inverse operation
- kvm_arch_irq_bypass_stop: halt guest execution
- kvm_arch_irq_bypass_start: resume guest execution

and set CONFIG_HAVE_KVM_IRQ_BYPASS for arm/arm64.

Signed-off-by: Eric Auger <eric.auger@linaro.org>

---
v3 -> v4:
- kvm_set/unset_forward now takes the virtual IRQ ID as a parameter

v2 -> v3:
- kvm_arch_irq_bypass_resume renamed into kvm_arch_irq_bypass_start
- kvm_vgic_unset_forward does not take the active bool param anymore
- kvm_arch_irq_bypass_add_producer now returns an error value
- remove kvm_arch_irq_bypass_update

v1 -> v2:
- struct kvm_kernel_irqfd is retrieved with container_of
- function names changed
---
 arch/arm/kvm/Kconfig   |  1 +
 arch/arm/kvm/arm.c     | 37 +++++++++++++++++++++++++++++++++++++
 arch/arm64/kvm/Kconfig |  1 +
 3 files changed, 39 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 73d7201..4c9a95e 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -34,6 +34,7 @@ config KVM
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
 	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
 	---help---
 	  Support hosting virtualized guest machines.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index eab83b2..99517bd 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -27,6 +27,8 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/kvm_irqfd.h>
+#include <linux/irqbypass.h>
 #include <trace/events/kvm.h>
 
 #define CREATE_TRACE_POINTS
@@ -1165,6 +1167,41 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
 	return NULL;
 }
 
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	return kvm_vgic_set_forward(irqfd->kvm, prod->irq,
+				    irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+				      struct irq_bypass_producer *prod)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_vgic_unset_forward(irqfd->kvm, prod->irq,
+			       irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+	struct kvm_kernel_irqfd *irqfd =
+		container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+	kvm_arm_resume_guest(irqfd->kvm);
+}
+
 /**
  * Initialize Hyp-mode and memory mappings on all CPUs.
  */
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 64f460d..f6829af 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -38,6 +38,7 @@ config KVM
 	select HAVE_KVM_IRQFD
 	select KVM_ARM_VGIC_V3
 	select IRQ_BYPASS_MANAGER
+	select HAVE_KVM_IRQ_BYPASS
 	---help---
 	  Support hosting virtualized guest machines.
 	  We don't support KVM with 16K page tables yet, due to the multiple
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273201 — [PATCH v4 03/13] VFIO: platform: test forwarded state when selecting the IRQ handler

Add a new forwarded flag in vfio_platform_irq.  In case the IRQ
is forwarded, the VFIO platform IRQ handler does not need to
disable the IRQ anymore.

When setting the IRQ handler we now also test the forwarded state. In
case the IRQ is forwarded we select the vfio_irq_handler.

Signed-off-by: Eric Auger <eric.auger@linaro.org>

---

v2:
- add a new forwarded flag and do not use irqd_irq_forwarded anymore
---
 drivers/vfio/platform/vfio_platform_irq.c     | 3 ++-
 drivers/vfio/platform/vfio_platform_private.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index ab7658a..d4462fd 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -265,7 +265,8 @@ static int vfio_platform_set_irq_trigger(struct vfio_platform_device *vdev,
 	struct vfio_platform_irq *irq = &vdev->irqs[index];
 	irq_handler_t handler;
 
-	if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED)
+	if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED &&
+			!irq->forwarded)
 		handler = vfio_automasked_irq_handler;
 	else
 		handler = vfio_irq_handler;
diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h
index 979c191..55e8043 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -39,6 +39,7 @@ struct vfio_platform_irq {
 	struct virqfd		*unmask;
 	struct virqfd		*mask;
 	struct irq_bypass_producer producer;
+	bool			forwarded;
 };
 
 struct vfio_platform_region {
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273202 — [PATCH v4 05/13] VFIO: platform: add vfio_platform_set_forwarded

This function allows to change the forwarded mode and selects
the IRQ handler accordingly.

Signed-off-by: Eric Auger <eric.auger@linaro.org>

---
v3 -> v3:
- renamed vfio_platform_set_automasked into
  vfio_platform_set_forwarded
- do not change VFIO_IRQ_INFO_AUTOMASKED setting when turning
  forwarding on/off

v1 -> v2:
- set forwarded flag
---
 drivers/vfio/platform/vfio_platform_irq.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index c58e0ad..257200b 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -186,6 +186,23 @@ static irqreturn_t vfio_handler(int irq, void *dev_id)
 	return ret;
 }
 
+static int vfio_platform_set_forwarded(struct vfio_platform_irq *irq,
+					   bool forwarded)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&irq->lock, flags);
+	irq->forwarded = forwarded;
+
+	if (!forwarded && (irq->flags & VFIO_IRQ_INFO_AUTOMASKED))
+		irq->handler = vfio_automasked_irq_handler;
+	else
+		irq->handler = vfio_irq_handler;
+
+	spin_unlock_irqrestore(&irq->lock, flags);
+	return 0;
+}
+
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
 }
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273203 — [PATCH v4 06/13] VFIO: platform: add vfio_platform_irq_is_active

This function returns whether the IRQ is active at irqchip level or
VFIO masked. If either is true, the IRQ is considered active.
Currently there is no way to differentiate userspace masked IRQ from
automasked IRQ. There might be false detection of activity. However
it is currently acceptable to have false detection.

Signed-off-by: Eric Auger <eric.auger@linaro.org>

---

v3 -> v4:
- rename vfio_platform_is_active into vfio_platform_irq_is_active
- remove BUG_ON and return the error if irq_get_irqchip_state fails
---
 drivers/vfio/platform/vfio_platform_irq.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index 257200b..cb7d5e9 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -203,6 +203,25 @@ static int vfio_platform_set_forwarded(struct vfio_platform_irq *irq,
 	return 0;
 }
 
+static int vfio_platform_irq_is_active(struct vfio_platform_irq *irq)
+{
+	unsigned long flags;
+	bool active;
+	int ret;
+
+	spin_lock_irqsave(&irq->lock, flags);
+
+	ret = irq_get_irqchip_state(irq->hwirq, IRQCHIP_STATE_ACTIVE, &active);
+	if (ret)
+		goto out;
+
+	ret = active || irq->masked;
+
+out:
+	spin_unlock_irqrestore(&irq->lock, flags);
+	return ret;
+}
+
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
 }
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273206 — [PATCH v4 11/13] KVM: arm/arm64: vgic: implement clear active for non shared mapped IRQ

When disabling a non shared mapped IRQs, we must manually deactivate
the corresponding physical IRQ on top of removing the active state
from the distributor.

Signed-off-by: Eric Auger <eric.auger@linaro.org>
---
 virt/kvm/arm/vgic.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bd500b4..98ae15f 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -604,14 +604,34 @@ bool vgic_handle_clear_active_reg(struct kvm *kvm,
 				  struct kvm_exit_mmio *mmio,
 				  phys_addr_t offset, int vcpu_id)
 {
-	u32 *reg;
+	u32 *reg, orig, cleared;
 	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, vcpu_id);
+	unsigned int i;
 
 	reg = vgic_bitmap_get_reg(&dist->irq_active, vcpu_id, offset);
+	orig = *reg;
 	vgic_reg_access(mmio, reg, offset,
 			ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
 
 	if (mmio->is_write) {
+		cleared = orig ^ *reg;
+
+		for (i = 0; i < 32; i++) {
+			struct irq_phys_map *map;
+			unsigned int irq_num;
+
+			if (!(cleared && (1 << i)))
+				continue;
+			irq_num = (offset * 8) + i;
+			map = vgic_irq_map_search(vcpu, irq_num);
+			if (!map || (map && map->shared))
+				continue;
+			irq_set_irqchip_state(map->irq,
+					      IRQCHIP_STATE_ACTIVE,
+					      false);
+		}
+
 		vgic_update_state(kvm);
 		return true;
 	}
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1273207 — [PATCH v4 01/13] KVM: arm/arm64: select IRQ_BYPASS_MANAGER

Select IRQ_BYPASS_MANAGER when CONFIG_KVM is set

Signed-off-by: Eric Auger <eric.auger@linaro.org>
---
 arch/arm/kvm/Kconfig   | 2 ++
 arch/arm64/kvm/Kconfig | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 95a0005..73d7201 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -32,6 +33,7 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
+	select IRQ_BYPASS_MANAGER
 	depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
 	---help---
 	  Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index a5272c0..64f460d 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
 	bool "Virtualization"
@@ -36,6 +37,7 @@ config KVM
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
 	select KVM_ARM_VGIC_V3
+	select IRQ_BYPASS_MANAGER
 	---help---
 	  Support hosting virtualized guest machines.
 	  We don't support KVM with 16K page tables yet, due to the multiple
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web