Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1381443 > unrolled thread

[PATCH RFC 0/3] virtio-pci: iommu support

Started by"Michael S. Tsirkin" <mst@redhat.com>
First post2016-04-18 12:00 +0200
Last post2016-04-18 21:00 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH RFC 0/3] virtio-pci: iommu support "Michael S. Tsirkin" <mst@redhat.com> - 2016-04-18 12:00 +0200
    [PATCH RFC 1/3] virtio: add features for IOMMU control "Michael S. Tsirkin" <mst@redhat.com> - 2016-04-18 12:00 +0200
    [PATCH RFC 2/3] vfio: report group noiommu status "Michael S. Tsirkin" <mst@redhat.com> - 2016-04-18 12:00 +0200
      Re: [PATCH RFC 2/3] vfio: report group noiommu status Alex Williamson <alex.williamson@redhat.com> - 2016-04-18 21:00 +0200

#1381443 — [PATCH RFC 0/3] virtio-pci: iommu support

From"Michael S. Tsirkin" <mst@redhat.com>
Date2016-04-18 12:00 +0200
Subject[PATCH RFC 0/3] virtio-pci: iommu support
Message-ID<rpe8i-3k6-9@gated-at.bofh.it>
This is an attempt to allow enabling IOMMU for DMA.
Design:
	- new feature bit IOMMU_PLATFORM which means
          host won't bypass IOMMU
	- virtio core uses DMA API if it sees IOMMU_PLATFORM
	- add quirk for vfio to disable device unless IOMMU_PLATFORM is set
          or the no-iommu mode is enabled
	- while I'm not sure how it will be used, it seems like a good idea to
	  also have ability to distinguish between a legacy device and one
	  where iommu is bypassed intentionally.  To this end, add another feature bit
	  IOMMU_PASSTHROUGH. We don't acknowledge it if IOMMU_PLATFORM is set.

TODO:
	- I'm not sure whether there are setups that mix IOMMU
	  and no-IOMMU configs. If so, failing on probe might not
	  be the right thing to do, should fail binding to IOMMU group instead.


Michael S. Tsirkin (3):
  virtio: add features for IOMMU control
  vfio: report group noiommu status
  vfio: add virtio pci quirk

 drivers/vfio/pci/vfio_pci_private.h          |   1 +
 include/uapi/linux/virtio_config.h           |  10 +-
 drivers/vfio/pci/vfio_pci.c                  |  13 ++-
 drivers/vfio/pci/vfio_pci_virtio.c           | 142 +++++++++++++++++++++++++++
 drivers/vfio/platform/vfio_platform_common.c |   2 +-
 drivers/vfio/vfio.c                          |   5 +-
 drivers/virtio/virtio_ring.c                 |  18 +++-
 Documentation/vfio.txt                       |   4 +-
 drivers/vfio/pci/Makefile                    |   1 +
 9 files changed, 190 insertions(+), 6 deletions(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_virtio.c

-- 
MST

[toc] | [next] | [standalone]

#1381444 — [PATCH RFC 1/3] virtio: add features for IOMMU control

From"Michael S. Tsirkin" <mst@redhat.com>
Date2016-04-18 12:00 +0200
Subject[PATCH RFC 1/3] virtio: add features for IOMMU control
Message-ID<rpe8j-3k6-17@gated-at.bofh.it>
In reply to#1381443
The interaction between virtio and DMA API is messy.

On most systems with virtio, physical addresses match bus addresses,
and it doesn't particularly matter whether we use the DMA API.

On some systems, including Xen and any system with a physical device
that speaks virtio behind a physical IOMMU, we must use the DMA API
for virtio DMA to work at all.

Add a feature bit to detect that: VIRTIO_F_IOMMU_PLATFORM.

On other systems, including SPARC and PPC64, virtio-pci devices are
enumerated as though they are behind an IOMMU, but the virtio host
ignores the IOMMU, so we must either pretend that the IOMMU isn't
there or somehow map everything as the identity.

Add a feature bit for that as well: VIRTIO_F_IOMMU_PASSTHROUGH: without
VIRTIO_F_IOMMU_PLATFORM, it means that virtio will bypass the IOMMU.
With VIRTIO_F_IOMMU_PLATFORM, it suggests that guest maps everything as
the identity (this last bit isn't trivial to implement so ignore this
hint for now).

If not there, we preserve historic behavior and bypass the DMA
API unless within Xen guest.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/uapi/linux/virtio_config.h | 10 +++++++++-
 drivers/virtio/virtio_ring.c       | 18 +++++++++++++++++-
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h
index 4cb65bb..952775c 100644
--- a/include/uapi/linux/virtio_config.h
+++ b/include/uapi/linux/virtio_config.h
@@ -49,7 +49,7 @@
  * transport being used (eg. virtio_ring), the rest are per-device feature
  * bits. */
 #define VIRTIO_TRANSPORT_F_START	28
-#define VIRTIO_TRANSPORT_F_END		33
+#define VIRTIO_TRANSPORT_F_END		35
 
 #ifndef VIRTIO_CONFIG_NO_LEGACY
 /* Do we get callbacks when the ring is completely used, even if we've
@@ -63,4 +63,12 @@
 /* v1.0 compliant. */
 #define VIRTIO_F_VERSION_1		32
 
+/* Request IOMMU passthrough (if available)
+ * Without VIRTIO_F_IOMMU_PLATFORM: bypass the IOMMU even if enabled.
+ * With VIRTIO_F_IOMMU_PLATFORM: suggest disabling IOMMU.
+ */
+#define VIRTIO_F_IOMMU_PASSTHROUGH	33
+
+/* Do not bypass the IOMMU (if configured) */
+#define VIRTIO_F_IOMMU_PLATFORM		34
 #endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5c802d4..0436bd2 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -117,7 +117,10 @@ struct vring_virtqueue {
 #define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
 
 /*
- * The interaction between virtio and a possible IOMMU is a mess.
+ * Modern virtio devices might set feature bits to specify whether
+ * they use or unconditionally bypass the platform IOMMU.
+ *
+ * If not there, the interaction between virtio and DMA API is messy.
  *
  * On most systems with virtio, physical addresses match bus addresses,
  * and it doesn't particularly matter whether we use the DMA API.
@@ -137,6 +140,13 @@ struct vring_virtqueue {
 
 static bool vring_use_dma_api(struct virtio_device *vdev)
 {
+	if (virtio_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM))
+		return true;
+
+	if (virtio_has_feature(vdev, VIRTIO_F_IOMMU_PASSTHROUGH))
+		return false;
+
+	/* Otherwise, we are left to guess. */
 	/*
 	 * In theory, it's possible to have a buggy QEMU-supposed
 	 * emulated Q35 IOMMU and Xen enabled at the same time.  On
@@ -1099,6 +1109,12 @@ void vring_transport_features(struct virtio_device *vdev)
 			break;
 		case VIRTIO_F_VERSION_1:
 			break;
+		case VIRTIO_F_IOMMU_PASSTHROUGH:
+			break;
+		case VIRTIO_F_IOMMU_PLATFORM:
+			/* Ignore passthrough hint for now, obey kernel config. */
+			__virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PASSTHROUGH);
+			break;
 		default:
 			/* We don't understand this bit. */
 			__virtio_clear_bit(vdev, i);
-- 
MST

[toc] | [prev] | [next] | [standalone]

#1381445 — [PATCH RFC 2/3] vfio: report group noiommu status

From"Michael S. Tsirkin" <mst@redhat.com>
Date2016-04-18 12:00 +0200
Subject[PATCH RFC 2/3] vfio: report group noiommu status
Message-ID<rpe8j-3k6-19@gated-at.bofh.it>
In reply to#1381443
When using vfio, callers might want to know whether device is added to a
regular group or an non-iommu group.

Report this status from vfio_add_group_dev.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 drivers/vfio/pci/vfio_pci.c                  | 2 +-
 drivers/vfio/platform/vfio_platform_common.c | 2 +-
 drivers/vfio/vfio.c                          | 5 ++++-
 Documentation/vfio.txt                       | 4 +++-
 4 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 712a849..d622a41 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -1119,7 +1119,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	spin_lock_init(&vdev->irqlock);
 
 	ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev);
-	if (ret) {
+	if (ret < 0) {
 		vfio_iommu_group_put(group, &pdev->dev);
 		kfree(vdev);
 		return ret;
diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
index e65b142..bf74e21 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -568,7 +568,7 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev,
 	}
 
 	ret = vfio_add_group_dev(dev, &vfio_platform_ops, vdev);
-	if (ret) {
+	if (ret < 0) {
 		iommu_group_put(group);
 		return ret;
 	}
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 6fd6fa5..67db231 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -756,6 +756,7 @@ int vfio_add_group_dev(struct device *dev,
 	struct iommu_group *iommu_group;
 	struct vfio_group *group;
 	struct vfio_device *device;
+	int noiommu;
 
 	iommu_group = iommu_group_get(dev);
 	if (!iommu_group)
@@ -791,6 +792,8 @@ int vfio_add_group_dev(struct device *dev,
 		return PTR_ERR(device);
 	}
 
+	noiommu = group->noiommu;
+
 	/*
 	 * Drop all but the vfio_device reference.  The vfio_device holds
 	 * a reference to the vfio_group, which holds a reference to the
@@ -798,7 +801,7 @@ int vfio_add_group_dev(struct device *dev,
 	 */
 	vfio_group_put(group);
 
-	return 0;
+	return noiommu;
 }
 EXPORT_SYMBOL_GPL(vfio_add_group_dev);
 
diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
index 1dd3fdd..d76be0f 100644
--- a/Documentation/vfio.txt
+++ b/Documentation/vfio.txt
@@ -259,7 +259,9 @@ extern void *vfio_del_group_dev(struct device *dev);
 
 vfio_add_group_dev() indicates to the core to begin tracking the
 specified iommu_group and register the specified dev as owned by
-a VFIO bus driver.  The driver provides an ops structure for callbacks
+a VFIO bus driver.  A negative return value indicates failure.
+A positive return value indicates that an unsafe noiommu mode
+is in use.  The driver provides an ops structure for callbacks
 similar to a file operations structure:
 
 struct vfio_device_ops {
-- 
MST

[toc] | [prev] | [next] | [standalone]

#1381974 — Re: [PATCH RFC 2/3] vfio: report group noiommu status

FromAlex Williamson <alex.williamson@redhat.com>
Date2016-04-18 21:00 +0200
SubjectRe: [PATCH RFC 2/3] vfio: report group noiommu status
Message-ID<rpmyS-1L1-7@gated-at.bofh.it>
In reply to#1381445
On Mon, 18 Apr 2016 12:58:20 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> When using vfio, callers might want to know whether device is added to a
> regular group or an non-iommu group.
> 
> Report this status from vfio_add_group_dev.
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---

What about making an interface to query this rather than playing games
with magic return values?

bool vfio_iommu_group_is_noiommu(struct iommu_group *group)
{
    return iommu_group_get_iommudata(group) == &noiommu;
}

>  drivers/vfio/pci/vfio_pci.c                  | 2 +-
>  drivers/vfio/platform/vfio_platform_common.c | 2 +-
>  drivers/vfio/vfio.c                          | 5 ++++-
>  Documentation/vfio.txt                       | 4 +++-
>  4 files changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> index 712a849..d622a41 100644
> --- a/drivers/vfio/pci/vfio_pci.c
> +++ b/drivers/vfio/pci/vfio_pci.c
> @@ -1119,7 +1119,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>  	spin_lock_init(&vdev->irqlock);
>  
>  	ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev);
> -	if (ret) {
> +	if (ret < 0) {
>  		vfio_iommu_group_put(group, &pdev->dev);
>  		kfree(vdev);
>  		return ret;
> diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
> index e65b142..bf74e21 100644
> --- a/drivers/vfio/platform/vfio_platform_common.c
> +++ b/drivers/vfio/platform/vfio_platform_common.c
> @@ -568,7 +568,7 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev,
>  	}
>  
>  	ret = vfio_add_group_dev(dev, &vfio_platform_ops, vdev);
> -	if (ret) {
> +	if (ret < 0) {
>  		iommu_group_put(group);
>  		return ret;
>  	}
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 6fd6fa5..67db231 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -756,6 +756,7 @@ int vfio_add_group_dev(struct device *dev,
>  	struct iommu_group *iommu_group;
>  	struct vfio_group *group;
>  	struct vfio_device *device;
> +	int noiommu;
>  
>  	iommu_group = iommu_group_get(dev);
>  	if (!iommu_group)
> @@ -791,6 +792,8 @@ int vfio_add_group_dev(struct device *dev,
>  		return PTR_ERR(device);
>  	}
>  
> +	noiommu = group->noiommu;
> +
>  	/*
>  	 * Drop all but the vfio_device reference.  The vfio_device holds
>  	 * a reference to the vfio_group, which holds a reference to the
> @@ -798,7 +801,7 @@ int vfio_add_group_dev(struct device *dev,
>  	 */
>  	vfio_group_put(group);
>  
> -	return 0;
> +	return noiommu;
>  }
>  EXPORT_SYMBOL_GPL(vfio_add_group_dev);
>  
> diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
> index 1dd3fdd..d76be0f 100644
> --- a/Documentation/vfio.txt
> +++ b/Documentation/vfio.txt
> @@ -259,7 +259,9 @@ extern void *vfio_del_group_dev(struct device *dev);
>  
>  vfio_add_group_dev() indicates to the core to begin tracking the
>  specified iommu_group and register the specified dev as owned by
> -a VFIO bus driver.  The driver provides an ops structure for callbacks
> +a VFIO bus driver.  A negative return value indicates failure.
> +A positive return value indicates that an unsafe noiommu mode
> +is in use.  The driver provides an ops structure for callbacks
>  similar to a file operations structure:
>  
>  struct vfio_device_ops {

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web