Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1243604 > unrolled thread

[RFC PATCH 2/2] vfio: Include no-iommu mode

Started byAlex Williamson <alex.williamson@redhat.com>
First post2015-10-09 20:50 +0200
Last post2015-10-12 18:40 +0200
Articles 13 — 5 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  [RFC PATCH 2/2] vfio: Include no-iommu mode Alex Williamson <alex.williamson@redhat.com> - 2015-10-09 20:50 +0200
    Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Avi Kivity <avi@scylladb.com> - 2015-10-11 10:20 +0200
      Re: [RFC PATCH 2/2] vfio: Include no-iommu mode "Michael S. Tsirkin" <mst@redhat.com> - 2015-10-11 11:00 +0200
        Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Avi Kivity <avi@scylladb.com> - 2015-10-11 11:10 +0200
          Re: [RFC PATCH 2/2] vfio: Include no-iommu mode "Michael S. Tsirkin" <mst@redhat.com> - 2015-10-11 11:30 +0200
            Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Gleb Natapov <gleb@scylladb.com> - 2015-10-11 11:30 +0200
      Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Alex Williamson <alex.williamson@redhat.com> - 2015-10-11 23:20 +0200
    Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Stephen Hemminger <stephen@networkplumber.org> - 2015-10-12 18:00 +0200
      Re: [RFC PATCH 2/2] vfio: Include no-iommu mode "Michael S. Tsirkin" <mst@redhat.com> - 2015-10-12 18:30 +0200
        Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Alex Williamson <alex.williamson@redhat.com> - 2015-10-12 19:50 +0200
          Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Alex Williamson <alex.williamson@redhat.com> - 2015-10-12 20:10 +0200
      Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Alex Williamson <alex.williamson@redhat.com> - 2015-10-12 18:30 +0200
        Re: [RFC PATCH 2/2] vfio: Include no-iommu mode Avi Kivity <avi@scylladb.com> - 2015-10-12 18:40 +0200

#1243604 — [RFC PATCH 2/2] vfio: Include no-iommu mode

FromAlex Williamson <alex.williamson@redhat.com>
Date2015-10-09 20:50 +0200
Subject[RFC PATCH 2/2] vfio: Include no-iommu mode
Message-ID<qhKTU-231-19@gated-at.bofh.it>
There is really no way to safely give a user full access to a PCI
without an IOMMU to protect the host from errant DMA.  There is also
no way to provide DMA translation, for use cases such as devices
assignment to virtual machines.  However, there are still those users
that want userspace drivers under those conditions.  The UIO driver
exists for this use case, but does not provide the degree of device
access and programming that VFIO has.  In an effort to avoid code
duplication, this introduces a No-IOMMU mode for VFIO.

This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
module with the option "enable_unsafe_pci_noiommu_mode".  This should
make it very clear that this mode is not safe.  In this mode, there is
no support for unprivileged users, CAP_SYS_ADMIN is required for
access to the necessary dev files.  Mixing no-iommu and secure VFIO is
also unsupported, as are any VFIO IOMMU backends other than the
vfio-noiommu backend.  Furthermore, unsafe group files are relocated
to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
kernel is tainted due to the dummy IOMMU put in place.  Unloading of
the module in this mode is also unsupported and will BUG due to the
lack of support for unregistering an IOMMU for a bus type.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 drivers/vfio/Kconfig        |   15 +++
 drivers/vfio/Makefile       |    3 +
 drivers/vfio/vfio_core.c    |   40 +++++++++
 drivers/vfio/vfio_noiommu.c |  185 +++++++++++++++++++++++++++++++++++++++++++
 drivers/vfio/vfio_private.h |   31 +++++++
 include/uapi/linux/vfio.h   |    2 
 6 files changed, 276 insertions(+)
 create mode 100644 drivers/vfio/vfio_noiommu.c
 create mode 100644 drivers/vfio/vfio_private.h

diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig
index 4540179..929de0d 100644
--- a/drivers/vfio/Kconfig
+++ b/drivers/vfio/Kconfig
@@ -31,5 +31,20 @@ menuconfig VFIO
 
 	  If you don't know what to do here, say N.
 
+menuconfig VFIO_NOIOMMU
+	bool "VFIO No IOMMU support"
+	depends on VFIO
+	help
+	  VFIO is built on the ability to isolate devices using the IOMMU.
+	  Only with an IOMMU can userspace access to DMA capable devices be
+	  considered secure.  VFIO No IOMMU support enables a dummy IOMMU
+	  for the purpose of re-using the VFIO infrastructure in a non-secure
+	  mode.  Use of this mode will result in an unsupportable kernel and
+	  will therefore taint the kernel.  Device assignment to virtual
+	  machines is also not possible with this mode since the dummy IOMMU
+	  cannot provide DMA translation.
+
+	  If you don't know what to do here, say N.
+
 source "drivers/vfio/pci/Kconfig"
 source "drivers/vfio/platform/Kconfig"
diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile
index e8fa248..9736449 100644
--- a/drivers/vfio/Makefile
+++ b/drivers/vfio/Makefile
@@ -1,5 +1,8 @@
 vfio_virqfd-y := virqfd.o
 vfio-y := vfio_core.o
+ifdef CONFIG_VFIO_NOIOMMU
+vfio-y += vfio_noiommu.o
+endif
 
 obj-$(CONFIG_VFIO) += vfio.o
 obj-$(CONFIG_VFIO_VIRQFD) += vfio_virqfd.o
diff --git a/drivers/vfio/vfio_core.c b/drivers/vfio/vfio_core.c
index 563c510..495a490 100644
--- a/drivers/vfio/vfio_core.c
+++ b/drivers/vfio/vfio_core.c
@@ -34,10 +34,18 @@
 #include <linux/vfio.h>
 #include <linux/wait.h>
 
+#include "vfio_private.h"
+
 #define DRIVER_VERSION	"0.3"
 #define DRIVER_AUTHOR	"Alex Williamson <alex.williamson@redhat.com>"
 #define DRIVER_DESC	"VFIO - User Level meta-driver"
 
+static bool pci_noiommu __read_mostly;
+#ifdef CONFIG_VFIO_NOIOMMU
+module_param_named(enable_unsafe_pci_noiommu_mode, pci_noiommu, bool, S_IRUGO);
+MODULE_PARM_DESC(enable_unsafe_pci_noiommu_mode, "Enable UNSAFE, no-IOMMU mode for PCI.  This mode provides no device isolation, no DMA translation, no host kernel protection, cannot be used for device assignment to virtual machines, requires root permissions, and will taint the kernel.  If you do not know what this is for, step away. (default: false)");
+#endif
+
 static struct vfio {
 	struct class			*class;
 	struct list_head		iommu_drivers_list;
@@ -101,6 +109,9 @@ int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops)
 {
 	struct vfio_iommu_driver *driver, *tmp;
 
+	if (pci_noiommu && strcmp(ops->name, "vfio-noiommu"))
+		return -EPERM;
+
 	driver = kzalloc(sizeof(*driver), GFP_KERNEL);
 	if (!driver)
 		return -ENOMEM;
@@ -998,6 +1009,9 @@ static int vfio_fops_open(struct inode *inode, struct file *filep)
 {
 	struct vfio_container *container;
 
+	if (pci_noiommu && !capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
 	container = kzalloc(sizeof(*container), GFP_KERNEL);
 	if (!container)
 		return -ENOMEM;
@@ -1221,6 +1235,9 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
 	    !group->container->iommu_driver || !vfio_group_viable(group))
 		return -EINVAL;
 
+	if (pci_noiommu && !capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
 	device = vfio_device_get_from_name(group, buf);
 	if (!device)
 		return -ENODEV;
@@ -1347,6 +1364,9 @@ static int vfio_group_fops_open(struct inode *inode, struct file *filep)
 	struct vfio_group *group;
 	int opened;
 
+	if (pci_noiommu && !capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
 	group = vfio_group_get_from_minor(iminor(inode));
 	if (!group)
 		return -ENODEV;
@@ -1507,6 +1527,9 @@ struct vfio_group *vfio_group_get_external_user(struct file *filep)
 {
 	struct vfio_group *group = filep->private_data;
 
+	if (pci_noiommu)
+		return ERR_PTR(-EPERM);
+
 	if (filep->f_op != &vfio_group_fops)
 		return ERR_PTR(-EINVAL);
 
@@ -1549,6 +1572,9 @@ EXPORT_SYMBOL_GPL(vfio_external_check_extension);
  */
 static char *vfio_devnode(struct device *dev, umode_t *mode)
 {
+	if (pci_noiommu)
+		return kasprintf(GFP_KERNEL, "vfio-noiommu/%s", dev_name(dev));
+
 	return kasprintf(GFP_KERNEL, "vfio/%s", dev_name(dev));
 }
 
@@ -1564,6 +1590,12 @@ static int __init vfio_init(void)
 {
 	int ret;
 
+	if (pci_noiommu) {
+		ret = vfio_noiommu_iommu_init(&pci_bus_type);
+		if (ret)
+			return ret;
+	}
+
 	idr_init(&vfio.group_idr);
 	mutex_init(&vfio.group_lock);
 	mutex_init(&vfio.iommu_drivers_lock);
@@ -1605,6 +1637,9 @@ static int __init vfio_init(void)
 	request_module_nowait("vfio_iommu_type1");
 	request_module_nowait("vfio_iommu_spapr_tce");
 
+	if (pci_noiommu)
+		vfio_noiommu_init();
+
 	return 0;
 
 err_cdev_add:
@@ -1621,6 +1656,11 @@ static void __exit vfio_cleanup(void)
 {
 	WARN_ON(!list_empty(&vfio.group_list));
 
+	if (pci_noiommu) {
+		vfio_noiommu_cleanup();
+		vfio_noiommu_iommu_cleanup(&pci_bus_type);
+	}
+
 	idr_destroy(&vfio.group_idr);
 	cdev_del(&vfio.group_cdev);
 	unregister_chrdev_region(vfio.group_devt, MINORMASK);
diff --git a/drivers/vfio/vfio_noiommu.c b/drivers/vfio/vfio_noiommu.c
new file mode 100644
index 0000000..b12bb290
--- /dev/null
+++ b/drivers/vfio/vfio_noiommu.c
@@ -0,0 +1,185 @@
+/*
+ * VFIO - No IOMMU support
+ *
+ * Copyright (C) 2015 Red Hat, Inc.  All rights reserved.
+ *     Author: Alex Williamson <alex.williamson@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/device.h>
+#include <linux/iommu.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <linux/vfio.h>
+
+/*
+ * VFIO is fundamentally built on IOMMU groups but we have a lot of
+ * infrastructure for exposing devices to userspace.  There is no way to
+ * make userspace device access safe without IOMMU protection, but some
+ * users want to do it anyway.  Allow, but make it very, very clear that
+ * there's nothing safe about this.
+ */
+
+struct vfio_noiommu_domain {
+	struct iommu_domain domain;
+	/* mutex to avoid attach/detach race? */
+	struct device *dev;
+};
+
+static struct iommu_domain *vfio_noiommu_domain_alloc(unsigned type)
+{
+	if (type != IOMMU_DOMAIN_UNMANAGED)
+		return NULL;
+
+	return kzalloc(sizeof(struct vfio_noiommu_domain), GFP_KERNEL);
+}
+
+static void vfio_noiommu_domain_free(struct iommu_domain *domain)
+{
+	kfree(domain);
+}
+
+static int vfio_noiommu_attach_dev(struct iommu_domain *domain,
+				   struct device *dev)
+{
+	struct vfio_noiommu_domain *vdomain;
+
+	vdomain = container_of(domain, struct vfio_noiommu_domain, domain);
+
+	if (vdomain->dev)
+		return -EBUSY;
+
+	vdomain->dev = dev;
+	return 0;
+}
+
+static void vfio_noiommu_detach_dev(struct iommu_domain *domain,
+				    struct device *dev)
+{
+	struct vfio_noiommu_domain *vdomain;
+
+	vdomain = container_of(domain, struct vfio_noiommu_domain, domain);
+
+	vdomain->dev = NULL;
+}
+
+static int vfio_noiommu_add_device(struct device *dev)
+{
+	struct iommu_group *group = iommu_group_alloc();
+	int ret;
+
+	if (IS_ERR(group))
+		return PTR_ERR(group);
+
+	ret = iommu_group_add_device(group, dev);
+	iommu_group_put(group);
+	return ret;
+}
+
+static void vfio_noiommu_remove_device(struct device *dev)
+{
+	iommu_group_remove_device(dev);
+}
+
+static struct iommu_ops vfio_noiommu = {
+	.domain_alloc = vfio_noiommu_domain_alloc,
+	.domain_free = vfio_noiommu_domain_free,
+	.attach_dev = vfio_noiommu_attach_dev,
+	.detach_dev = vfio_noiommu_detach_dev,
+	.add_device = vfio_noiommu_add_device,
+	.remove_device = vfio_noiommu_remove_device,
+};
+
+int vfio_noiommu_iommu_init(struct bus_type *bus)
+{
+	int ret;
+
+	if (iommu_present(bus)) {
+		pr_warn("IOMMU present on bus %s, "
+			"cannot register vfio-noiommu\n", bus->name);
+		return -EBUSY;
+	}
+
+	ret = bus_set_iommu(bus, &vfio_noiommu);
+	if (!ret) {
+		add_taint(TAINT_USER, LOCKDEP_STILL_OK);
+		pr_warn("vfio-noiommu IOMMU driver registered on bus %s, "
+			"kernel tainted\n", bus->name);
+	}
+
+	return ret;
+}
+
+void vfio_noiommu_iommu_cleanup(struct bus_type *bus)
+{
+	BUG(); /* Unsetting an IOMMU from a bus is not supported */
+}
+
+static void *vfio_noiommu_open(unsigned long arg)
+{
+	if (arg != VFIO_NOIOMMU_IOMMU)
+		return ERR_PTR(-EINVAL);
+	if (!capable(CAP_SYS_ADMIN))
+		return ERR_PTR(-EPERM);
+
+	/* TODO: Should probably at least track pinnings */
+
+	return NULL;
+}
+
+static void vfio_noiommu_release(void *iommu_data)
+{
+	/* TODO: Track stuff an unpin */
+}
+
+static int vfio_noiommu_attach_group(void *iommu_data,
+				     struct iommu_group *iommu_group)
+{
+	return 0;
+}
+
+static void vfio_noiommu_detach_group(void *iommu_data,
+				      struct iommu_group *iommu_group)
+{
+}
+
+static long vfio_noiommu_ioctl(void *iommu_data,
+			       unsigned int cmd, unsigned long arg)
+{
+	if (cmd == VFIO_CHECK_EXTENSION) {
+		switch (arg) {
+		case VFIO_NOIOMMU_IOMMU:
+			return 1;
+		default:
+			return 0;
+		}
+	}
+
+	return -ENOTTY;
+}
+
+static struct vfio_iommu_driver_ops vfio_noiommu_ops = {
+	.name = "vfio-noiommu",
+	.owner = THIS_MODULE,
+	.open = vfio_noiommu_open,
+	.release = vfio_noiommu_release,
+	.ioctl = vfio_noiommu_ioctl,
+	.attach_group = vfio_noiommu_attach_group,
+	.detach_group = vfio_noiommu_detach_group,
+};
+
+int vfio_noiommu_init(void)
+{
+	return vfio_register_iommu_driver(&vfio_noiommu_ops);
+}
+
+void vfio_noiommu_cleanup(void)
+{
+	vfio_unregister_iommu_driver(&vfio_noiommu_ops);
+}
diff --git a/drivers/vfio/vfio_private.h b/drivers/vfio/vfio_private.h
new file mode 100644
index 0000000..6fe073d
--- /dev/null
+++ b/drivers/vfio/vfio_private.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2015 Red Hat, Inc.  All rights reserved
+ * Author: Alex Williamson <alex.williamson@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef VFIO_PRIVATE_H
+#define VFIO_PRIVATE_H
+
+#ifdef CONFIG_VFIO_NOIOMMU
+int vfio_noiommu_iommu_init(struct bus_type *bus);
+void vfio_noiommu_iommu_cleanup(struct bus_type *bus);
+int vfio_noiommu_init(void);
+void vfio_noiommu_cleanup(void);
+#else
+static inline int vfio_noiommu_iommu_init(struct bus_type *bus)
+{
+	return -ENODEV;
+}
+static inline void vfio_noiommu_iommu_cleanup(struct bus_type *bus) { }
+static inline int vfio_noiommu_init(void)
+{
+	return -ENODEV;
+}
+static inline void vfio_noiommu_cleanup(void) { }
+#endif
+#endif /* VFIO_PRIVATE_H */
+
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 9fd7b5d..c221614 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -38,6 +38,8 @@
 
 #define VFIO_SPAPR_TCE_v2_IOMMU		7
 
+#define VFIO_NOIOMMU_IOMMU		8
+
 /*
  * The IOCTL interface is designed for extensibility by embedding the
  * structure length (argsz) and flags into structures passed between

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]


#1244071

FromAvi Kivity <avi@scylladb.com>
Date2015-10-11 10:20 +0200
Message-ID<qik1k-2Ay-15@gated-at.bofh.it>
In reply to#1243604

On 10/09/2015 09:41 PM, Alex Williamson wrote:
> There is really no way to safely give a user full access to a PCI
> without an IOMMU to protect the host from errant DMA.  There is also
> no way to provide DMA translation, for use cases such as devices
> assignment to virtual machines.  However, there are still those users
> that want userspace drivers under those conditions.  The UIO driver
> exists for this use case, but does not provide the degree of device
> access and programming that VFIO has.  In an effort to avoid code
> duplication, this introduces a No-IOMMU mode for VFIO.
>
> This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> module with the option "enable_unsafe_pci_noiommu_mode".  This should
> make it very clear that this mode is not safe.  In this mode, there is
> no support for unprivileged users, CAP_SYS_ADMIN is required for
> access to the necessary dev files.

CAP_SYS_RAWIO seems a better match (in particular, it allows access to 
/dev/mem, which is the same thing).

>    Mixing no-iommu and secure VFIO is
> also unsupported, as are any VFIO IOMMU backends other than the
> vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> the module in this mode is also unsupported and will BUG due to the
> lack of support for unregistering an IOMMU for a bus type.

I did not see an API for detecting whether memory translation is 
provided or not.  We can have the caller guess this by looking at the 
device name, or by requiring the user to specify this, but I think it's 
cleaner to provide programmatic access to this attribute.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244074

From"Michael S. Tsirkin" <mst@redhat.com>
Date2015-10-11 11:00 +0200
Message-ID<qikE1-3lP-5@gated-at.bofh.it>
In reply to#1244071
On Sun, Oct 11, 2015 at 11:12:14AM +0300, Avi Kivity wrote:
> >   Mixing no-iommu and secure VFIO is
> >also unsupported, as are any VFIO IOMMU backends other than the
> >vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> >to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> >kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> >the module in this mode is also unsupported and will BUG due to the
> >lack of support for unregistering an IOMMU for a bus type.
> 
> I did not see an API for detecting whether memory translation is provided or
> not.  We can have the caller guess this by looking at the device name, or by
> requiring the user to specify this, but I think it's cleaner to provide
> programmatic access to this attribute.

It seems that caller can just check for VFIO_NOIOMMU_IOMMU.

Isn't this why it's there?

VFIO_IOMMU_MAP_DMA, VFIO_IOMMU_ENABLE and VFIO_IOMMU_DISABLE
will probably also fail ...

-- 
MST
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244075

FromAvi Kivity <avi@scylladb.com>
Date2015-10-11 11:10 +0200
Message-ID<qikNH-3Mq-5@gated-at.bofh.it>
In reply to#1244074

On 10/11/2015 11:57 AM, Michael S. Tsirkin wrote:
> On Sun, Oct 11, 2015 at 11:12:14AM +0300, Avi Kivity wrote:
>>>    Mixing no-iommu and secure VFIO is
>>> also unsupported, as are any VFIO IOMMU backends other than the
>>> vfio-noiommu backend.  Furthermore, unsafe group files are relocated
>>> to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
>>> kernel is tainted due to the dummy IOMMU put in place.  Unloading of
>>> the module in this mode is also unsupported and will BUG due to the
>>> lack of support for unregistering an IOMMU for a bus type.
>> I did not see an API for detecting whether memory translation is provided or
>> not.  We can have the caller guess this by looking at the device name, or by
>> requiring the user to specify this, but I think it's cleaner to provide
>> programmatic access to this attribute.
> It seems that caller can just check for VFIO_NOIOMMU_IOMMU.
>
> Isn't this why it's there?

That's just means the capability is there, not that it's active.

But since you must pass the same value to open(), you already know that 
you're using noiommu.

> VFIO_IOMMU_MAP_DMA, VFIO_IOMMU_ENABLE and VFIO_IOMMU_DISABLE
> will probably also fail ...
>

Don't you have to call MAP_DMA to pin the memory?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244083

From"Michael S. Tsirkin" <mst@redhat.com>
Date2015-10-11 11:30 +0200
Message-ID<qil74-49b-3@gated-at.bofh.it>
In reply to#1244075
On Sun, Oct 11, 2015 at 12:03:17PM +0300, Avi Kivity wrote:
> 
> 
> On 10/11/2015 11:57 AM, Michael S. Tsirkin wrote:
> >On Sun, Oct 11, 2015 at 11:12:14AM +0300, Avi Kivity wrote:
> >>>   Mixing no-iommu and secure VFIO is
> >>>also unsupported, as are any VFIO IOMMU backends other than the
> >>>vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> >>>to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> >>>kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> >>>the module in this mode is also unsupported and will BUG due to the
> >>>lack of support for unregistering an IOMMU for a bus type.
> >>I did not see an API for detecting whether memory translation is provided or
> >>not.  We can have the caller guess this by looking at the device name, or by
> >>requiring the user to specify this, but I think it's cleaner to provide
> >>programmatic access to this attribute.
> >It seems that caller can just check for VFIO_NOIOMMU_IOMMU.
> >
> >Isn't this why it's there?
> 
> That's just means the capability is there, not that it's active.

Well it's currently exactly the same.
I guess you can check the return value of VFIO_SET_IOMMU as well.

> But since you must pass the same value to open(), you already know that
> you're using noiommu.
> 
> >VFIO_IOMMU_MAP_DMA, VFIO_IOMMU_ENABLE and VFIO_IOMMU_DISABLE
> >will probably also fail ...
> >
> 
> Don't you have to call MAP_DMA to pin the memory?

Well check it out - the patch in question doesn't implement this ioctl.
In fact it doesn't implement anything except CHECK_EXTENSION.

And this makes sense to me: MAP_DMA
maps a virtual address to io address, and that doesn't
work for the dummy iommu.

You can pin memory using many other ways, including
mlock and hugetlbfs.

-- 
MST
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244085

FromGleb Natapov <gleb@scylladb.com>
Date2015-10-11 11:30 +0200
Message-ID<qil74-49b-11@gated-at.bofh.it>
In reply to#1244083
On Sun, Oct 11, 2015 at 12:19:54PM +0300, Michael S. Tsirkin wrote:
> > But since you must pass the same value to open(), you already know that
> > you're using noiommu.
> > 
> > >VFIO_IOMMU_MAP_DMA, VFIO_IOMMU_ENABLE and VFIO_IOMMU_DISABLE
> > >will probably also fail ...
> > >
> > 
> > Don't you have to call MAP_DMA to pin the memory?
> 
> Well check it out - the patch in question doesn't implement this ioctl.
> In fact it doesn't implement anything except CHECK_EXTENSION.
> 
> And this makes sense to me: MAP_DMA
> maps a virtual address to io address, and that doesn't
> work for the dummy iommu.
> 
> You can pin memory using many other ways, including
> mlock and hugetlbfs.
> 
mlock() does not pin memory.

--
			Gleb.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244228

FromAlex Williamson <alex.williamson@redhat.com>
Date2015-10-11 23:20 +0200
Message-ID<qiwc9-3ta-9@gated-at.bofh.it>
In reply to#1244071
On Sun, 2015-10-11 at 11:12 +0300, Avi Kivity wrote:
> 
> On 10/09/2015 09:41 PM, Alex Williamson wrote:
> > There is really no way to safely give a user full access to a PCI
> > without an IOMMU to protect the host from errant DMA.  There is also
> > no way to provide DMA translation, for use cases such as devices
> > assignment to virtual machines.  However, there are still those users
> > that want userspace drivers under those conditions.  The UIO driver
> > exists for this use case, but does not provide the degree of device
> > access and programming that VFIO has.  In an effort to avoid code
> > duplication, this introduces a No-IOMMU mode for VFIO.
> >
> > This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> > module with the option "enable_unsafe_pci_noiommu_mode".  This should
> > make it very clear that this mode is not safe.  In this mode, there is
> > no support for unprivileged users, CAP_SYS_ADMIN is required for
> > access to the necessary dev files.
> 
> CAP_SYS_RAWIO seems a better match (in particular, it allows access to 
> /dev/mem, which is the same thing).

Sure, that seems reasonable.

> >    Mixing no-iommu and secure VFIO is
> > also unsupported, as are any VFIO IOMMU backends other than the
> > vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> > to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> > kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> > the module in this mode is also unsupported and will BUG due to the
> > lack of support for unregistering an IOMMU for a bus type.
> 
> I did not see an API for detecting whether memory translation is 
> provided or not.  We can have the caller guess this by looking at the 
> device name, or by requiring the user to specify this, but I think it's 
> cleaner to provide programmatic access to this attribute.

The VFIO user can probe and needs to set the IOMMU model in use before
they can access a device file descriptor.  In this mode, the
VFIO_NOIOMMU_IOMMU is the only model available, which as proposed here
provides no translation, and in fact no mapping ioctls.  Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244854

FromStephen Hemminger <stephen@networkplumber.org>
Date2015-10-12 18:00 +0200
Message-ID<qiNG3-3wf-29@gated-at.bofh.it>
In reply to#1243604
On Fri, 09 Oct 2015 12:41:10 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:

> There is really no way to safely give a user full access to a PCI
> without an IOMMU to protect the host from errant DMA.  There is also
> no way to provide DMA translation, for use cases such as devices
> assignment to virtual machines.  However, there are still those users
> that want userspace drivers under those conditions.  The UIO driver
> exists for this use case, but does not provide the degree of device
> access and programming that VFIO has.  In an effort to avoid code
> duplication, this introduces a No-IOMMU mode for VFIO.
> 
> This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> module with the option "enable_unsafe_pci_noiommu_mode".  This should
> make it very clear that this mode is not safe.  In this mode, there is
> no support for unprivileged users, CAP_SYS_ADMIN is required for
> access to the necessary dev files.  Mixing no-iommu and secure VFIO is
> also unsupported, as are any VFIO IOMMU backends other than the
> vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> the module in this mode is also unsupported and will BUG due to the
> lack of support for unregistering an IOMMU for a bus type.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Will this work for distro's where chaning kernel command line options
is really not that practical. We need to boot with one command line
and then decide to use IOMMU (or not) later on during the service
startup of the dataplane application. Recent experience is that IOMMU's
are broken on many platforms so the only way to make a DPDK application
it to write a test program that can be used to check if VFIO+IOMMU
works first.

Also, although you think the long option will set the bar high
enough it probably will not satisfy anyone. It is annoying enough, that
I would just carry a patch to remove it the silly requirement.
And the the people who believe
all user mode DMA is evil won't be satisfied either.

But I really like having the same consistent API for handling device
access with IOMMU and when IOMMU will/won't work.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244873

From"Michael S. Tsirkin" <mst@redhat.com>
Date2015-10-12 18:30 +0200
Message-ID<qiO94-4kn-7@gated-at.bofh.it>
In reply to#1244854
On Mon, Oct 12, 2015 at 08:56:07AM -0700, Stephen Hemminger wrote:
> On Fri, 09 Oct 2015 12:41:10 -0600
> Alex Williamson <alex.williamson@redhat.com> wrote:
> 
> > There is really no way to safely give a user full access to a PCI
> > without an IOMMU to protect the host from errant DMA.  There is also
> > no way to provide DMA translation, for use cases such as devices
> > assignment to virtual machines.  However, there are still those users
> > that want userspace drivers under those conditions.  The UIO driver
> > exists for this use case, but does not provide the degree of device
> > access and programming that VFIO has.  In an effort to avoid code
> > duplication, this introduces a No-IOMMU mode for VFIO.
> > 
> > This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> > module with the option "enable_unsafe_pci_noiommu_mode".  This should
> > make it very clear that this mode is not safe.  In this mode, there is
> > no support for unprivileged users, CAP_SYS_ADMIN is required for
> > access to the necessary dev files.  Mixing no-iommu and secure VFIO is
> > also unsupported, as are any VFIO IOMMU backends other than the
> > vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> > to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> > kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> > the module in this mode is also unsupported and will BUG due to the
> > lack of support for unregistering an IOMMU for a bus type.
> > 
> > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> 
> Will this work for distro's where chaning kernel command line options
> is really not that practical. We need to boot with one command line
> and then decide to use IOMMU (or not) later on during the service
> startup of the dataplane application.

On open? That's too late in my opinion. But maybe the flag can be
tweaked so that it will probe for iommu, if there - do the
right thing, but if that fails, enable the dummy one.
And maybe defer tainting until device open.

Won't address the "old IOMMUs add performance overhead"
usecase but I'm not very impressed by that in any case.

> Recent experience is that IOMMU's
> are broken on many platforms so the only way to make a DPDK application
> it to write a test program that can be used to check if VFIO+IOMMU
> works first.

In userspace? Well that's just piling up work-arounds.  And assuming
hardware is broken, who knows what's going on security-wise.  These
broken systems need to be identified and black-listed in kernel.

> Also, although you think the long option will set the bar high
> enough it probably will not satisfy anyone. It is annoying enough, that
> I would just carry a patch to remove it the silly requirement.

That sounds reasonable. Anyone who can carry a kernel patch
does not need the warning.

> And the the people who believe
> all user mode DMA is evil won't be satisfied either.
> But I really like having the same consistent API for handling device
> access with IOMMU and when IOMMU will/won't work.

I agree that's good. Makes it easier to migrate applications to
the safe configuration down the road. Thanks Alex!

-- 
mST
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244952

FromAlex Williamson <alex.williamson@redhat.com>
Date2015-10-12 19:50 +0200
Message-ID<qiPou-64e-9@gated-at.bofh.it>
In reply to#1244873
On Mon, 2015-10-12 at 19:27 +0300, Michael S. Tsirkin wrote:
> On Mon, Oct 12, 2015 at 08:56:07AM -0700, Stephen Hemminger wrote:
> > On Fri, 09 Oct 2015 12:41:10 -0600
> > Alex Williamson <alex.williamson@redhat.com> wrote:
> > 
> > > There is really no way to safely give a user full access to a PCI
> > > without an IOMMU to protect the host from errant DMA.  There is also
> > > no way to provide DMA translation, for use cases such as devices
> > > assignment to virtual machines.  However, there are still those users
> > > that want userspace drivers under those conditions.  The UIO driver
> > > exists for this use case, but does not provide the degree of device
> > > access and programming that VFIO has.  In an effort to avoid code
> > > duplication, this introduces a No-IOMMU mode for VFIO.
> > > 
> > > This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> > > module with the option "enable_unsafe_pci_noiommu_mode".  This should
> > > make it very clear that this mode is not safe.  In this mode, there is
> > > no support for unprivileged users, CAP_SYS_ADMIN is required for
> > > access to the necessary dev files.  Mixing no-iommu and secure VFIO is
> > > also unsupported, as are any VFIO IOMMU backends other than the
> > > vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> > > to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> > > kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> > > the module in this mode is also unsupported and will BUG due to the
> > > lack of support for unregistering an IOMMU for a bus type.
> > > 
> > > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> > 
> > Will this work for distro's where chaning kernel command line options
> > is really not that practical. We need to boot with one command line
> > and then decide to use IOMMU (or not) later on during the service
> > startup of the dataplane application.
> 
> On open? That's too late in my opinion. But maybe the flag can be
> tweaked so that it will probe for iommu, if there - do the
> right thing, but if that fails, enable the dummy one.
> And maybe defer tainting until device open.

The vfio mechanics are that a vfio bus driver, such as vfio-pci binds to
a device.  In the probe function, we check for an iommu group, which
vfio-core then uses to create the vfio group.  So there's nothing to
open(), the iommu association needs to be made prior to even binding the
device to vfio-pci.  Probing for an iommu can also only be done on a per
bus_type basis, which will likely eventually become a per bus instance
to support heterogeneous iommus, so vfio can't simply determine that an
iommu is not present globally.  This is why the new module option
includes the word "pci", so that it can probe for and attach the dummy
iommu specifically on the pci_bus_type.

We can still consider if there are better points at which to initiate
the fake iommu group.  Trying to think through vfio-pci doing it on
probe(), but it seems pretty ugly.

In this RFC, I specifically avoided making the vfio no-iommu iommu
driver just another modular iommu backend, I wanted it to be tied to a
vfio module option such that vfio behaves differently with open()s and
certain ioctls.  I think it would be really confusing to users if safe
and unsafe modes could be used concurrently for different devices.

> Won't address the "old IOMMUs add performance overhead"
> usecase but I'm not very impressed by that in any case.

Yep, me neither, certainly not for static mappings.  There's a lot of
FUD left over from latencies in the streaming DMA mapping paths where
mappings are created and destroyed at a high rate.  That has more to do
with flushing mappings out of the hardware than with iotlb miss latency
or actual translation, which is all that should be in play for most uses
here.

> > Recent experience is that IOMMU's
> > are broken on many platforms so the only way to make a DPDK application
> > it to write a test program that can be used to check if VFIO+IOMMU
> > works first.
> 
> In userspace? Well that's just piling up work-arounds.  And assuming
> hardware is broken, who knows what's going on security-wise.  These
> broken systems need to be identified and black-listed in kernel.
> 
> > Also, although you think the long option will set the bar high
> > enough it probably will not satisfy anyone. It is annoying enough, that
> > I would just carry a patch to remove it the silly requirement.
> 
> That sounds reasonable. Anyone who can carry a kernel patch
> does not need the warning.
> 
> > And the the people who believe
> > all user mode DMA is evil won't be satisfied either.
> > But I really like having the same consistent API for handling device
> > access with IOMMU and when IOMMU will/won't work.
> 
> I agree that's good. Makes it easier to migrate applications to
> the safe configuration down the road. Thanks Alex!
> 



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244976

FromAlex Williamson <alex.williamson@redhat.com>
Date2015-10-12 20:10 +0200
Message-ID<qiPHQ-6Go-17@gated-at.bofh.it>
In reply to#1244952
On Mon, 2015-10-12 at 11:46 -0600, Alex Williamson wrote:
> On Mon, 2015-10-12 at 19:27 +0300, Michael S. Tsirkin wrote:
> > On Mon, Oct 12, 2015 at 08:56:07AM -0700, Stephen Hemminger wrote:
> > > On Fri, 09 Oct 2015 12:41:10 -0600
> > > Alex Williamson <alex.williamson@redhat.com> wrote:
> > > 
> > > > There is really no way to safely give a user full access to a PCI
> > > > without an IOMMU to protect the host from errant DMA.  There is also
> > > > no way to provide DMA translation, for use cases such as devices
> > > > assignment to virtual machines.  However, there are still those users
> > > > that want userspace drivers under those conditions.  The UIO driver
> > > > exists for this use case, but does not provide the degree of device
> > > > access and programming that VFIO has.  In an effort to avoid code
> > > > duplication, this introduces a No-IOMMU mode for VFIO.
> > > > 
> > > > This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> > > > module with the option "enable_unsafe_pci_noiommu_mode".  This should
> > > > make it very clear that this mode is not safe.  In this mode, there is
> > > > no support for unprivileged users, CAP_SYS_ADMIN is required for
> > > > access to the necessary dev files.  Mixing no-iommu and secure VFIO is
> > > > also unsupported, as are any VFIO IOMMU backends other than the
> > > > vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> > > > to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> > > > kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> > > > the module in this mode is also unsupported and will BUG due to the
> > > > lack of support for unregistering an IOMMU for a bus type.
> > > > 
> > > > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> > > 
> > > Will this work for distro's where chaning kernel command line options
> > > is really not that practical. We need to boot with one command line
> > > and then decide to use IOMMU (or not) later on during the service
> > > startup of the dataplane application.
> > 
> > On open? That's too late in my opinion. But maybe the flag can be
> > tweaked so that it will probe for iommu, if there - do the
> > right thing, but if that fails, enable the dummy one.
> > And maybe defer tainting until device open.

I forgot to address the tainting point; I think we were previously
talking about tainting at the point where bus master is enabled, but I
chose to do it much earlier here because the act of registering a dummy
iommu_ops for a bus type is pretty much the point at which we have a
good chance of breaking the system.  I also considered that some devices
can manipulate their config space registers using device specific
registers (such as the example of GPUs that mirror config space in
mmio).  It's therefore not always possible to taint at the point where
we think the user has done something bad.  The best case would be
tainting at the point where the device file descriptor is opened as you
suggested, but we can't do that while we're exposing dummy iommu_ops to
the whole bus type.  Maybe another option would be to create vfio
wrappers for the iommu callbacks to have the iommu facade more local to
vfio.

My main requirements are that I do not want be disruptive to the
existing vfio code or add a significant amount of code that needs to be
maintained for the purpose of supporting a use mode that we don't really
think is supportable.  Thanks,

Alex

> The vfio mechanics are that a vfio bus driver, such as vfio-pci binds to
> a device.  In the probe function, we check for an iommu group, which
> vfio-core then uses to create the vfio group.  So there's nothing to
> open(), the iommu association needs to be made prior to even binding the
> device to vfio-pci.  Probing for an iommu can also only be done on a per
> bus_type basis, which will likely eventually become a per bus instance
> to support heterogeneous iommus, so vfio can't simply determine that an
> iommu is not present globally.  This is why the new module option
> includes the word "pci", so that it can probe for and attach the dummy
> iommu specifically on the pci_bus_type.
> 
> We can still consider if there are better points at which to initiate
> the fake iommu group.  Trying to think through vfio-pci doing it on
> probe(), but it seems pretty ugly.
> 
> In this RFC, I specifically avoided making the vfio no-iommu iommu
> driver just another modular iommu backend, I wanted it to be tied to a
> vfio module option such that vfio behaves differently with open()s and
> certain ioctls.  I think it would be really confusing to users if safe
> and unsafe modes could be used concurrently for different devices.
> 
> > Won't address the "old IOMMUs add performance overhead"
> > usecase but I'm not very impressed by that in any case.
> 
> Yep, me neither, certainly not for static mappings.  There's a lot of
> FUD left over from latencies in the streaming DMA mapping paths where
> mappings are created and destroyed at a high rate.  That has more to do
> with flushing mappings out of the hardware than with iotlb miss latency
> or actual translation, which is all that should be in play for most uses
> here.
> 
> > > Recent experience is that IOMMU's
> > > are broken on many platforms so the only way to make a DPDK application
> > > it to write a test program that can be used to check if VFIO+IOMMU
> > > works first.
> > 
> > In userspace? Well that's just piling up work-arounds.  And assuming
> > hardware is broken, who knows what's going on security-wise.  These
> > broken systems need to be identified and black-listed in kernel.
> > 
> > > Also, although you think the long option will set the bar high
> > > enough it probably will not satisfy anyone. It is annoying enough, that
> > > I would just carry a patch to remove it the silly requirement.
> > 
> > That sounds reasonable. Anyone who can carry a kernel patch
> > does not need the warning.
> > 
> > > And the the people who believe
> > > all user mode DMA is evil won't be satisfied either.
> > > But I really like having the same consistent API for handling device
> > > access with IOMMU and when IOMMU will/won't work.
> > 
> > I agree that's good. Makes it easier to migrate applications to
> > the safe configuration down the road. Thanks Alex!
> > 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244885

FromAlex Williamson <alex.williamson@redhat.com>
Date2015-10-12 18:30 +0200
Message-ID<qiO95-4kn-41@gated-at.bofh.it>
In reply to#1244854
On Mon, 2015-10-12 at 08:56 -0700, Stephen Hemminger wrote:
> On Fri, 09 Oct 2015 12:41:10 -0600
> Alex Williamson <alex.williamson@redhat.com> wrote:
> 
> > There is really no way to safely give a user full access to a PCI
> > without an IOMMU to protect the host from errant DMA.  There is also
> > no way to provide DMA translation, for use cases such as devices
> > assignment to virtual machines.  However, there are still those users
> > that want userspace drivers under those conditions.  The UIO driver
> > exists for this use case, but does not provide the degree of device
> > access and programming that VFIO has.  In an effort to avoid code
> > duplication, this introduces a No-IOMMU mode for VFIO.
> > 
> > This mode requires enabling CONFIG_VFIO_NOIOMMU and loading the vfio
> > module with the option "enable_unsafe_pci_noiommu_mode".  This should
> > make it very clear that this mode is not safe.  In this mode, there is
> > no support for unprivileged users, CAP_SYS_ADMIN is required for
> > access to the necessary dev files.  Mixing no-iommu and secure VFIO is
> > also unsupported, as are any VFIO IOMMU backends other than the
> > vfio-noiommu backend.  Furthermore, unsafe group files are relocated
> > to /dev/vfio-noiommu/.  Upon successful loading in this mode, the
> > kernel is tainted due to the dummy IOMMU put in place.  Unloading of
> > the module in this mode is also unsupported and will BUG due to the
> > lack of support for unregistering an IOMMU for a bus type.
> > 
> > Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> 
> Will this work for distro's where chaning kernel command line options
> is really not that practical. We need to boot with one command line
> and then decide to use IOMMU (or not) later on during the service
> startup of the dataplane application. Recent experience is that IOMMU's
> are broken on many platforms so the only way to make a DPDK application
> it to write a test program that can be used to check if VFIO+IOMMU
> works first.

There's no kernel command line dependency, if you find that VFIO+IOMMU
doesn't work, unload the vfio modules and reload with the no-iommu
option and try again.  VFIO itself cannot simply fall back to no-iommu,
that definitely needs to be a user opt-in.  The userspace app though
should easily be able to tell when the type1 model is not available and
fall back to no-iommu.

> Also, although you think the long option will set the bar high
> enough it probably will not satisfy anyone. It is annoying enough, that
> I would just carry a patch to remove it the silly requirement.
> And the the people who believe
> all user mode DMA is evil won't be satisfied either.

I find that many users blindly follow howtos and only sometimes do they
question the options if they sound scary enough.  So yeah, I would
intend to make the option upstream sound scary enough for people to
think twice about using it and maybe even read the description.  That
still doesn't prevent pasting it into modprobe.d and forgetting about
it.

I don't see non-IOMMU protected usermode DMA as evil, it's just
unsupportable and I want to be able to immediately know that it's a
possibility if I'm looking at a kernel bug.

> But I really like having the same consistent API for handling device
> access with IOMMU and when IOMMU will/won't work.

We still need to hear from IOMMU folks, AIUI there are plans to
implement dma_iops via iommu_ops, which would make squatting on
iommu_ops much more difficult for a hack like this.  Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1244905

FromAvi Kivity <avi@scylladb.com>
Date2015-10-12 18:40 +0200
Message-ID<qiOiL-4vG-39@gated-at.bofh.it>
In reply to#1244885

On 10/12/2015 07:23 PM, Alex Williamson wrote:
>> Also, although you think the long option will set the bar high
>> enough it probably will not satisfy anyone. It is annoying enough, that
>> I would just carry a patch to remove it the silly requirement.
>> And the the people who believe
>> all user mode DMA is evil won't be satisfied either.
> I find that many users blindly follow howtos and only sometimes do they
> question the options if they sound scary enough.  So yeah, I would
> intend to make the option upstream sound scary enough for people to
> think twice about using it and maybe even read the description.  That
> still doesn't prevent pasting it into modprobe.d and forgetting about
> it.

I think we need to allow for packages to work with this.  I.e. drop 
files into config directories rather than require editing of config 
files.  I think this is mostly workable via modprobe.d.

(for our own use case, we don't require the extreme performance of many 
L2/L3 dpdk apps so we'll work with regular iommued vfio for bare-metal 
installations and ship prebuilt virtual machine images for clouds, so it 
doesn't matter that much to me).

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]


Back to top | Article view | linux.kernel


csiph-web