Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1240652 > unrolled thread

[PATCH v4 0/4] uio: add MSI/MSI-X support to uio_pci_generic driver

Back to article view | Back to linux.kernel

Contents

  [PATCH v4 0/4] uio: add MSI/MSI-X support to uio_pci_generic driver Vlad Zolotarov <vladz@cloudius-systems.com> - 2015-10-06 18:20 +0200
    [PATCH v4 4/4] Documentation: update uio-howto Vlad Zolotarov <vladz@cloudius-systems.com> - 2015-10-06 18:20 +0200
    [PATCH v4 1/4] uio: add ioctl support Vlad Zolotarov <vladz@cloudius-systems.com> - 2015-10-06 18:30 +0200

#1240652 — [PATCH v4 0/4] uio: add MSI/MSI-X support to uio_pci_generic driver

This series add support for MSI and MSI-X interrupts to uio_pci_generic driver.

Currently uio_pci_generic demands INT#x interrupts source be available. However
there are devices that simply don't have INT#x capability, for instance SR-IOV
VF devices that simply don't have INT#x capability. For such devices
uio_pci_generic will simply fail (more specifically its probe() will fail).

When IOMMU is either not available (e.g. Amazon EC2) or not acceptable due to
performance overhead and thus VFIO is not an option users that develop
user-space drivers are left without any option but to develop some proprietary
UIO drivers (e.g. igb_uio driver in Intel's DPDK) just to be able to use UIO
infrastructure.

This series provides a generic solution for this problem while preserving the
original behaviour for devices for which the original uio_pci_generic had worked
before (i.e. INT#x will be used by default).

New in v4:
   - Use portable __u32 and __s32 types from asm/types.h for
     defining uio_pci_generic_irq_set fields.
   - Use proper _IO macros for defining read and write ioctl()
     commands.
   - Moved bars mapping setting into a separate patch.
   - Update uio_pci_generic example in uio-howto.tmpl.

New in v3:
   - Add __iomem qualifier to temp buffer receiving ioremap value.  
 
New in v2:
   - Added #include <linux/uaccess.h> to uio_pci_generic.c


Vlad Zolotarov (4):
  uio: add ioctl support
  uio_pci_generic: properly initialize PCI bars mappings towards UIO
  uio_pci_generic: add MSI/MSI-X support
  Documentation: update uio-howto

 Documentation/DocBook/uio-howto.tmpl | 139 ++++++++++--
 drivers/uio/uio.c                    |  15 ++
 drivers/uio/uio_pci_generic.c        | 409 +++++++++++++++++++++++++++++++++--
 include/linux/uio_driver.h           |   3 +
 include/uapi/linux/uio_pci_generic.h |  51 +++++
 5 files changed, 574 insertions(+), 43 deletions(-)
 create mode 100644 include/uapi/linux/uio_pci_generic.h

-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1240654 — [PATCH v4 4/4] Documentation: update uio-howto

Change the chapters related to uio_pci_generic that refer interrupt mode.
Add the relevant explanation regarding MSI and MSI-X interrupt modes
support.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
---
New in v4:
   - Update uio_pci_generic example in uio-howto.tmpl
---
 Documentation/DocBook/uio-howto.tmpl | 133 ++++++++++++++++++++++++++++++-----
 1 file changed, 116 insertions(+), 17 deletions(-)

diff --git a/Documentation/DocBook/uio-howto.tmpl b/Documentation/DocBook/uio-howto.tmpl
index cd0e452..507b2ca 100644
--- a/Documentation/DocBook/uio-howto.tmpl
+++ b/Documentation/DocBook/uio-howto.tmpl
@@ -46,6 +46,12 @@ GPL version 2.
 
 <revhistory>
 	<revision>
+	<revnumber>0.10</revnumber>
+	<date>2015-10-04</date>
+	<authorinitials>vz</authorinitials>
+	<revremark>Added MSI and MSI-X support to uio_pci_generic.</revremark>
+	</revision>
+	<revision>
 	<revnumber>0.9</revnumber>
 	<date>2009-07-16</date>
 	<authorinitials>mst</authorinitials>
@@ -935,15 +941,32 @@ and look in the output for failure reasons
 <sect1 id="uio_pci_generic_internals">
 <title>Things to know about uio_pci_generic</title>
 	<para>
-Interrupts are handled using the Interrupt Disable bit in the PCI command
+Interrupts are handled either as MSI-X or MSI interrupts (if the device supports it) or
+as legacy INTx interrupts. By default INTx interrupts are used.
+	</para>
+	<para>
+uio_pci_generic automatically configures a device to use INTx interrupt for backward
+compatibility. If INTx are not available MSI-X interrupts will be used if the device
+supports it and if not MSI interrupts are going to be used. If none of the interrupts
+modes is supported probe() will fail.
+	</para>
+	<para>
+To get the used interrupt mode application has to use UIO_PCI_GENERIC_INT_MODE_GET ioctl
+command.
+UIO_PCI_GENERIC_IRQ_NUM_GET ioctl command may be used to get the total number of IRQs.
+Then UIO_PCI_GENERIC_IRQ_SET ioctl command may be used to bind a specific eventfd to a specific
+IRQ vector.
+	</para>
+	<para>
+Legacy interrupts are handled using the Interrupt Disable bit in the PCI command
 register and Interrupt Status bit in the PCI status register.  All devices
 compliant to PCI 2.3 (circa 2002) and all compliant PCI Express devices should
 support these bits.  uio_pci_generic detects this support, and won't bind to
 devices which do not support the Interrupt Disable Bit in the command register.
 	</para>
 	<para>
-On each interrupt, uio_pci_generic sets the Interrupt Disable bit.
-This prevents the device from generating further interrupts
+If legacy interrupts are used, uio_pci_generic sets the Interrupt Disable bit on
+each interrupt. This prevents the device from generating further interrupts
 until the bit is cleared. The userspace driver should clear this
 bit before blocking and waiting for more interrupts.
 	</para>
@@ -966,17 +989,23 @@ Here is some sample userspace driver code using uio_pci_generic:
 #include &lt;unistd.h&gt;
 #include &lt;sys/types.h&gt;
 #include &lt;sys/stat.h&gt;
+#include &lt;linux/uio_pci_generic.h&gt;
 #include &lt;fcntl.h&gt;
 #include &lt;errno.h&gt;
+#include &lt;sys/eventfd.h&gt;
+#include &lt;sys/ioctl.h&gt;
 
 int main()
 {
-	int uiofd;
+	int uiofd, event_fd;
 	int configfd;
 	int err;
 	int i;
 	unsigned icount;
+	__u64 read_buf;
 	unsigned char command_high;
+	__u32 int_mode, num_irqs;
+	int bytes_to_read;
 
 	uiofd = open(&quot;/dev/uio0&quot;, O_RDONLY);
 	if (uiofd &lt; 0) {
@@ -989,13 +1018,65 @@ int main()
 		return errno;
 	}
 
-	/* Read and cache command value */
-	err = pread(configfd, &amp;command_high, 1, 5);
-	if (err != 1) {
-		perror(&quot;command config read:&quot;);
+	/* Get the interrupt mode */
+	if (ioctl(uiofd, UIO_PCI_GENERIC_INT_MODE_GET, &amp;int_mode) < 0)
+		{ perror(&quot;getting interrupt mode&quot;); return errno;
+	}
+
+	/* Read the number of available IRQs - for INT#x and MSI 1 (one) will be returned */
+	if (ioctl(uiofd, UIO_PCI_GENERIC_IRQ_NUM_GET, &amp;num_irqs) < 0) {
+		perror(&quot;getting IRQs number&quot;);
 		return errno;
 	}
-	command_high &amp;= ~0x4;
+
+	/* if interrupt mode is MSI-X - allocate eventfd file descriptor and bind it to
+	 * the IRQ you need. We'll bind it to the first available IRQ in this example.
+	 */
+	if (int_mode == UIO_INT_MODE_MSIX) {
+		struct uio_pci_generic_irq_set irq_set;
+
+		if (num_irqs < 1) {
+			printf(&quot;Hmmm... Zero IRQ numbers. Something wrong with MSI-X configuration\n&quot;);
+			return -1;
+		}
+
+		printf(&quot;Interrupt mode is MSI-X, we are going to use IRQ[0]\n&quot;);
+
+		/* set up an eventfd for an interrupt */
+		event_fd = eventfd(0, EFD_CLOEXEC);
+		if (event_fd < 0) {
+			perror(&quot;cannot create irq eventfd&quot;);
+			return errno;
+		}
+
+		/* connect to the first IRQ */
+		irq_set.vec = 0;
+		irq_set.fd = event_fd;
+
+		if (ioctl(uiofd, UIO_PCI_GENERIC_IRQ_SET, &amp;irq_set) < 0) {
+			perror(&quot;binding the eventfd descriptor to IRQ[0]&quot;);
+			return errno;
+		}
+
+		/* eventfd read() requires to read 8 bytes, while UIO - requires 4 bytes */
+		bytes_to_read = 8;
+	} else if (int_mode == UIO_INT_MODE_MSI || int_mode == UIO_INT_MODE_INTX) {
+		event_fd = uiofd;
+		bytes_to_read = 4;
+
+		if (int_mode == UIO_INT_MODE_INTX) {
+			/* Read and cache command value */
+			err = pread(configfd, &amp;command_high, 1, 5);
+			if (err != 1) {
+				perror(&quot;command config read:&quot;);
+				return errno;
+			}
+			command_high &amp;= ~0x4;
+		}
+	} else {
+		printf(&quot;Interrupts are not supported\n&quot;);
+		return -1;
+	}
 
 	for(i = 0;; ++i) {
 		/* Print out a message, for debugging. */
@@ -1006,24 +1087,42 @@ int main()
 
 		/****************************************/
 		/* Here we got an interrupt from the
-		   device. Do something to it. */
+		   device. Do something to it and handle the HW
+		   interrupt state machine if needed. */
 		/****************************************/
 
-		/* Re-enable interrupts. */
-		err = pwrite(configfd, &amp;command_high, 1, 5);
-		if (err != 1) {
-			perror(&quot;config write:&quot;);
-			break;
+		if (int_mode == UIO_INT_MODE_INTX) {
+			/* Re-enable interrupts. */
+			err = pwrite(configfd, &amp;command_high, 1, 5);
+			if (err != 1) {
+				perror(&quot;config write:&quot;);
+				break;
+			}
 		}
 
 		/* Wait for next interrupt. */
-		err = read(uiofd, &amp;icount, 4);
-		if (err != 4) {
+		err = read(event_fd, &amp;read_buf, bytes_to_read);
+		if (err != bytes_to_read) {
 			perror(&quot;uio read:&quot;);
 			break;
 		}
 
+		icount++;
 	}
+
+	/* optional: unbind the eventfd from the IRQ */
+	if (int_mode == UIO_INT_MODE_MSIX) {
+		struct uio_pci_generic_irq_set irq_set = {
+			.vec = 0,
+			.fd = -1
+		};
+
+		if (ioctl(uiofd, UIO_PCI_GENERIC_IRQ_SET, &amp;irq_set) < 0) {
+			perror(&quot;unbinding the eventfd descriptor from IRQ[0]&quot;);
+			return errno;
+		}
+	}
+
 	return errno;
 }
 
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1240660 — [PATCH v4 1/4] uio: add ioctl support

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
---
 drivers/uio/uio.c          | 15 +++++++++++++++
 include/linux/uio_driver.h |  3 +++
 2 files changed, 18 insertions(+)

diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index 8196581..714b0e5 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -704,6 +704,20 @@ static int uio_mmap(struct file *filep, struct vm_area_struct *vma)
 	}
 }
 
+static long uio_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
+{
+	struct uio_listener *listener = filep->private_data;
+	struct uio_device *idev = listener->dev;
+
+	if (!idev->info)
+		return -EIO;
+
+	if (!idev->info->ioctl)
+		return -ENOTTY;
+
+	return idev->info->ioctl(idev->info, cmd, arg);
+}
+
 static const struct file_operations uio_fops = {
 	.owner		= THIS_MODULE,
 	.open		= uio_open,
@@ -712,6 +726,7 @@ static const struct file_operations uio_fops = {
 	.write		= uio_write,
 	.mmap		= uio_mmap,
 	.poll		= uio_poll,
+	.unlocked_ioctl	= uio_ioctl,
 	.fasync		= uio_fasync,
 	.llseek		= noop_llseek,
 };
diff --git a/include/linux/uio_driver.h b/include/linux/uio_driver.h
index 32c0e83..10d7833 100644
--- a/include/linux/uio_driver.h
+++ b/include/linux/uio_driver.h
@@ -89,6 +89,7 @@ struct uio_device {
  * @mmap:		mmap operation for this uio device
  * @open:		open operation for this uio device
  * @release:		release operation for this uio device
+ * @ioctl:		ioctl handler
  * @irqcontrol:		disable/enable irqs when 0/1 is written to /dev/uioX
  */
 struct uio_info {
@@ -105,6 +106,8 @@ struct uio_info {
 	int (*open)(struct uio_info *info, struct inode *inode);
 	int (*release)(struct uio_info *info, struct inode *inode);
 	int (*irqcontrol)(struct uio_info *info, s32 irq_on);
+	int (*ioctl)(struct uio_info *info, unsigned int cmd,
+		     unsigned long arg);
 };
 
 extern int __must_check
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web