Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1509844 > unrolled thread

[PATCH v10 00/19] Add Mediated device support

Back to article view | Back to linux.kernel

Contents

  [PATCH v10 00/19] Add Mediated device support Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:30 +0200
    [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:30 +0200
      Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn() Jike Song <jike.song@intel.com> - 2016-10-27 16:00 +0200
        Re: [PATCH v10 07/19] vfio iommu type1: Update argument of  vaddr_get_pfn() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-27 17:50 +0200
          Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn() Jike Song <jike.song@intel.com> - 2016-10-28 08:10 +0200
            Re: [PATCH v10 07/19] vfio iommu type1: Update argument of  vaddr_get_pfn() Kirti Wankhede <kwankhede@nvidia.com> - 2016-11-02 09:10 +0100
    [PATCH v10 01/19] vfio: Mediated device Core driver Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:30 +0200
      Re: [PATCH v10 01/19] vfio: Mediated device Core driver Jike Song <jike.song@intel.com> - 2016-10-29 06:40 +0200
        Re: [PATCH v10 01/19] vfio: Mediated device Core driver Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-29 12:10 +0200
          Re: [PATCH v10 01/19] vfio: Mediated device Core driver Jike Song <jike.song@intel.com> - 2016-10-29 20:20 +0200
            Re: [PATCH v10 01/19] vfio: Mediated device Core driver Kirti Wankhede <kwankhede@nvidia.com> - 2016-11-02 09:10 +0100
              Re: [Qemu-devel] [PATCH v10 01/19] vfio: Mediated device Core driver Jike Song <jike.song@intel.com> - 2016-11-02 11:40 +0100
      Re: [PATCH v10 01/19] vfio: Mediated device Core driver Jike Song <jike.song@intel.com> - 2016-11-01 04:20 +0100
        Re: [PATCH v10 01/19] vfio: Mediated device Core driver Alex Williamson <alex.williamson@redhat.com> - 2016-11-01 04:50 +0100
          Re: [PATCH v10 01/19] vfio: Mediated device Core driver Jike Song <jike.song@intel.com> - 2016-11-01 06:40 +0100
    [PATCH v10 17/19] docs: Add Documentation for Mediated devices Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 03/19] vfio: Rearrange functions to get vfio_group from dev Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
      Re: [PATCH v10 03/19] vfio: Rearrange functions to get vfio_group  from dev Jike Song <jike.song@intel.com> - 2016-11-02 11:50 +0100
    [PATCH v10 18/19] docs: Sysfs ABI for mediated device framework Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
      Re: [PATCH v10 18/19] docs: Sysfs ABI for mediated device framework Jike Song <jike.song@intel.com> - 2016-10-31 08:30 +0100
        Re: [PATCH v10 18/19] docs: Sysfs ABI for mediated device framework Kirti Wankhede <kwankhede@nvidia.com> - 2016-11-02 09:00 +0100
    [PATCH v10 09/19] vfio iommu type1: Add support for mediated devices Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
      Re: [PATCH v10 09/19] vfio iommu type1: Add support for mediated  devices Alex Williamson <alex.williamson@redhat.com> - 2016-10-28 01:10 +0200
      Re: [PATCH v10 09/19] vfio iommu type1: Add support for mediated  devices Jike Song <jike.song@intel.com> - 2016-11-02 14:40 +0100
    [PATCH v10 16/19] vfio: Define device_api strings Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 14/19] vfio_pci: Updated to use vfio_set_irqs_validate_and_prepare() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 15/19] vfio_platform: Updated to use vfio_set_irqs_validate_and_prepare() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 11/19] vfio: Introduce common function to add capabilities Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 06/19] vfio iommu type1: Update arguments of vfio_lock_acct Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 13/19] vfio: Introduce vfio_set_irqs_validate_and_prepare() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 12/19] vfio_pci: Update vfio_pci to use vfio_info_add_capability() Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
    [PATCH v10 02/19] vfio: VFIO based driver for Mediated devices Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
      Re: [PATCH v10 02/19] vfio: VFIO based driver for Mediated devices Jike Song <jike.song@intel.com> - 2016-11-02 11:50 +0100
    [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-26 23:40 +0200
      Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Jike Song <jike.song@intel.com> - 2016-10-28 09:40 +0200
        Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-28 14:40 +0200
        Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Alex Williamson <alex.williamson@redhat.com> - 2016-10-28 14:50 +0200
          Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-28 22:10 +0200
            Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Alex Williamson <alex.williamson@redhat.com> - 2016-10-28 22:40 +0200
              Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-29 12:40 +0200
                Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Alex Williamson <alex.williamson@redhat.com> - 2016-10-29 16:10 +0200
                  Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-11-01 08:50 +0100
      Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Jike Song <jike.song@intel.com> - 2016-10-31 05:00 +0100
        Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Jike Song <jike.song@intel.com> - 2016-10-31 07:10 +0100
        Re: [PATCH v10 10/19] vfio iommu: Add blocking notifier to notify  DMA_UNMAP Kirti Wankhede <kwankhede@nvidia.com> - 2016-10-31 07:10 +0100
    Re: [PATCH v10 19/19] docs: Sample driver to demonstrate how to use  Mediated device framework. Jonathan Corbet <corbet@lwn.net> - 2016-10-27 16:40 +0200
    Re: [PATCH v10 00/19] Add Mediated device support Jike Song <jike.song@intel.com> - 2016-11-01 09:40 +0100
      Re: [PATCH v10 00/19] Add Mediated device support Gerd Hoffmann <kraxel@redhat.com> - 2016-11-01 16:30 +0100
        Re: [Qemu-devel] [PATCH v10 00/19] Add Mediated device support Jike Song <jike.song@intel.com> - 2016-11-02 02:10 +0100

Page 1 of 3  [1] 2 3  Next page →

#1509844 — [PATCH v10 00/19] Add Mediated device support

This series adds Mediated device support to Linux host kernel. Purpose
of this series is to provide a common interface for mediated device
management that can be used by different devices. This series introduces
Mdev core module that creates and manages mediated devices, VFIO based
driver for mediated devices that are created by mdev core module and
update VFIO type1 IOMMU module to support pinning & unpinning for mediated
devices.

What changed in v10?
vfio:
 - Split commits in multple individual commits.
 - Removed the function added in v9 to get device_api string.
 - Defined constant strings in include/uapi/linux/vfio.h that should be used by
   vendor driver for device_api attribute.

vfio_iommu_type1:
 - Fixed accounting when pass through device is unplugged while mdev device
   exist in a domain.
 - Added blocking notifier to notify DMA_UNMAP to vendor driver to invalidate
   mappings.
 - Exported APIs to register notifier for DMA_UNMAP action.

Documentation:
 - Added sysfs ABI for mediated device framework.
 - Updated Documentation/vfio-mdev/vfio-mediated-device.txt.
 - Updated mtty.c with bug fixes.

Kirti Wankhede (19):
  vfio: Mediated device Core driver
  vfio: VFIO based driver for Mediated devices
  vfio: Rearrange functions to get vfio_group from dev
  vfio: Common function to increment container_users
  vfio iommu: Added pin and unpin callback functions to
    vfio_iommu_driver_ops
  vfio iommu type1: Update arguments of vfio_lock_acct
  vfio iommu type1: Update argument of vaddr_get_pfn()
  vfio iommu type1: Add find_iommu_group() function
  vfio iommu type1: Add support for mediated devices
  vfio iommu: Add blocking notifier to notify DMA_UNMAP
  vfio: Introduce common function to add capabilities
  vfio_pci: Update vfio_pci to use vfio_info_add_capability()
  vfio: Introduce vfio_set_irqs_validate_and_prepare()
  vfio_pci: Updated to use vfio_set_irqs_validate_and_prepare()
  vfio_platform: Updated to use vfio_set_irqs_validate_and_prepare()
  vfio: Define device_api strings
  docs: Add Documentation for Mediated devices
  docs: Sysfs ABI for mediated device framework
  docs: Sample driver to demonstrate how to use Mediated device
    framework.

 Documentation/ABI/testing/sysfs-bus-vfio-mdev    |  111 ++
 Documentation/vfio-mdev/Makefile                 |   13 +
 Documentation/vfio-mdev/mtty.c                   | 1503 ++++++++++++++++++++++
 Documentation/vfio-mdev/vfio-mediated-device.txt |  398 ++++++
 drivers/vfio/Kconfig                             |    1 +
 drivers/vfio/Makefile                            |    1 +
 drivers/vfio/mdev/Kconfig                        |   17 +
 drivers/vfio/mdev/Makefile                       |    5 +
 drivers/vfio/mdev/mdev_core.c                    |  384 ++++++
 drivers/vfio/mdev/mdev_driver.c                  |  122 ++
 drivers/vfio/mdev/mdev_private.h                 |   41 +
 drivers/vfio/mdev/mdev_sysfs.c                   |  286 ++++
 drivers/vfio/mdev/vfio_mdev.c                    |  148 +++
 drivers/vfio/pci/vfio_pci.c                      |   78 +-
 drivers/vfio/platform/vfio_platform_common.c     |   31 +-
 drivers/vfio/vfio.c                              |  322 ++++-
 drivers/vfio/vfio_iommu_type1.c                  |  808 ++++++++++--
 include/linux/mdev.h                             |  167 +++
 include/linux/vfio.h                             |   30 +-
 include/uapi/linux/vfio.h                        |   10 +
 20 files changed, 4270 insertions(+), 206 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev
 create mode 100644 Documentation/vfio-mdev/Makefile
 create mode 100644 Documentation/vfio-mdev/mtty.c
 create mode 100644 Documentation/vfio-mdev/vfio-mediated-device.txt
 create mode 100644 drivers/vfio/mdev/Kconfig
 create mode 100644 drivers/vfio/mdev/Makefile
 create mode 100644 drivers/vfio/mdev/mdev_core.c
 create mode 100644 drivers/vfio/mdev/mdev_driver.c
 create mode 100644 drivers/vfio/mdev/mdev_private.h
 create mode 100644 drivers/vfio/mdev/mdev_sysfs.c
 create mode 100644 drivers/vfio/mdev/vfio_mdev.c
 create mode 100644 include/linux/mdev.h

-- 
2.7.0

[toc] | [next] | [standalone]

#1509846 — [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn()

Update arguments of vaddr_get_pfn() to take struct mm_struct *mm as input
argument.

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Change-Id: I885fd4cd4a9f66f4ee2c1caf58267464ec239f52
---
 drivers/vfio/vfio_iommu_type1.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 72fee415044a..3d916b965492 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -228,20 +228,31 @@ static int put_pfn(unsigned long pfn, int prot)
 	return 0;
 }
 
-static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
+static int vaddr_get_pfn(struct mm_struct *remote_mm, unsigned long vaddr,
+			 int prot, unsigned long *pfn)
 {
 	struct page *page[1];
 	struct vm_area_struct *vma;
+	struct mm_struct *mm = (remote_mm ? remote_mm : current->mm);
 	int ret = -EFAULT;
 
-	if (get_user_pages_fast(vaddr, 1, !!(prot & IOMMU_WRITE), page) == 1) {
+	if (remote_mm) {
+		down_read(&mm->mmap_sem);
+		ret = get_user_pages_remote(NULL, mm, vaddr, 1,
+					!!(prot & IOMMU_WRITE), 0, page, NULL);
+		up_read(&mm->mmap_sem);
+	} else
+		ret = get_user_pages_fast(vaddr, 1,
+					  !!(prot & IOMMU_WRITE), page);
+
+	if (ret == 1) {
 		*pfn = page_to_pfn(page[0]);
 		return 0;
 	}
 
-	down_read(&current->mm->mmap_sem);
+	down_read(&mm->mmap_sem);
 
-	vma = find_vma_intersection(current->mm, vaddr, vaddr + 1);
+	vma = find_vma_intersection(mm, vaddr, vaddr + 1);
 
 	if (vma && vma->vm_flags & VM_PFNMAP) {
 		*pfn = ((vaddr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
@@ -249,7 +260,7 @@ static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
 			ret = 0;
 	}
 
-	up_read(&current->mm->mmap_sem);
+	up_read(&mm->mmap_sem);
 
 	return ret;
 }
@@ -270,7 +281,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
 	if (!current->mm)
 		return -ENODEV;
 
-	ret = vaddr_get_pfn(vaddr, prot, pfn_base);
+	ret = vaddr_get_pfn(NULL, vaddr, prot, pfn_base);
 	if (ret)
 		return ret;
 
@@ -293,7 +304,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
 	for (i = 1, vaddr += PAGE_SIZE; i < npage; i++, vaddr += PAGE_SIZE) {
 		unsigned long pfn = 0;
 
-		ret = vaddr_get_pfn(vaddr, prot, &pfn);
+		ret = vaddr_get_pfn(NULL, vaddr, prot, &pfn);
 		if (ret)
 			break;
 
-- 
2.7.0

[toc] | [prev] | [next] | [standalone]

#1510179 — Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn()

On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> Update arguments of vaddr_get_pfn() to take struct mm_struct *mm as input
> argument.
> 
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> Signed-off-by: Neo Jia <cjia@nvidia.com>
> Change-Id: I885fd4cd4a9f66f4ee2c1caf58267464ec239f52
> ---
>  drivers/vfio/vfio_iommu_type1.c | 25 ++++++++++++++++++-------
>  1 file changed, 18 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 72fee415044a..3d916b965492 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -228,20 +228,31 @@ static int put_pfn(unsigned long pfn, int prot)
>  	return 0;
>  }
>  
> -static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
> +static int vaddr_get_pfn(struct mm_struct *remote_mm, unsigned long vaddr,
> +			 int prot, unsigned long *pfn)
>  {
>  	struct page *page[1];
>  	struct vm_area_struct *vma;
> +	struct mm_struct *mm = (remote_mm ? remote_mm : current->mm);
>  	int ret = -EFAULT;
>  
> -	if (get_user_pages_fast(vaddr, 1, !!(prot & IOMMU_WRITE), page) == 1) {
> +	if (remote_mm) {
> +		down_read(&mm->mmap_sem);
> +		ret = get_user_pages_remote(NULL, mm, vaddr, 1,
> +					!!(prot & IOMMU_WRITE), 0, page, NULL);

Hi Kirti,

It seems that get_user_pages_remote has only 7 args but you somehow gives 8
here?

--
Thanks,
Jike

> +		up_read(&mm->mmap_sem);
> +	} else
> +		ret = get_user_pages_fast(vaddr, 1,
> +					  !!(prot & IOMMU_WRITE), page);
> +
> +	if (ret == 1) {
>  		*pfn = page_to_pfn(page[0]);
>  		return 0;
>  	}
>  
> -	down_read(&current->mm->mmap_sem);
> +	down_read(&mm->mmap_sem);
>  
> -	vma = find_vma_intersection(current->mm, vaddr, vaddr + 1);
> +	vma = find_vma_intersection(mm, vaddr, vaddr + 1);
>  
>  	if (vma && vma->vm_flags & VM_PFNMAP) {
>  		*pfn = ((vaddr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
> @@ -249,7 +260,7 @@ static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
>  			ret = 0;
>  	}
>  
> -	up_read(&current->mm->mmap_sem);
> +	up_read(&mm->mmap_sem);
>  
>  	return ret;
>  }
> @@ -270,7 +281,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
>  	if (!current->mm)
>  		return -ENODEV;
>  
> -	ret = vaddr_get_pfn(vaddr, prot, pfn_base);
> +	ret = vaddr_get_pfn(NULL, vaddr, prot, pfn_base);
>  	if (ret)
>  		return ret;
>  
> @@ -293,7 +304,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
>  	for (i = 1, vaddr += PAGE_SIZE; i < npage; i++, vaddr += PAGE_SIZE) {
>  		unsigned long pfn = 0;
>  
> -		ret = vaddr_get_pfn(vaddr, prot, &pfn);
> +		ret = vaddr_get_pfn(NULL, vaddr, prot, &pfn);
>  		if (ret)
>  			break;
>  
> 

[toc] | [prev] | [next] | [standalone]

#1510412 — Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn()

On 10/27/2016 5:41 PM, Jike Song wrote:
> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>> Update arguments of vaddr_get_pfn() to take struct mm_struct *mm as input
>> argument.
>>
>> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
>> Signed-off-by: Neo Jia <cjia@nvidia.com>
>> Change-Id: I885fd4cd4a9f66f4ee2c1caf58267464ec239f52
>> ---
>>  drivers/vfio/vfio_iommu_type1.c | 25 ++++++++++++++++++-------
>>  1 file changed, 18 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
>> index 72fee415044a..3d916b965492 100644
>> --- a/drivers/vfio/vfio_iommu_type1.c
>> +++ b/drivers/vfio/vfio_iommu_type1.c
>> @@ -228,20 +228,31 @@ static int put_pfn(unsigned long pfn, int prot)
>>  	return 0;
>>  }
>>  
>> -static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
>> +static int vaddr_get_pfn(struct mm_struct *remote_mm, unsigned long vaddr,
>> +			 int prot, unsigned long *pfn)
>>  {
>>  	struct page *page[1];
>>  	struct vm_area_struct *vma;
>> +	struct mm_struct *mm = (remote_mm ? remote_mm : current->mm);
>>  	int ret = -EFAULT;
>>  
>> -	if (get_user_pages_fast(vaddr, 1, !!(prot & IOMMU_WRITE), page) == 1) {
>> +	if (remote_mm) {
>> +		down_read(&mm->mmap_sem);
>> +		ret = get_user_pages_remote(NULL, mm, vaddr, 1,
>> +					!!(prot & IOMMU_WRITE), 0, page, NULL);
> 
> Hi Kirti,
> 
> It seems that get_user_pages_remote has only 7 args but you somehow gives 8
> here?
> 

I had tested this patch with linux-next. It has 8 arguments, see mm/gup.c:

long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
                unsigned long start, unsigned long nr_pages,
                int write, int force, struct page **pages,
                struct vm_area_struct **vmas)
{
        return __get_user_pages_locked(tsk, mm, start, nr_pages, write,
force,
                                       pages, vmas, NULL, false,
                                       FOLL_TOUCH | FOLL_REMOTE);
}
EXPORT_SYMBOL(get_user_pages_remote);


Thanks,
Kirti

> --
> Thanks,
> Jike
> 
>> +		up_read(&mm->mmap_sem);
>> +	} else
>> +		ret = get_user_pages_fast(vaddr, 1,
>> +					  !!(prot & IOMMU_WRITE), page);
>> +
>> +	if (ret == 1) {
>>  		*pfn = page_to_pfn(page[0]);
>>  		return 0;
>>  	}
>>  
>> -	down_read(&current->mm->mmap_sem);
>> +	down_read(&mm->mmap_sem);
>>  
>> -	vma = find_vma_intersection(current->mm, vaddr, vaddr + 1);
>> +	vma = find_vma_intersection(mm, vaddr, vaddr + 1);
>>  
>>  	if (vma && vma->vm_flags & VM_PFNMAP) {
>>  		*pfn = ((vaddr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
>> @@ -249,7 +260,7 @@ static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
>>  			ret = 0;
>>  	}
>>  
>> -	up_read(&current->mm->mmap_sem);
>> +	up_read(&mm->mmap_sem);
>>  
>>  	return ret;
>>  }
>> @@ -270,7 +281,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
>>  	if (!current->mm)
>>  		return -ENODEV;
>>  
>> -	ret = vaddr_get_pfn(vaddr, prot, pfn_base);
>> +	ret = vaddr_get_pfn(NULL, vaddr, prot, pfn_base);
>>  	if (ret)
>>  		return ret;
>>  
>> @@ -293,7 +304,7 @@ static long vfio_pin_pages(unsigned long vaddr, long npage,
>>  	for (i = 1, vaddr += PAGE_SIZE; i < npage; i++, vaddr += PAGE_SIZE) {
>>  		unsigned long pfn = 0;
>>  
>> -		ret = vaddr_get_pfn(vaddr, prot, &pfn);
>> +		ret = vaddr_get_pfn(NULL, vaddr, prot, &pfn);
>>  		if (ret)
>>  			break;
>>  
>>

[toc] | [prev] | [next] | [standalone]

#1510868 — Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn()

On 10/27/2016 08:24 PM, Kirti Wankhede wrote:
> 
> 
> On 10/27/2016 5:41 PM, Jike Song wrote:
>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>> Update arguments of vaddr_get_pfn() to take struct mm_struct *mm as input
>>> argument.
>>>
>>> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
>>> Signed-off-by: Neo Jia <cjia@nvidia.com>
>>> Change-Id: I885fd4cd4a9f66f4ee2c1caf58267464ec239f52
>>> ---
>>>  drivers/vfio/vfio_iommu_type1.c | 25 ++++++++++++++++++-------
>>>  1 file changed, 18 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
>>> index 72fee415044a..3d916b965492 100644
>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>> @@ -228,20 +228,31 @@ static int put_pfn(unsigned long pfn, int prot)
>>>  	return 0;
>>>  }
>>>  
>>> -static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
>>> +static int vaddr_get_pfn(struct mm_struct *remote_mm, unsigned long vaddr,
>>> +			 int prot, unsigned long *pfn)
>>>  {
>>>  	struct page *page[1];
>>>  	struct vm_area_struct *vma;
>>> +	struct mm_struct *mm = (remote_mm ? remote_mm : current->mm);
>>>  	int ret = -EFAULT;
>>>  
>>> -	if (get_user_pages_fast(vaddr, 1, !!(prot & IOMMU_WRITE), page) == 1) {
>>> +	if (remote_mm) {
>>> +		down_read(&mm->mmap_sem);
>>> +		ret = get_user_pages_remote(NULL, mm, vaddr, 1,
>>> +					!!(prot & IOMMU_WRITE), 0, page, NULL);
>>
>> Hi Kirti,
>>
>> It seems that get_user_pages_remote has only 7 args but you somehow gives 8
>> here?
>>
> 
> I had tested this patch with linux-next. It has 8 arguments, see mm/gup.c:
> 
> long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
>                 unsigned long start, unsigned long nr_pages,
>                 int write, int force, struct page **pages,
>                 struct vm_area_struct **vmas)
> {
>         return __get_user_pages_locked(tsk, mm, start, nr_pages, write,
> force,
>                                        pages, vmas, NULL, false,
>                                        FOLL_TOUCH | FOLL_REMOTE);
> }
> EXPORT_SYMBOL(get_user_pages_remote);

It's queer. I checked:

https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/mm/gup.c

It still has 7 args.

I believe you have seen this change somewhere, would you share the repo url
and branch name of the linux-next you based on?

--
Thanks,
Jike

[toc] | [prev] | [next] | [standalone]

#1513724 — Re: [PATCH v10 07/19] vfio iommu type1: Update argument of vaddr_get_pfn()

On 10/28/2016 11:31 AM, Jike Song wrote:
> On 10/27/2016 08:24 PM, Kirti Wankhede wrote:
>>
>>
>> On 10/27/2016 5:41 PM, Jike Song wrote:
>>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>>> Update arguments of vaddr_get_pfn() to take struct mm_struct *mm as input
>>>> argument.
>>>>
>>>> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
>>>> Signed-off-by: Neo Jia <cjia@nvidia.com>
>>>> Change-Id: I885fd4cd4a9f66f4ee2c1caf58267464ec239f52
>>>> ---
>>>>  drivers/vfio/vfio_iommu_type1.c | 25 ++++++++++++++++++-------
>>>>  1 file changed, 18 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
>>>> index 72fee415044a..3d916b965492 100644
>>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>>> @@ -228,20 +228,31 @@ static int put_pfn(unsigned long pfn, int prot)
>>>>  	return 0;
>>>>  }
>>>>  
>>>> -static int vaddr_get_pfn(unsigned long vaddr, int prot, unsigned long *pfn)
>>>> +static int vaddr_get_pfn(struct mm_struct *remote_mm, unsigned long vaddr,
>>>> +			 int prot, unsigned long *pfn)
>>>>  {
>>>>  	struct page *page[1];
>>>>  	struct vm_area_struct *vma;
>>>> +	struct mm_struct *mm = (remote_mm ? remote_mm : current->mm);
>>>>  	int ret = -EFAULT;
>>>>  
>>>> -	if (get_user_pages_fast(vaddr, 1, !!(prot & IOMMU_WRITE), page) == 1) {
>>>> +	if (remote_mm) {
>>>> +		down_read(&mm->mmap_sem);
>>>> +		ret = get_user_pages_remote(NULL, mm, vaddr, 1,
>>>> +					!!(prot & IOMMU_WRITE), 0, page, NULL);
>>>
>>> Hi Kirti,
>>>
>>> It seems that get_user_pages_remote has only 7 args but you somehow gives 8
>>> here?
>>>
>>
>> I had tested this patch with linux-next. It has 8 arguments, see mm/gup.c:
>>
>> long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
>>                 unsigned long start, unsigned long nr_pages,
>>                 int write, int force, struct page **pages,
>>                 struct vm_area_struct **vmas)
>> {
>>         return __get_user_pages_locked(tsk, mm, start, nr_pages, write,
>> force,
>>                                        pages, vmas, NULL, false,
>>                                        FOLL_TOUCH | FOLL_REMOTE);
>> }
>> EXPORT_SYMBOL(get_user_pages_remote);
> 
> It's queer. I checked:
> 
> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/mm/gup.c
> 
> It still has 7 args.
> 
> I believe you have seen this change somewhere, would you share the repo url
> and branch name of the linux-next you based on?
> 

Sorry, my git was synced just few changes behind this change. I'll make
sure that I test the patch set at top of tree.

Thanks,
Kirti

[toc] | [prev] | [next] | [standalone]

#1509847 — [PATCH v10 01/19] vfio: Mediated device Core driver

Design for Mediated Device Driver:
Main purpose of this driver is to provide a common interface for mediated
device management that can be used by different drivers of different
devices.

This module provides a generic interface to create the device, add it to
mediated bus, add device to IOMMU group and then add it to vfio group.

Below is the high Level block diagram, with Nvidia, Intel and IBM devices
as example, since these are the devices which are going to actively use
this module as of now.

 +---------------+
 |               |
 | +-----------+ |  mdev_register_driver() +--------------+
 | |           | +<------------------------+ __init()     |
 | |  mdev     | |                         |              |
 | |  bus      | +------------------------>+              |<-> VFIO user
 | |  driver   | |     probe()/remove()    | vfio_mdev.ko |    APIs
 | |           | |                         |              |
 | +-----------+ |                         +--------------+
 |               |
 |  MDEV CORE    |
 |   MODULE      |
 |   mdev.ko     |
 | +-----------+ |  mdev_register_device() +--------------+
 | |           | +<------------------------+              |
 | |           | |                         |  nvidia.ko   |<-> physical
 | |           | +------------------------>+              |    device
 | |           | |        callback         +--------------+
 | | Physical  | |
 | |  device   | |  mdev_register_device() +--------------+
 | | interface | |<------------------------+              |
 | |           | |                         |  i915.ko     |<-> physical
 | |           | +------------------------>+              |    device
 | |           | |        callback         +--------------+
 | |           | |
 | |           | |  mdev_register_device() +--------------+
 | |           | +<------------------------+              |
 | |           | |                         | ccw_device.ko|<-> physical
 | |           | +------------------------>+              |    device
 | |           | |        callback         +--------------+
 | +-----------+ |
 +---------------+

Core driver provides two types of registration interfaces:
1. Registration interface for mediated bus driver:

/**
  * struct mdev_driver - Mediated device's driver
  * @name: driver name
  * @probe: called when new device created
  * @remove:called when device removed
  * @driver:device driver structure
  *
  **/
struct mdev_driver {
         const char *name;
         int  (*probe)  (struct device *dev);
         void (*remove) (struct device *dev);
         struct device_driver    driver;
};

Mediated bus driver for mdev device should use this interface to register
and unregister with core driver respectively:

int  mdev_register_driver(struct mdev_driver *drv, struct module *owner);
void mdev_unregister_driver(struct mdev_driver *drv);

Mediated bus driver is responsible to add/delete mediated devices to/from
VFIO group when devices are bound and unbound to the driver.

2. Physical device driver interface
This interface provides vendor driver the set APIs to manage physical
device related work in its driver. APIs are :

* dev_attr_groups: attributes of the parent device.
* mdev_attr_groups: attributes of the mediated device.
* supported_type_groups: attributes to define supported type. This is
			 mandatory field.
* create: to allocate basic resources in vendor driver for a mediated
         device. This is mandatory to be provided by vendor driver.
* remove: to free resources in vendor driver when mediated device is
         destroyed. This is mandatory to be provided by vendor driver.
* open: open callback of mediated device
* release: release callback of mediated device
* read : read emulation callback.
* write: write emulation callback.
* mmap: mmap emulation callback.
* ioctl: ioctl callback.

Drivers should use these interfaces to register and unregister device to
mdev core driver respectively:

extern int  mdev_register_device(struct device *dev,
                                 const struct parent_ops *ops);
extern void mdev_unregister_device(struct device *dev);

There are no locks to serialize above callbacks in mdev driver and
vfio_mdev driver. If required, vendor driver can have locks to serialize
above APIs in their driver.

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Change-Id: I73a5084574270b14541c529461ea2f03c292d510
---
 drivers/vfio/Kconfig             |   1 +
 drivers/vfio/Makefile            |   1 +
 drivers/vfio/mdev/Kconfig        |  10 +
 drivers/vfio/mdev/Makefile       |   4 +
 drivers/vfio/mdev/mdev_core.c    | 384 +++++++++++++++++++++++++++++++++++++++
 drivers/vfio/mdev/mdev_driver.c  | 122 +++++++++++++
 drivers/vfio/mdev/mdev_private.h |  41 +++++
 drivers/vfio/mdev/mdev_sysfs.c   | 286 +++++++++++++++++++++++++++++
 include/linux/mdev.h             | 167 +++++++++++++++++
 9 files changed, 1016 insertions(+)
 create mode 100644 drivers/vfio/mdev/Kconfig
 create mode 100644 drivers/vfio/mdev/Makefile
 create mode 100644 drivers/vfio/mdev/mdev_core.c
 create mode 100644 drivers/vfio/mdev/mdev_driver.c
 create mode 100644 drivers/vfio/mdev/mdev_private.h
 create mode 100644 drivers/vfio/mdev/mdev_sysfs.c
 create mode 100644 include/linux/mdev.h

diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig
index da6e2ce77495..23eced02aaf6 100644
--- a/drivers/vfio/Kconfig
+++ b/drivers/vfio/Kconfig
@@ -48,4 +48,5 @@ menuconfig VFIO_NOIOMMU
 
 source "drivers/vfio/pci/Kconfig"
 source "drivers/vfio/platform/Kconfig"
+source "drivers/vfio/mdev/Kconfig"
 source "virt/lib/Kconfig"
diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile
index 7b8a31f63fea..4a23c13b6be4 100644
--- a/drivers/vfio/Makefile
+++ b/drivers/vfio/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
 obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o
 obj-$(CONFIG_VFIO_PCI) += pci/
 obj-$(CONFIG_VFIO_PLATFORM) += platform/
+obj-$(CONFIG_VFIO_MDEV) += mdev/
diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
new file mode 100644
index 000000000000..303c14ce2847
--- /dev/null
+++ b/drivers/vfio/mdev/Kconfig
@@ -0,0 +1,10 @@
+
+config VFIO_MDEV
+    tristate "Mediated device driver framework"
+    depends on VFIO
+    default n
+    help
+        Provides a framework to virtualize devices.
+	See Documentation/vfio-mdev/vfio-mediated-device.txt for more details.
+
+        If you don't know what do here, say N.
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
new file mode 100644
index 000000000000..31bc04801d94
--- /dev/null
+++ b/drivers/vfio/mdev/Makefile
@@ -0,0 +1,4 @@
+
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
+
+obj-$(CONFIG_VFIO_MDEV) += mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
new file mode 100644
index 000000000000..9d8fa5c91c2e
--- /dev/null
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -0,0 +1,384 @@
+/*
+ * Mediated device Core Driver
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *	       Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/uuid.h>
+#include <linux/sysfs.h>
+#include <linux/mdev.h>
+
+#include "mdev_private.h"
+
+#define DRIVER_VERSION		"0.1"
+#define DRIVER_AUTHOR		"NVIDIA Corporation"
+#define DRIVER_DESC		"Mediated device Core Driver"
+
+static LIST_HEAD(parent_list);
+static DEFINE_MUTEX(parent_list_lock);
+static struct class_compat *mdev_bus_compat_class;
+
+static int _find_mdev_device(struct device *dev, void *data)
+{
+	struct mdev_device *mdev;
+
+	if (!dev_is_mdev(dev))
+		return 0;
+
+	mdev = to_mdev_device(dev);
+
+	if (uuid_le_cmp(mdev->uuid, *(uuid_le *)data) == 0)
+		return 1;
+
+	return 0;
+}
+
+static bool mdev_device_exist(struct parent_device *parent, uuid_le uuid)
+{
+	struct device *dev;
+
+	dev = device_find_child(parent->dev, &uuid, _find_mdev_device);
+	if (dev) {
+		put_device(dev);
+		return true;
+	}
+
+	return false;
+}
+
+/* Should be called holding parent_list_lock */
+static struct parent_device *__find_parent_device(struct device *dev)
+{
+	struct parent_device *parent;
+
+	list_for_each_entry(parent, &parent_list, next) {
+		if (parent->dev == dev)
+			return parent;
+	}
+	return NULL;
+}
+
+static void mdev_release_parent(struct kref *kref)
+{
+	struct parent_device *parent = container_of(kref, struct parent_device,
+						    ref);
+	struct device *dev = parent->dev;
+
+	kfree(parent);
+	put_device(dev);
+}
+
+static
+inline struct parent_device *mdev_get_parent(struct parent_device *parent)
+{
+	if (parent)
+		kref_get(&parent->ref);
+
+	return parent;
+}
+
+static inline void mdev_put_parent(struct parent_device *parent)
+{
+	if (parent)
+		kref_put(&parent->ref, mdev_release_parent);
+}
+
+static int mdev_device_create_ops(struct kobject *kobj,
+				  struct mdev_device *mdev)
+{
+	struct parent_device *parent = mdev->parent;
+	int ret;
+
+	ret = parent->ops->create(kobj, mdev);
+	if (ret)
+		return ret;
+
+	ret = sysfs_create_groups(&mdev->dev.kobj,
+				  parent->ops->mdev_attr_groups);
+	if (ret)
+		parent->ops->remove(mdev);
+
+	return ret;
+}
+
+/*
+ * mdev_device_remove_ops gets called from sysfs's 'remove' and when parent
+ * device is being unregistered from mdev device framework.
+ * - 'force_remove' is set to 'false' when called from sysfs's 'remove' which
+ *   indicates that if the mdev device is active, used by VMM or userspace
+ *   application, vendor driver could return error then don't remove the device.
+ * - 'force_remove' is set to 'true' when called from mdev_unregister_device()
+ *   which indicate that parent device is being removed from mdev device
+ *   framework so remove mdev device forcefully.
+ */
+static int mdev_device_remove_ops(struct mdev_device *mdev, bool force_remove)
+{
+	struct parent_device *parent = mdev->parent;
+	int ret;
+
+	/*
+	 * Vendor driver can return error if VMM or userspace application is
+	 * using this mdev device.
+	 */
+	ret = parent->ops->remove(mdev);
+	if (ret && !force_remove)
+		return -EBUSY;
+
+	sysfs_remove_groups(&mdev->dev.kobj, parent->ops->mdev_attr_groups);
+	return 0;
+}
+
+static int mdev_device_remove_cb(struct device *dev, void *data)
+{
+	if (!dev_is_mdev(dev))
+		return 0;
+
+	return mdev_device_remove(dev, data ? *(bool *)data : true);
+}
+
+/*
+ * mdev_register_device : Register a device
+ * @dev: device structure representing parent device.
+ * @ops: Parent device operation structure to be registered.
+ *
+ * Add device to list of registered parent devices.
+ * Returns a negative value on error, otherwise 0.
+ */
+int mdev_register_device(struct device *dev, const struct parent_ops *ops)
+{
+	int ret;
+	struct parent_device *parent;
+
+	/* check for mandatory ops */
+	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
+		return -EINVAL;
+
+	dev = get_device(dev);
+	if (!dev)
+		return -EINVAL;
+
+	mutex_lock(&parent_list_lock);
+
+	/* Check for duplicate */
+	parent = __find_parent_device(dev);
+	if (parent) {
+		ret = -EEXIST;
+		goto add_dev_err;
+	}
+
+	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
+	if (!parent) {
+		ret = -ENOMEM;
+		goto add_dev_err;
+	}
+
+	kref_init(&parent->ref);
+	mutex_init(&parent->lock);
+
+	parent->dev = dev;
+	parent->ops = ops;
+
+	ret = parent_create_sysfs_files(parent);
+	if (ret) {
+		mutex_unlock(&parent_list_lock);
+		mdev_put_parent(parent);
+		return ret;
+	}
+
+	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
+	if (ret)
+		dev_warn(dev, "Failed to create compatibility class link\n");
+
+	list_add(&parent->next, &parent_list);
+	mutex_unlock(&parent_list_lock);
+
+	dev_info(dev, "MDEV: Registered\n");
+	return 0;
+
+add_dev_err:
+	mutex_unlock(&parent_list_lock);
+	put_device(dev);
+	return ret;
+}
+EXPORT_SYMBOL(mdev_register_device);
+
+/*
+ * mdev_unregister_device : Unregister a parent device
+ * @dev: device structure representing parent device.
+ *
+ * Remove device from list of registered parent devices. Give a chance to free
+ * existing mediated devices for given device.
+ */
+
+void mdev_unregister_device(struct device *dev)
+{
+	struct parent_device *parent;
+	bool force_remove = true;
+
+	mutex_lock(&parent_list_lock);
+	parent = __find_parent_device(dev);
+
+	if (!parent) {
+		mutex_unlock(&parent_list_lock);
+		return;
+	}
+	dev_info(dev, "MDEV: Unregistering\n");
+
+	list_del(&parent->next);
+	class_compat_remove_link(mdev_bus_compat_class, dev, NULL);
+
+	device_for_each_child(dev, (void *)&force_remove,
+			      mdev_device_remove_cb);
+
+	parent_remove_sysfs_files(parent);
+
+	mutex_unlock(&parent_list_lock);
+	mdev_put_parent(parent);
+}
+EXPORT_SYMBOL(mdev_unregister_device);
+
+static void mdev_device_release(struct device *dev)
+{
+	struct mdev_device *mdev = to_mdev_device(dev);
+
+	dev_dbg(&mdev->dev, "MDEV: destroying\n");
+	kfree(mdev);
+}
+
+int mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid)
+{
+	int ret;
+	struct mdev_device *mdev;
+	struct parent_device *parent;
+	struct mdev_type *type = to_mdev_type(kobj);
+
+	parent = mdev_get_parent(type->parent);
+	if (!parent)
+		return -EINVAL;
+
+	mutex_lock(&parent->lock);
+
+	/* Check for duplicate */
+	if (mdev_device_exist(parent, uuid)) {
+		ret = -EEXIST;
+		goto create_err;
+	}
+
+	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
+	if (!mdev) {
+		ret = -ENOMEM;
+		goto create_err;
+	}
+
+	memcpy(&mdev->uuid, &uuid, sizeof(uuid_le));
+	mdev->parent = parent;
+	kref_init(&mdev->ref);
+
+	mdev->dev.parent  = dev;
+	mdev->dev.bus     = &mdev_bus_type;
+	mdev->dev.release = mdev_device_release;
+	dev_set_name(&mdev->dev, "%pUl", uuid.b);
+
+	ret = device_register(&mdev->dev);
+	if (ret) {
+		put_device(&mdev->dev);
+		goto create_err;
+	}
+
+	ret = mdev_device_create_ops(kobj, mdev);
+	if (ret)
+		goto create_failed;
+
+	ret = mdev_create_sysfs_files(&mdev->dev, type);
+	if (ret) {
+		mdev_device_remove_ops(mdev, true);
+		goto create_failed;
+	}
+
+	mdev->type_kobj = kobj;
+	dev_dbg(&mdev->dev, "MDEV: created\n");
+
+	mutex_unlock(&parent->lock);
+	return ret;
+
+create_failed:
+	device_unregister(&mdev->dev);
+
+create_err:
+	mutex_unlock(&parent->lock);
+	mdev_put_parent(parent);
+	return ret;
+}
+
+int mdev_device_remove(struct device *dev, bool force_remove)
+{
+	struct mdev_device *mdev;
+	struct parent_device *parent;
+	struct mdev_type *type;
+	int ret;
+
+	mdev = to_mdev_device(dev);
+	type = to_mdev_type(mdev->type_kobj);
+	parent = mdev->parent;
+	mutex_lock(&parent->lock);
+
+	ret = mdev_device_remove_ops(mdev, force_remove);
+	if (ret) {
+		mutex_unlock(&parent->lock);
+		return ret;
+	}
+
+	mdev_remove_sysfs_files(dev, type);
+	device_unregister(dev);
+	mutex_unlock(&parent->lock);
+	mdev_put_parent(parent);
+	return ret;
+}
+
+static int __init mdev_init(void)
+{
+	int ret;
+
+	ret = mdev_bus_register();
+	if (ret) {
+		pr_err("Failed to register mdev bus\n");
+		return ret;
+	}
+
+	mdev_bus_compat_class = class_compat_register("mdev_bus");
+	if (!mdev_bus_compat_class) {
+		mdev_bus_unregister();
+		return -ENOMEM;
+	}
+
+	/*
+	 * Attempt to load known vfio_mdev.  This gives us a working environment
+	 * without the user needing to explicitly load vfio_mdev driver.
+	 */
+	request_module_nowait("vfio_mdev");
+
+	return ret;
+}
+
+static void __exit mdev_exit(void)
+{
+	class_compat_unregister(mdev_bus_compat_class);
+	mdev_bus_unregister();
+}
+
+module_init(mdev_init)
+module_exit(mdev_exit)
+
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
new file mode 100644
index 000000000000..0b3250044a80
--- /dev/null
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -0,0 +1,122 @@
+/*
+ * MDEV driver
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *	       Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/device.h>
+#include <linux/iommu.h>
+#include <linux/mdev.h>
+
+#include "mdev_private.h"
+
+static int mdev_attach_iommu(struct mdev_device *mdev)
+{
+	int ret;
+	struct iommu_group *group;
+
+	group = iommu_group_alloc();
+	if (IS_ERR(group))
+		return PTR_ERR(group);
+
+	ret = iommu_group_add_device(group, &mdev->dev);
+	if (ret)
+		goto attach_fail;
+
+	dev_info(&mdev->dev, "MDEV: group_id = %d\n",
+				 iommu_group_id(group));
+attach_fail:
+	iommu_group_put(group);
+	return ret;
+}
+
+static void mdev_detach_iommu(struct mdev_device *mdev)
+{
+	iommu_group_remove_device(&mdev->dev);
+	dev_info(&mdev->dev, "MDEV: detaching iommu\n");
+}
+
+static int mdev_probe(struct device *dev)
+{
+	struct mdev_driver *drv = to_mdev_driver(dev->driver);
+	struct mdev_device *mdev = to_mdev_device(dev);
+	int ret;
+
+	ret = mdev_attach_iommu(mdev);
+	if (ret)
+		return ret;
+
+	if (drv && drv->probe)
+		ret = drv->probe(dev);
+
+	if (ret)
+		mdev_detach_iommu(mdev);
+
+	return ret;
+}
+
+static int mdev_remove(struct device *dev)
+{
+	struct mdev_driver *drv = to_mdev_driver(dev->driver);
+	struct mdev_device *mdev = to_mdev_device(dev);
+
+	if (drv && drv->remove)
+		drv->remove(dev);
+
+	mdev_detach_iommu(mdev);
+
+	return 0;
+}
+
+struct bus_type mdev_bus_type = {
+	.name		= "mdev",
+	.probe		= mdev_probe,
+	.remove		= mdev_remove,
+};
+EXPORT_SYMBOL_GPL(mdev_bus_type);
+
+/*
+ * mdev_register_driver - register a new MDEV driver
+ * @drv: the driver to register
+ * @owner: module owner of driver to be registered
+ *
+ * Returns a negative value on error, otherwise 0.
+ */
+int mdev_register_driver(struct mdev_driver *drv, struct module *owner)
+{
+	/* initialize common driver fields */
+	drv->driver.name = drv->name;
+	drv->driver.bus = &mdev_bus_type;
+	drv->driver.owner = owner;
+
+	/* register with core */
+	return driver_register(&drv->driver);
+}
+EXPORT_SYMBOL(mdev_register_driver);
+
+/*
+ * mdev_unregister_driver - unregister MDEV driver
+ * @drv: the driver to unregister
+ *
+ */
+void mdev_unregister_driver(struct mdev_driver *drv)
+{
+	driver_unregister(&drv->driver);
+}
+EXPORT_SYMBOL(mdev_unregister_driver);
+
+int mdev_bus_register(void)
+{
+	return bus_register(&mdev_bus_type);
+}
+
+void mdev_bus_unregister(void)
+{
+	bus_unregister(&mdev_bus_type);
+}
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
new file mode 100644
index 000000000000..000c93fcfdbd
--- /dev/null
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -0,0 +1,41 @@
+/*
+ * Mediated device interal definitions
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *	       Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef MDEV_PRIVATE_H
+#define MDEV_PRIVATE_H
+
+int  mdev_bus_register(void);
+void mdev_bus_unregister(void);
+
+struct mdev_type {
+	struct kobject kobj;
+	struct kobject *devices_kobj;
+	struct parent_device *parent;
+	struct list_head next;
+	struct attribute_group *group;
+};
+
+#define to_mdev_type_attr(_attr)	\
+	container_of(_attr, struct mdev_type_attribute, attr)
+#define to_mdev_type(_kobj)		\
+	container_of(_kobj, struct mdev_type, kobj)
+
+int  parent_create_sysfs_files(struct parent_device *parent);
+void parent_remove_sysfs_files(struct parent_device *parent);
+
+int  mdev_create_sysfs_files(struct device *dev, struct mdev_type *type);
+void mdev_remove_sysfs_files(struct device *dev, struct mdev_type *type);
+
+int  mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid);
+int  mdev_device_remove(struct device *dev, bool force_remove);
+
+#endif /* MDEV_PRIVATE_H */
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
new file mode 100644
index 000000000000..d808a41e4f68
--- /dev/null
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -0,0 +1,286 @@
+/*
+ * File attributes for Mediated devices
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *	       Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/sysfs.h>
+#include <linux/ctype.h>
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/uuid.h>
+#include <linux/mdev.h>
+
+#include "mdev_private.h"
+
+/* Static functions */
+
+static ssize_t mdev_type_attr_show(struct kobject *kobj,
+				     struct attribute *__attr, char *buf)
+{
+	struct mdev_type_attribute *attr = to_mdev_type_attr(__attr);
+	struct mdev_type *type = to_mdev_type(kobj);
+	ssize_t ret = -EIO;
+
+	if (attr->show)
+		ret = attr->show(kobj, type->parent->dev, buf);
+	return ret;
+}
+
+static ssize_t mdev_type_attr_store(struct kobject *kobj,
+				      struct attribute *__attr,
+				      const char *buf, size_t count)
+{
+	struct mdev_type_attribute *attr = to_mdev_type_attr(__attr);
+	struct mdev_type *type = to_mdev_type(kobj);
+	ssize_t ret = -EIO;
+
+	if (attr->store)
+		ret = attr->store(&type->kobj, type->parent->dev, buf, count);
+	return ret;
+}
+
+static const struct sysfs_ops mdev_type_sysfs_ops = {
+	.show = mdev_type_attr_show,
+	.store = mdev_type_attr_store,
+};
+
+static ssize_t create_store(struct kobject *kobj, struct device *dev,
+			    const char *buf, size_t count)
+{
+	char *str;
+	uuid_le uuid;
+	int ret;
+
+	if ((count < UUID_STRING_LEN) || (count > UUID_STRING_LEN + 1))
+		return -EINVAL;
+
+	str = kstrndup(buf, count, GFP_KERNEL);
+	if (!str)
+		return -ENOMEM;
+
+	ret = uuid_le_to_bin(str, &uuid);
+	kfree(str);
+	if (ret)
+		return ret;
+
+	ret = mdev_device_create(kobj, dev, uuid);
+	if (ret)
+		return ret;
+
+	return count;
+}
+
+MDEV_TYPE_ATTR_WO(create);
+
+static void mdev_type_release(struct kobject *kobj)
+{
+	struct mdev_type *type = to_mdev_type(kobj);
+
+	pr_debug("Releasing group %s\n", kobj->name);
+	kfree(type);
+}
+
+static struct kobj_type mdev_type_ktype = {
+	.sysfs_ops = &mdev_type_sysfs_ops,
+	.release = mdev_type_release,
+};
+
+struct mdev_type *add_mdev_supported_type(struct parent_device *parent,
+					  struct attribute_group *group)
+{
+	struct mdev_type *type;
+	int ret;
+
+	if (!group->name) {
+		pr_err("%s: Type name empty!\n", __func__);
+		return ERR_PTR(-EINVAL);
+	}
+
+	type = kzalloc(sizeof(*type), GFP_KERNEL);
+	if (!type)
+		return ERR_PTR(-ENOMEM);
+
+	type->kobj.kset = parent->mdev_types_kset;
+
+	ret = kobject_init_and_add(&type->kobj, &mdev_type_ktype, NULL,
+				   "%s-%s", dev_driver_string(parent->dev),
+				   group->name);
+	if (ret) {
+		kfree(type);
+		return ERR_PTR(ret);
+	}
+
+	ret = sysfs_create_file(&type->kobj, &mdev_type_attr_create.attr);
+	if (ret)
+		goto attr_create_failed;
+
+	type->devices_kobj = kobject_create_and_add("devices", &type->kobj);
+	if (!type->devices_kobj) {
+		ret = -ENOMEM;
+		goto attr_devices_failed;
+	}
+
+	ret = sysfs_create_files(&type->kobj,
+				 (const struct attribute **)group->attrs);
+	if (ret) {
+		ret = -ENOMEM;
+		goto attrs_failed;
+	}
+
+	type->group = group;
+	type->parent = parent;
+	return type;
+
+attrs_failed:
+	kobject_put(type->devices_kobj);
+attr_devices_failed:
+	sysfs_remove_file(&type->kobj, &mdev_type_attr_create.attr);
+attr_create_failed:
+	kobject_del(&type->kobj);
+	kobject_put(&type->kobj);
+	return ERR_PTR(ret);
+}
+
+static void remove_mdev_supported_type(struct mdev_type *type)
+{
+	sysfs_remove_files(&type->kobj,
+			   (const struct attribute **)type->group->attrs);
+	kobject_put(type->devices_kobj);
+	sysfs_remove_file(&type->kobj, &mdev_type_attr_create.attr);
+	kobject_del(&type->kobj);
+	kobject_put(&type->kobj);
+}
+
+static int add_mdev_supported_type_groups(struct parent_device *parent)
+{
+	int i;
+
+	for (i = 0; parent->ops->supported_type_groups[i]; i++) {
+		struct mdev_type *type;
+
+		type = add_mdev_supported_type(parent,
+					parent->ops->supported_type_groups[i]);
+		if (IS_ERR(type)) {
+			struct mdev_type *ltype, *tmp;
+
+			list_for_each_entry_safe(ltype, tmp, &parent->type_list,
+						  next) {
+				list_del(&ltype->next);
+				remove_mdev_supported_type(ltype);
+			}
+			return PTR_ERR(type);
+		}
+		list_add(&type->next, &parent->type_list);
+	}
+	return 0;
+}
+
+/* mdev sysfs Functions */
+void parent_remove_sysfs_files(struct parent_device *parent)
+{
+	struct mdev_type *type, *tmp;
+
+	list_for_each_entry_safe(type, tmp, &parent->type_list, next) {
+		list_del(&type->next);
+		remove_mdev_supported_type(type);
+	}
+
+	sysfs_remove_groups(&parent->dev->kobj, parent->ops->dev_attr_groups);
+	kset_unregister(parent->mdev_types_kset);
+}
+
+int parent_create_sysfs_files(struct parent_device *parent)
+{
+	int ret;
+
+	parent->mdev_types_kset = kset_create_and_add("mdev_supported_types",
+					       NULL, &parent->dev->kobj);
+
+	if (!parent->mdev_types_kset)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&parent->type_list);
+
+	ret = sysfs_create_groups(&parent->dev->kobj,
+				  parent->ops->dev_attr_groups);
+	if (ret)
+		goto create_err;
+
+	ret = add_mdev_supported_type_groups(parent);
+	if (ret)
+		sysfs_remove_groups(&parent->dev->kobj,
+				    parent->ops->dev_attr_groups);
+	else
+		return ret;
+
+create_err:
+	kset_unregister(parent->mdev_types_kset);
+	return ret;
+}
+
+static ssize_t remove_store(struct device *dev, struct device_attribute *attr,
+			    const char *buf, size_t count)
+{
+	unsigned long val;
+
+	if (kstrtoul(buf, 0, &val) < 0)
+		return -EINVAL;
+
+	if (val && device_remove_file_self(dev, attr)) {
+		int ret;
+
+		ret = mdev_device_remove(dev, false);
+		if (ret) {
+			device_create_file(dev, attr);
+			return ret;
+		}
+	}
+
+	return count;
+}
+
+static DEVICE_ATTR_WO(remove);
+
+static const struct attribute *mdev_device_attrs[] = {
+	&dev_attr_remove.attr,
+	NULL,
+};
+
+int  mdev_create_sysfs_files(struct device *dev, struct mdev_type *type)
+{
+	int ret;
+
+	ret = sysfs_create_files(&dev->kobj, mdev_device_attrs);
+	if (ret)
+		return ret;
+
+	ret = sysfs_create_link(type->devices_kobj, &dev->kobj, dev_name(dev));
+	if (ret)
+		goto device_link_failed;
+
+	ret = sysfs_create_link(&dev->kobj, &type->kobj, "mdev_type");
+	if (ret)
+		goto type_link_failed;
+
+	return ret;
+
+type_link_failed:
+	sysfs_remove_link(type->devices_kobj, dev_name(dev));
+device_link_failed:
+	sysfs_remove_files(&dev->kobj, mdev_device_attrs);
+	return ret;
+}
+
+void mdev_remove_sysfs_files(struct device *dev, struct mdev_type *type)
+{
+	sysfs_remove_link(&dev->kobj, "mdev_type");
+	sysfs_remove_link(type->devices_kobj, dev_name(dev));
+	sysfs_remove_files(&dev->kobj, mdev_device_attrs);
+}
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
new file mode 100644
index 000000000000..0352febc1944
--- /dev/null
+++ b/include/linux/mdev.h
@@ -0,0 +1,167 @@
+/*
+ * Mediated device definition
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *	       Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef MDEV_H
+#define MDEV_H
+
+/* Parent Device */
+struct parent_device {
+	struct device		*dev;
+	const struct parent_ops	*ops;
+
+	/* internal */
+	struct kref		ref;
+	struct mutex		lock;
+	struct list_head	next;
+	struct kset		*mdev_types_kset;
+	struct list_head	type_list;
+};
+
+/* Mediated device */
+struct mdev_device {
+	struct device		dev;
+	struct parent_device	*parent;
+	uuid_le			uuid;
+	void			*driver_data;
+
+	/* internal */
+	struct kref		ref;
+	struct list_head	next;
+	struct kobject		*type_kobj;
+};
+
+
+/**
+ * struct parent_ops - Structure to be registered for each parent device to
+ * register the device to mdev module.
+ *
+ * @owner:		The module owner.
+ * @dev_attr_groups:	Attributes of the parent device.
+ * @mdev_attr_groups:	Attributes of the mediated device.
+ * @supported_type_groups: Attributes to define supported types. It is mandatory
+ *			to provide supported types.
+ * @create:		Called to allocate basic resources in parent device's
+ *			driver for a particular mediated device. It is
+ *			mandatory to provide create ops.
+ *			@kobj: kobject of type for which 'create' is called.
+ *			@mdev: mdev_device structure on of mediated device
+ *			      that is being created
+ *			Returns integer: success (0) or error (< 0)
+ * @remove:		Called to free resources in parent device's driver for a
+ *			a mediated device. It is mandatory to provide 'remove'
+ *			ops.
+ *			@mdev: mdev_device device structure which is being
+ *			       destroyed
+ *			Returns integer: success (0) or error (< 0)
+ * @open:		Open mediated device.
+ *			@mdev: mediated device.
+ *			Returns integer: success (0) or error (< 0)
+ * @release:		release mediated device
+ *			@mdev: mediated device.
+ * @read:		Read emulation callback
+ *			@mdev: mediated device structure
+ *			@buf: read buffer
+ *			@count: number of bytes to read
+ *			@ppos: address.
+ *			Retuns number on bytes read on success or error.
+ * @write:		Write emulation callback
+ *			@mdev: mediated device structure
+ *			@buf: write buffer
+ *			@count: number of bytes to be written
+ *			@ppos: address.
+ *			Retuns number on bytes written on success or error.
+ * @ioctl:		IOCTL callback
+ *			@mdev: mediated device structure
+ *			@cmd: mediated device structure
+ *			@arg: mediated device structure
+ * @mmap:		mmap callback
+ * Parent device that support mediated device should be registered with mdev
+ * module with parent_ops structure.
+ **/
+
+struct parent_ops {
+	struct module   *owner;
+	const struct attribute_group **dev_attr_groups;
+	const struct attribute_group **mdev_attr_groups;
+	struct attribute_group **supported_type_groups;
+
+	int     (*create)(struct kobject *kobj, struct mdev_device *mdev);
+	int     (*remove)(struct mdev_device *mdev);
+	int     (*open)(struct mdev_device *mdev);
+	void    (*release)(struct mdev_device *mdev);
+	ssize_t (*read)(struct mdev_device *mdev, char __user *buf,
+			size_t count, loff_t *ppos);
+	ssize_t (*write)(struct mdev_device *mdev, const char __user *buf,
+			 size_t count, loff_t *ppos);
+	ssize_t (*ioctl)(struct mdev_device *mdev, unsigned int cmd,
+			 unsigned long arg);
+	int	(*mmap)(struct mdev_device *mdev, struct vm_area_struct *vma);
+};
+
+/* interface for exporting mdev supported type attributes */
+struct mdev_type_attribute {
+	struct attribute attr;
+	ssize_t (*show)(struct kobject *kobj, struct device *dev, char *buf);
+	ssize_t (*store)(struct kobject *kobj, struct device *dev,
+			 const char *buf, size_t count);
+};
+
+#define MDEV_TYPE_ATTR(_name, _mode, _show, _store)		\
+struct mdev_type_attribute mdev_type_attr_##_name =		\
+	__ATTR(_name, _mode, _show, _store)
+#define MDEV_TYPE_ATTR_RW(_name) \
+	struct mdev_type_attribute mdev_type_attr_##_name = __ATTR_RW(_name)
+#define MDEV_TYPE_ATTR_RO(_name) \
+	struct mdev_type_attribute mdev_type_attr_##_name = __ATTR_RO(_name)
+#define MDEV_TYPE_ATTR_WO(_name) \
+	struct mdev_type_attribute mdev_type_attr_##_name = __ATTR_WO(_name)
+
+/**
+ * struct mdev_driver - Mediated device driver
+ * @name: driver name
+ * @probe: called when new device created
+ * @remove: called when device removed
+ * @driver: device driver structure
+ *
+ **/
+struct mdev_driver {
+	const char *name;
+	int  (*probe)(struct device *dev);
+	void (*remove)(struct device *dev);
+	struct device_driver driver;
+};
+
+#define to_mdev_driver(drv)	container_of(drv, struct mdev_driver, driver)
+#define to_mdev_device(dev)	container_of(dev, struct mdev_device, dev)
+
+static inline void *mdev_get_drvdata(struct mdev_device *mdev)
+{
+	return mdev->driver_data;
+}
+
+static inline void mdev_set_drvdata(struct mdev_device *mdev, void *data)
+{
+	mdev->driver_data = data;
+}
+
+extern struct bus_type mdev_bus_type;
+
+#define dev_is_mdev(d) ((d)->bus == &mdev_bus_type)
+
+extern int  mdev_register_device(struct device *dev,
+				 const struct parent_ops *ops);
+extern void mdev_unregister_device(struct device *dev);
+
+extern int  mdev_register_driver(struct mdev_driver *drv, struct module *owner);
+extern void mdev_unregister_driver(struct mdev_driver *drv);
+
+#endif /* MDEV_H */
-- 
2.7.0

[toc] | [prev] | [next] | [standalone]

#1511507 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
> +{
> +	int ret;
> +	struct parent_device *parent;
> +
> +	/* check for mandatory ops */
> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
> +		return -EINVAL;
> +
> +	dev = get_device(dev);
> +	if (!dev)
> +		return -EINVAL;
> +
> +	mutex_lock(&parent_list_lock);
> +
> +	/* Check for duplicate */
> +	parent = __find_parent_device(dev);
> +	if (parent) {
> +		ret = -EEXIST;
> +		goto add_dev_err;
> +	}
> +
> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
> +	if (!parent) {
> +		ret = -ENOMEM;
> +		goto add_dev_err;
> +	}
> +
> +	kref_init(&parent->ref);
> +	mutex_init(&parent->lock);
> +
> +	parent->dev = dev;
> +	parent->ops = ops;
> +
> +	ret = parent_create_sysfs_files(parent);
> +	if (ret) {
> +		mutex_unlock(&parent_list_lock);
> +		mdev_put_parent(parent);
> +		return ret;
> +	}
> +
> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
> +	if (ret)
> +		dev_warn(dev, "Failed to create compatibility class link\n");
> +

Hi Kirti,

Like I replied to previous version:

	http://www.spinics.net/lists/kvm/msg139331.html

You can always check if mdev_bus_compat_class already registered
here, and register it if not yet. Same logic should be adopted to
mdev_init.

Current implementation will simply panic if configured as builtin,
which is rare but far from impossible.

--
Thanks,
Jike


> +	list_add(&parent->next, &parent_list);
> +	mutex_unlock(&parent_list_lock);
> +
> +	dev_info(dev, "MDEV: Registered\n");
> +	return 0;
> +
> +add_dev_err:
> +	mutex_unlock(&parent_list_lock);
> +	put_device(dev);
> +	return ret;
> +}
> +EXPORT_SYMBOL(mdev_register_device);
> +
> +/*
> + * mdev_unregister_device : Unregister a parent device
> + * @dev: device structure representing parent device.
> + *
> + * Remove device from list of registered parent devices. Give a chance to free
> + * existing mediated devices for given device.
> + */
> +
> +void mdev_unregister_device(struct device *dev)
> +{
> +	struct parent_device *parent;
> +	bool force_remove = true;
> +
> +	mutex_lock(&parent_list_lock);
> +	parent = __find_parent_device(dev);
> +
> +	if (!parent) {
> +		mutex_unlock(&parent_list_lock);
> +		return;
> +	}
> +	dev_info(dev, "MDEV: Unregistering\n");
> +
> +	list_del(&parent->next);
> +	class_compat_remove_link(mdev_bus_compat_class, dev, NULL);
> +
> +	device_for_each_child(dev, (void *)&force_remove,
> +			      mdev_device_remove_cb);
> +
> +	parent_remove_sysfs_files(parent);
> +
> +	mutex_unlock(&parent_list_lock);
> +	mdev_put_parent(parent);
> +}
> +EXPORT_SYMBOL(mdev_unregister_device);
> +
> +static void mdev_device_release(struct device *dev)
> +{
> +	struct mdev_device *mdev = to_mdev_device(dev);
> +
> +	dev_dbg(&mdev->dev, "MDEV: destroying\n");
> +	kfree(mdev);
> +}
> +
> +int mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid)
> +{
> +	int ret;
> +	struct mdev_device *mdev;
> +	struct parent_device *parent;
> +	struct mdev_type *type = to_mdev_type(kobj);
> +
> +	parent = mdev_get_parent(type->parent);
> +	if (!parent)
> +		return -EINVAL;
> +
> +	mutex_lock(&parent->lock);
> +
> +	/* Check for duplicate */
> +	if (mdev_device_exist(parent, uuid)) {
> +		ret = -EEXIST;
> +		goto create_err;
> +	}
> +
> +	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
> +	if (!mdev) {
> +		ret = -ENOMEM;
> +		goto create_err;
> +	}
> +
> +	memcpy(&mdev->uuid, &uuid, sizeof(uuid_le));
> +	mdev->parent = parent;
> +	kref_init(&mdev->ref);
> +
> +	mdev->dev.parent  = dev;
> +	mdev->dev.bus     = &mdev_bus_type;
> +	mdev->dev.release = mdev_device_release;
> +	dev_set_name(&mdev->dev, "%pUl", uuid.b);
> +
> +	ret = device_register(&mdev->dev);
> +	if (ret) {
> +		put_device(&mdev->dev);
> +		goto create_err;
> +	}
> +
> +	ret = mdev_device_create_ops(kobj, mdev);
> +	if (ret)
> +		goto create_failed;
> +
> +	ret = mdev_create_sysfs_files(&mdev->dev, type);
> +	if (ret) {
> +		mdev_device_remove_ops(mdev, true);
> +		goto create_failed;
> +	}
> +
> +	mdev->type_kobj = kobj;
> +	dev_dbg(&mdev->dev, "MDEV: created\n");
> +
> +	mutex_unlock(&parent->lock);
> +	return ret;
> +
> +create_failed:
> +	device_unregister(&mdev->dev);
> +
> +create_err:
> +	mutex_unlock(&parent->lock);
> +	mdev_put_parent(parent);
> +	return ret;
> +}
> +
> +int mdev_device_remove(struct device *dev, bool force_remove)
> +{
> +	struct mdev_device *mdev;
> +	struct parent_device *parent;
> +	struct mdev_type *type;
> +	int ret;
> +
> +	mdev = to_mdev_device(dev);
> +	type = to_mdev_type(mdev->type_kobj);
> +	parent = mdev->parent;
> +	mutex_lock(&parent->lock);
> +
> +	ret = mdev_device_remove_ops(mdev, force_remove);
> +	if (ret) {
> +		mutex_unlock(&parent->lock);
> +		return ret;
> +	}
> +
> +	mdev_remove_sysfs_files(dev, type);
> +	device_unregister(dev);
> +	mutex_unlock(&parent->lock);
> +	mdev_put_parent(parent);
> +	return ret;
> +}
> +
> +static int __init mdev_init(void)
> +{
> +	int ret;
> +
> +	ret = mdev_bus_register();
> +	if (ret) {
> +		pr_err("Failed to register mdev bus\n");
> +		return ret;
> +	}
> +
> +	mdev_bus_compat_class = class_compat_register("mdev_bus");
> +	if (!mdev_bus_compat_class) {
> +		mdev_bus_unregister();
> +		return -ENOMEM;
> +	}
> +
> +	/*
> +	 * Attempt to load known vfio_mdev.  This gives us a working environment
> +	 * without the user needing to explicitly load vfio_mdev driver.
> +	 */
> +	request_module_nowait("vfio_mdev");
> +
> +	return ret;
> +}
> +
> +static void __exit mdev_exit(void)
> +{
> +	class_compat_unregister(mdev_bus_compat_class);
> +	mdev_bus_unregister();
> +}

[toc] | [prev] | [next] | [standalone]

#1511611 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

[Multipart message — attachments visible in raw view] — view raw

On 10/29/2016 10:00 AM, Jike Song wrote:
> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
>> +{
>> +	int ret;
>> +	struct parent_device *parent;
>> +
>> +	/* check for mandatory ops */
>> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
>> +		return -EINVAL;
>> +
>> +	dev = get_device(dev);
>> +	if (!dev)
>> +		return -EINVAL;
>> +
>> +	mutex_lock(&parent_list_lock);
>> +
>> +	/* Check for duplicate */
>> +	parent = __find_parent_device(dev);
>> +	if (parent) {
>> +		ret = -EEXIST;
>> +		goto add_dev_err;
>> +	}
>> +
>> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
>> +	if (!parent) {
>> +		ret = -ENOMEM;
>> +		goto add_dev_err;
>> +	}
>> +
>> +	kref_init(&parent->ref);
>> +	mutex_init(&parent->lock);
>> +
>> +	parent->dev = dev;
>> +	parent->ops = ops;
>> +
>> +	ret = parent_create_sysfs_files(parent);
>> +	if (ret) {
>> +		mutex_unlock(&parent_list_lock);
>> +		mdev_put_parent(parent);
>> +		return ret;
>> +	}
>> +
>> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
>> +	if (ret)
>> +		dev_warn(dev, "Failed to create compatibility class link\n");
>> +
> 
> Hi Kirti,
> 
> Like I replied to previous version:
> 
> 	http://www.spinics.net/lists/kvm/msg139331.html
> 

Hi Jike,

I saw your reply but by that time v10 version of patch series was out
for review.

> You can always check if mdev_bus_compat_class already registered
> here, and register it if not yet. Same logic should be adopted to
> mdev_init.
> 
> Current implementation will simply panic if configured as builtin,
> which is rare but far from impossible.
> 

Can you verify attached patch with v10 patch-set whether this works for you?
I'll incorporate this change in my next version.

Thanks,
Kirti

[toc] | [prev] | [next] | [standalone]

#1511857 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On 10/29/2016 06:06 PM, Kirti Wankhede wrote:
> 
> 
> On 10/29/2016 10:00 AM, Jike Song wrote:
>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
>>> +{
>>> +	int ret;
>>> +	struct parent_device *parent;
>>> +
>>> +	/* check for mandatory ops */
>>> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
>>> +		return -EINVAL;
>>> +
>>> +	dev = get_device(dev);
>>> +	if (!dev)
>>> +		return -EINVAL;
>>> +
>>> +	mutex_lock(&parent_list_lock);
>>> +
>>> +	/* Check for duplicate */
>>> +	parent = __find_parent_device(dev);
>>> +	if (parent) {
>>> +		ret = -EEXIST;
>>> +		goto add_dev_err;
>>> +	}
>>> +
>>> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
>>> +	if (!parent) {
>>> +		ret = -ENOMEM;
>>> +		goto add_dev_err;
>>> +	}
>>> +
>>> +	kref_init(&parent->ref);
>>> +	mutex_init(&parent->lock);
>>> +
>>> +	parent->dev = dev;
>>> +	parent->ops = ops;
>>> +
>>> +	ret = parent_create_sysfs_files(parent);
>>> +	if (ret) {
>>> +		mutex_unlock(&parent_list_lock);
>>> +		mdev_put_parent(parent);
>>> +		return ret;
>>> +	}
>>> +
>>> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
>>> +	if (ret)
>>> +		dev_warn(dev, "Failed to create compatibility class link\n");
>>> +
>>
>> Hi Kirti,
>>
>> Like I replied to previous version:
>>
>> 	http://www.spinics.net/lists/kvm/msg139331.html
>>
> 
> Hi Jike,
> 
> I saw your reply but by that time v10 version of patch series was out
> for review.
> 

Ah..yes, I forgot that :)

>> You can always check if mdev_bus_compat_class already registered
>> here, and register it if not yet. Same logic should be adopted to
>> mdev_init.
>>
>> Current implementation will simply panic if configured as builtin,
>> which is rare but far from impossible.
>>
> 
> Can you verify attached patch with v10 patch-set whether this works for you?
> I'll incorporate this change in my next version.
> 

Seems cool. But would you please also keep the register in mdev_init(),
just check the 'in case it was already registered' case? Thanks!

--
Thanks,
Jike

[toc] | [prev] | [next] | [standalone]

#1513726 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On 10/29/2016 11:41 PM, Jike Song wrote:
> On 10/29/2016 06:06 PM, Kirti Wankhede wrote:
>>
>>
>> On 10/29/2016 10:00 AM, Jike Song wrote:
>>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>>> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
>>>> +{
>>>> +	int ret;
>>>> +	struct parent_device *parent;
>>>> +
>>>> +	/* check for mandatory ops */
>>>> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
>>>> +		return -EINVAL;
>>>> +
>>>> +	dev = get_device(dev);
>>>> +	if (!dev)
>>>> +		return -EINVAL;
>>>> +
>>>> +	mutex_lock(&parent_list_lock);
>>>> +
>>>> +	/* Check for duplicate */
>>>> +	parent = __find_parent_device(dev);
>>>> +	if (parent) {
>>>> +		ret = -EEXIST;
>>>> +		goto add_dev_err;
>>>> +	}
>>>> +
>>>> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
>>>> +	if (!parent) {
>>>> +		ret = -ENOMEM;
>>>> +		goto add_dev_err;
>>>> +	}
>>>> +
>>>> +	kref_init(&parent->ref);
>>>> +	mutex_init(&parent->lock);
>>>> +
>>>> +	parent->dev = dev;
>>>> +	parent->ops = ops;
>>>> +
>>>> +	ret = parent_create_sysfs_files(parent);
>>>> +	if (ret) {
>>>> +		mutex_unlock(&parent_list_lock);
>>>> +		mdev_put_parent(parent);
>>>> +		return ret;
>>>> +	}
>>>> +
>>>> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
>>>> +	if (ret)
>>>> +		dev_warn(dev, "Failed to create compatibility class link\n");
>>>> +
>>>
>>> Hi Kirti,
>>>
>>> Like I replied to previous version:
>>>
>>> 	http://www.spinics.net/lists/kvm/msg139331.html
>>>
>>
>> Hi Jike,
>>
>> I saw your reply but by that time v10 version of patch series was out
>> for review.
>>
> 
> Ah..yes, I forgot that :)
> 
>>> You can always check if mdev_bus_compat_class already registered
>>> here, and register it if not yet. Same logic should be adopted to
>>> mdev_init.
>>>
>>> Current implementation will simply panic if configured as builtin,
>>> which is rare but far from impossible.
>>>
>>
>> Can you verify attached patch with v10 patch-set whether this works for you?
>> I'll incorporate this change in my next version.
>>
> 
> Seems cool. But would you please also keep the register in mdev_init(),
> just check the 'in case it was already registered' case? Thanks!
> 

The class is used only to keep symbolic to the devices which are
registered to mdev framework. So I think its ok to register this class
when first device is registered.

Thanks,
Kirti

[toc] | [prev] | [next] | [standalone]

#1513788 — Re: [Qemu-devel] [PATCH v10 01/19] vfio: Mediated device Core driver

On 11/02/2016 03:59 PM, Kirti Wankhede wrote:
> On 10/29/2016 11:41 PM, Jike Song wrote:
>> On 10/29/2016 06:06 PM, Kirti Wankhede wrote:
>>>
>>>
>>> On 10/29/2016 10:00 AM, Jike Song wrote:
>>>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>>>> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
>>>>> +{
>>>>> +	int ret;
>>>>> +	struct parent_device *parent;
>>>>> +
>>>>> +	/* check for mandatory ops */
>>>>> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
>>>>> +		return -EINVAL;
>>>>> +
>>>>> +	dev = get_device(dev);
>>>>> +	if (!dev)
>>>>> +		return -EINVAL;
>>>>> +
>>>>> +	mutex_lock(&parent_list_lock);
>>>>> +
>>>>> +	/* Check for duplicate */
>>>>> +	parent = __find_parent_device(dev);
>>>>> +	if (parent) {
>>>>> +		ret = -EEXIST;
>>>>> +		goto add_dev_err;
>>>>> +	}
>>>>> +
>>>>> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
>>>>> +	if (!parent) {
>>>>> +		ret = -ENOMEM;
>>>>> +		goto add_dev_err;
>>>>> +	}
>>>>> +
>>>>> +	kref_init(&parent->ref);
>>>>> +	mutex_init(&parent->lock);
>>>>> +
>>>>> +	parent->dev = dev;
>>>>> +	parent->ops = ops;
>>>>> +
>>>>> +	ret = parent_create_sysfs_files(parent);
>>>>> +	if (ret) {
>>>>> +		mutex_unlock(&parent_list_lock);
>>>>> +		mdev_put_parent(parent);
>>>>> +		return ret;
>>>>> +	}
>>>>> +
>>>>> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
>>>>> +	if (ret)
>>>>> +		dev_warn(dev, "Failed to create compatibility class link\n");
>>>>> +
>>>>
>>>> Hi Kirti,
>>>>
>>>> Like I replied to previous version:
>>>>
>>>> 	http://www.spinics.net/lists/kvm/msg139331.html
>>>>
>>>
>>> Hi Jike,
>>>
>>> I saw your reply but by that time v10 version of patch series was out
>>> for review.
>>>
>>
>> Ah..yes, I forgot that :)
>>
>>>> You can always check if mdev_bus_compat_class already registered
>>>> here, and register it if not yet. Same logic should be adopted to
>>>> mdev_init.
>>>>
>>>> Current implementation will simply panic if configured as builtin,
>>>> which is rare but far from impossible.
>>>>
>>>
>>> Can you verify attached patch with v10 patch-set whether this works for you?
>>> I'll incorporate this change in my next version.
>>>
>>
>> Seems cool. But would you please also keep the register in mdev_init(),
>> just check the 'in case it was already registered' case? Thanks!
>>
> 
> The class is used only to keep symbolic to the devices which are
> registered to mdev framework. So I think its ok to register this class
> when first device is registered.
> 

That's also cool :-)

So if you like it:

Reviewed-by: Jike Song <jike.song@intel.com>


--
Thanks,
Jike

[toc] | [prev] | [next] | [standalone]

#1513056 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> Design for Mediated Device Driver:
> Main purpose of this driver is to provide a common interface for mediated
> device management that can be used by different drivers of different
> devices.
> 
> This module provides a generic interface to create the device, add it to
> mediated bus, add device to IOMMU group and then add it to vfio group.
> 
> Below is the high Level block diagram, with Nvidia, Intel and IBM devices
> as example, since these are the devices which are going to actively use
> this module as of now.
> 
>  +---------------+
>  |               |
>  | +-----------+ |  mdev_register_driver() +--------------+
>  | |           | +<------------------------+ __init()     |
>  | |  mdev     | |                         |              |
>  | |  bus      | +------------------------>+              |<-> VFIO user
>  | |  driver   | |     probe()/remove()    | vfio_mdev.ko |    APIs
>  | |           | |                         |              |
>  | +-----------+ |                         +--------------+
>  |               |
>  |  MDEV CORE    |
>  |   MODULE      |
>  |   mdev.ko     |
>  | +-----------+ |  mdev_register_device() +--------------+
>  | |           | +<------------------------+              |
>  | |           | |                         |  nvidia.ko   |<-> physical
>  | |           | +------------------------>+              |    device
>  | |           | |        callback         +--------------+
>  | | Physical  | |
>  | |  device   | |  mdev_register_device() +--------------+
>  | | interface | |<------------------------+              |
>  | |           | |                         |  i915.ko     |<-> physical
>  | |           | +------------------------>+              |    device
>  | |           | |        callback         +--------------+
>  | |           | |
>  | |           | |  mdev_register_device() +--------------+
>  | |           | +<------------------------+              |
>  | |           | |                         | ccw_device.ko|<-> physical
>  | |           | +------------------------>+              |    device
>  | |           | |        callback         +--------------+
>  | +-----------+ |
>  +---------------+
> 
> Core driver provides two types of registration interfaces:
> 1. Registration interface for mediated bus driver:
> 
> /**
>   * struct mdev_driver - Mediated device's driver
>   * @name: driver name
>   * @probe: called when new device created
>   * @remove:called when device removed
>   * @driver:device driver structure
>   *
>   **/
> struct mdev_driver {
>          const char *name;
>          int  (*probe)  (struct device *dev);
>          void (*remove) (struct device *dev);
>          struct device_driver    driver;
> };
> 
> Mediated bus driver for mdev device should use this interface to register
> and unregister with core driver respectively:
> 
> int  mdev_register_driver(struct mdev_driver *drv, struct module *owner);
> void mdev_unregister_driver(struct mdev_driver *drv);
> 
> Mediated bus driver is responsible to add/delete mediated devices to/from
> VFIO group when devices are bound and unbound to the driver.
> 
> 2. Physical device driver interface
> This interface provides vendor driver the set APIs to manage physical
> device related work in its driver. APIs are :
> 
> * dev_attr_groups: attributes of the parent device.
> * mdev_attr_groups: attributes of the mediated device.
> * supported_type_groups: attributes to define supported type. This is
> 			 mandatory field.
> * create: to allocate basic resources in vendor driver for a mediated
>          device. This is mandatory to be provided by vendor driver.
> * remove: to free resources in vendor driver when mediated device is
>          destroyed. This is mandatory to be provided by vendor driver.
> * open: open callback of mediated device
> * release: release callback of mediated device
> * read : read emulation callback.
> * write: write emulation callback.
> * mmap: mmap emulation callback.
> * ioctl: ioctl callback.
> 
> Drivers should use these interfaces to register and unregister device to
> mdev core driver respectively:
> 
> extern int  mdev_register_device(struct device *dev,
>                                  const struct parent_ops *ops);
> extern void mdev_unregister_device(struct device *dev);
> 
> There are no locks to serialize above callbacks in mdev driver and
> vfio_mdev driver. If required, vendor driver can have locks to serialize
> above APIs in their driver.

Maybe some information above could be placed under Documentation instead?
It's kind of weird to have the block diagram and interfaces documentation
in the git commit message :-)

> 
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> Signed-off-by: Neo Jia <cjia@nvidia.com>
> Change-Id: I73a5084574270b14541c529461ea2f03c292d510
> ---
>  drivers/vfio/Kconfig             |   1 +
>  drivers/vfio/Makefile            |   1 +
>  drivers/vfio/mdev/Kconfig        |  10 +
>  drivers/vfio/mdev/Makefile       |   4 +
>  drivers/vfio/mdev/mdev_core.c    | 384 +++++++++++++++++++++++++++++++++++++++
>  drivers/vfio/mdev/mdev_driver.c  | 122 +++++++++++++
>  drivers/vfio/mdev/mdev_private.h |  41 +++++
>  drivers/vfio/mdev/mdev_sysfs.c   | 286 +++++++++++++++++++++++++++++
>  include/linux/mdev.h             | 167 +++++++++++++++++
>  9 files changed, 1016 insertions(+)
>  create mode 100644 drivers/vfio/mdev/Kconfig
>  create mode 100644 drivers/vfio/mdev/Makefile
>  create mode 100644 drivers/vfio/mdev/mdev_core.c
>  create mode 100644 drivers/vfio/mdev/mdev_driver.c
>  create mode 100644 drivers/vfio/mdev/mdev_private.h
>  create mode 100644 drivers/vfio/mdev/mdev_sysfs.c
>  create mode 100644 include/linux/mdev.h
> 
> diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig
> index da6e2ce77495..23eced02aaf6 100644
> --- a/drivers/vfio/Kconfig
> +++ b/drivers/vfio/Kconfig
> @@ -48,4 +48,5 @@ menuconfig VFIO_NOIOMMU
>  
>  source "drivers/vfio/pci/Kconfig"
>  source "drivers/vfio/platform/Kconfig"
> +source "drivers/vfio/mdev/Kconfig"
>  source "virt/lib/Kconfig"
> diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile
> index 7b8a31f63fea..4a23c13b6be4 100644
> --- a/drivers/vfio/Makefile
> +++ b/drivers/vfio/Makefile
> @@ -7,3 +7,4 @@ obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
>  obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o
>  obj-$(CONFIG_VFIO_PCI) += pci/
>  obj-$(CONFIG_VFIO_PLATFORM) += platform/
> +obj-$(CONFIG_VFIO_MDEV) += mdev/
> diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
> new file mode 100644
> index 000000000000..303c14ce2847
> --- /dev/null
> +++ b/drivers/vfio/mdev/Kconfig
> @@ -0,0 +1,10 @@
> +
> +config VFIO_MDEV
> +    tristate "Mediated device driver framework"
> +    depends on VFIO
> +    default n
> +    help
> +        Provides a framework to virtualize devices.
> +	See Documentation/vfio-mdev/vfio-mediated-device.txt for more details.
> +
> +        If you don't know what do here, say N.
> diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> new file mode 100644
> index 000000000000..31bc04801d94
> --- /dev/null
> +++ b/drivers/vfio/mdev/Makefile
> @@ -0,0 +1,4 @@
> +
> +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
> +
> +obj-$(CONFIG_VFIO_MDEV) += mdev.o
> diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> new file mode 100644
> index 000000000000..9d8fa5c91c2e
> --- /dev/null
> +++ b/drivers/vfio/mdev/mdev_core.c
> @@ -0,0 +1,384 @@
> +/*
> + * Mediated device Core Driver
> + *
> + * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
> + *     Author: Neo Jia <cjia@nvidia.com>
> + *	       Kirti Wankhede <kwankhede@nvidia.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/slab.h>
> +#include <linux/uuid.h>
> +#include <linux/sysfs.h>
> +#include <linux/mdev.h>
> +
> +#include "mdev_private.h"
> +
> +#define DRIVER_VERSION		"0.1"
> +#define DRIVER_AUTHOR		"NVIDIA Corporation"
> +#define DRIVER_DESC		"Mediated device Core Driver"
> +
> +static LIST_HEAD(parent_list);
> +static DEFINE_MUTEX(parent_list_lock);
> +static struct class_compat *mdev_bus_compat_class;
> +
> +static int _find_mdev_device(struct device *dev, void *data)
> +{
> +	struct mdev_device *mdev;
> +
> +	if (!dev_is_mdev(dev))
> +		return 0;
> +
> +	mdev = to_mdev_device(dev);
> +
> +	if (uuid_le_cmp(mdev->uuid, *(uuid_le *)data) == 0)
> +		return 1;
> +
> +	return 0;
> +}

> +
> +static bool mdev_device_exist(struct parent_device *parent, uuid_le uuid)
> +{
> +	struct device *dev;
> +
> +	dev = device_find_child(parent->dev, &uuid, _find_mdev_device);

'_find_mdev_device' is actually a match, better rename to '__match_mdev' or
something alike.

> +	if (dev) {
> +		put_device(dev);
> +		return true;
> +	}
> +
> +	return false;
> +}
> +
> +/* Should be called holding parent_list_lock */
> +static struct parent_device *__find_parent_device(struct device *dev)
> +{
> +	struct parent_device *parent;
> +
> +	list_for_each_entry(parent, &parent_list, next) {
> +		if (parent->dev == dev)
> +			return parent;
> +	}
> +	return NULL;
> +}
> +
> +static void mdev_release_parent(struct kref *kref)
> +{
> +	struct parent_device *parent = container_of(kref, struct parent_device,
> +						    ref);
> +	struct device *dev = parent->dev;
> +
> +	kfree(parent);
> +	put_device(dev);
> +}
> +
> +static
> +inline struct parent_device *mdev_get_parent(struct parent_device *parent)
> +{
> +	if (parent)
> +		kref_get(&parent->ref);
> +
> +	return parent;
> +}
> +
> +static inline void mdev_put_parent(struct parent_device *parent)
> +{
> +	if (parent)
> +		kref_put(&parent->ref, mdev_release_parent);
> +}
> +
> +static int mdev_device_create_ops(struct kobject *kobj,
> +				  struct mdev_device *mdev)
> +{
> +	struct parent_device *parent = mdev->parent;
> +	int ret;
> +
> +	ret = parent->ops->create(kobj, mdev);
> +	if (ret)
> +		return ret;
> +
> +	ret = sysfs_create_groups(&mdev->dev.kobj,
> +				  parent->ops->mdev_attr_groups);
> +	if (ret)
> +		parent->ops->remove(mdev);
> +
> +	return ret;
> +}
> +
> +/*
> + * mdev_device_remove_ops gets called from sysfs's 'remove' and when parent
> + * device is being unregistered from mdev device framework.
> + * - 'force_remove' is set to 'false' when called from sysfs's 'remove' which
> + *   indicates that if the mdev device is active, used by VMM or userspace
> + *   application, vendor driver could return error then don't remove the device.
> + * - 'force_remove' is set to 'true' when called from mdev_unregister_device()
> + *   which indicate that parent device is being removed from mdev device
> + *   framework so remove mdev device forcefully.
> + */
> +static int mdev_device_remove_ops(struct mdev_device *mdev, bool force_remove)
> +{
> +	struct parent_device *parent = mdev->parent;
> +	int ret;
> +
> +	/*
> +	 * Vendor driver can return error if VMM or userspace application is
> +	 * using this mdev device.
> +	 */
> +	ret = parent->ops->remove(mdev);
> +	if (ret && !force_remove)
> +		return -EBUSY;
> +
> +	sysfs_remove_groups(&mdev->dev.kobj, parent->ops->mdev_attr_groups);
> +	return 0;
> +}
> +
> +static int mdev_device_remove_cb(struct device *dev, void *data)
> +{
> +	if (!dev_is_mdev(dev))
> +		return 0;
> +
> +	return mdev_device_remove(dev, data ? *(bool *)data : true);
> +}
> +
> +/*
> + * mdev_register_device : Register a device
> + * @dev: device structure representing parent device.
> + * @ops: Parent device operation structure to be registered.
> + *
> + * Add device to list of registered parent devices.
> + * Returns a negative value on error, otherwise 0.
> + */
> +int mdev_register_device(struct device *dev, const struct parent_ops *ops)
> +{
> +	int ret;
> +	struct parent_device *parent;
> +
> +	/* check for mandatory ops */
> +	if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
> +		return -EINVAL;
> +
> +	dev = get_device(dev);
> +	if (!dev)
> +		return -EINVAL;
> +
> +	mutex_lock(&parent_list_lock);
> +
> +	/* Check for duplicate */
> +	parent = __find_parent_device(dev);
> +	if (parent) {
> +		ret = -EEXIST;
> +		goto add_dev_err;
> +	}
> +
> +	parent = kzalloc(sizeof(*parent), GFP_KERNEL);
> +	if (!parent) {
> +		ret = -ENOMEM;
> +		goto add_dev_err;
> +	}
> +
> +	kref_init(&parent->ref);
> +	mutex_init(&parent->lock);
> +
> +	parent->dev = dev;
> +	parent->ops = ops;
> +
> +	ret = parent_create_sysfs_files(parent);
> +	if (ret) {
> +		mutex_unlock(&parent_list_lock);
> +		mdev_put_parent(parent);
> +		return ret;
> +	}
> +
> +	ret = class_compat_create_link(mdev_bus_compat_class, dev, NULL);
> +	if (ret)
> +		dev_warn(dev, "Failed to create compatibility class link\n");
> +
> +	list_add(&parent->next, &parent_list);
> +	mutex_unlock(&parent_list_lock);
> +
> +	dev_info(dev, "MDEV: Registered\n");
> +	return 0;
> +
> +add_dev_err:
> +	mutex_unlock(&parent_list_lock);
> +	put_device(dev);
> +	return ret;
> +}
> +EXPORT_SYMBOL(mdev_register_device);
> +
> +/*
> + * mdev_unregister_device : Unregister a parent device
> + * @dev: device structure representing parent device.
> + *
> + * Remove device from list of registered parent devices. Give a chance to free
> + * existing mediated devices for given device.
> + */
> +
> +void mdev_unregister_device(struct device *dev)
> +{
> +	struct parent_device *parent;
> +	bool force_remove = true;
> +
> +	mutex_lock(&parent_list_lock);
> +	parent = __find_parent_device(dev);
> +
> +	if (!parent) {
> +		mutex_unlock(&parent_list_lock);
> +		return;
> +	}
> +	dev_info(dev, "MDEV: Unregistering\n");
> +
> +	list_del(&parent->next);
> +	class_compat_remove_link(mdev_bus_compat_class, dev, NULL);
> +
> +	device_for_each_child(dev, (void *)&force_remove,
> +			      mdev_device_remove_cb);
> +
> +	parent_remove_sysfs_files(parent);
> +
> +	mutex_unlock(&parent_list_lock);
> +	mdev_put_parent(parent);
> +}
> +EXPORT_SYMBOL(mdev_unregister_device);
> +
> +static void mdev_device_release(struct device *dev)
> +{
> +	struct mdev_device *mdev = to_mdev_device(dev);
> +
> +	dev_dbg(&mdev->dev, "MDEV: destroying\n");
> +	kfree(mdev);
> +}
> +
> +int mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid)
> +{
> +	int ret;
> +	struct mdev_device *mdev;
> +	struct parent_device *parent;
> +	struct mdev_type *type = to_mdev_type(kobj);
> +
> +	parent = mdev_get_parent(type->parent);
> +	if (!parent)
> +		return -EINVAL;
> +
> +	mutex_lock(&parent->lock);
> +
> +	/* Check for duplicate */
> +	if (mdev_device_exist(parent, uuid)) {
> +		ret = -EEXIST;
> +		goto create_err;
> +	}
> +
> +	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
> +	if (!mdev) {
> +		ret = -ENOMEM;
> +		goto create_err;
> +	}
> +
> +	memcpy(&mdev->uuid, &uuid, sizeof(uuid_le));
> +	mdev->parent = parent;
> +	kref_init(&mdev->ref);
> +
> +	mdev->dev.parent  = dev;
> +	mdev->dev.bus     = &mdev_bus_type;
> +	mdev->dev.release = mdev_device_release;
> +	dev_set_name(&mdev->dev, "%pUl", uuid.b);
> +
> +	ret = device_register(&mdev->dev);
> +	if (ret) {
> +		put_device(&mdev->dev);
> +		goto create_err;
> +	}
> +
> +	ret = mdev_device_create_ops(kobj, mdev);
> +	if (ret)
> +		goto create_failed;
> +
> +	ret = mdev_create_sysfs_files(&mdev->dev, type);
> +	if (ret) {
> +		mdev_device_remove_ops(mdev, true);
> +		goto create_failed;
> +	}
> +
> +	mdev->type_kobj = kobj;
> +	dev_dbg(&mdev->dev, "MDEV: created\n");
> +
> +	mutex_unlock(&parent->lock);
> +	return ret;
> +
> +create_failed:
> +	device_unregister(&mdev->dev);
> +
> +create_err:
> +	mutex_unlock(&parent->lock);
> +	mdev_put_parent(parent);
> +	return ret;
> +}
> +
> +int mdev_device_remove(struct device *dev, bool force_remove)
> +{
> +	struct mdev_device *mdev;
> +	struct parent_device *parent;
> +	struct mdev_type *type;
> +	int ret;
> +
> +	mdev = to_mdev_device(dev);
> +	type = to_mdev_type(mdev->type_kobj);
> +	parent = mdev->parent;
> +	mutex_lock(&parent->lock);
> +
> +	ret = mdev_device_remove_ops(mdev, force_remove);
> +	if (ret) {
> +		mutex_unlock(&parent->lock);
> +		return ret;
> +	}
> +
> +	mdev_remove_sysfs_files(dev, type);
> +	device_unregister(dev);
> +	mutex_unlock(&parent->lock);
> +	mdev_put_parent(parent);
> +	return ret;
> +}
> +
> +static int __init mdev_init(void)
> +{
> +	int ret;
> +
> +	ret = mdev_bus_register();
> +	if (ret) {
> +		pr_err("Failed to register mdev bus\n");
> +		return ret;
> +	}
> +
> +	mdev_bus_compat_class = class_compat_register("mdev_bus");
> +	if (!mdev_bus_compat_class) {
> +		mdev_bus_unregister();
> +		return -ENOMEM;
> +	}
> +
> +	/*
> +	 * Attempt to load known vfio_mdev.  This gives us a working environment
> +	 * without the user needing to explicitly load vfio_mdev driver.
> +	 */
> +	request_module_nowait("vfio_mdev");
> +
> +	return ret;
> +}
> +
> +static void __exit mdev_exit(void)
> +{
> +	class_compat_unregister(mdev_bus_compat_class);
> +	mdev_bus_unregister();
> +}
> +
> +module_init(mdev_init)
> +module_exit(mdev_exit)
> +
> +MODULE_VERSION(DRIVER_VERSION);
> +MODULE_LICENSE("GPL");
> +MODULE_AUTHOR(DRIVER_AUTHOR);
> +MODULE_DESCRIPTION(DRIVER_DESC);
> diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> new file mode 100644
> index 000000000000..0b3250044a80
> --- /dev/null
> +++ b/drivers/vfio/mdev/mdev_driver.c
> @@ -0,0 +1,122 @@
> +/*
> + * MDEV driver
> + *
> + * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
> + *     Author: Neo Jia <cjia@nvidia.com>
> + *	       Kirti Wankhede <kwankhede@nvidia.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/iommu.h>
> +#include <linux/mdev.h>
> +
> +#include "mdev_private.h"
> +
> +static int mdev_attach_iommu(struct mdev_device *mdev)
> +{
> +	int ret;
> +	struct iommu_group *group;
> +
> +	group = iommu_group_alloc();
> +	if (IS_ERR(group))
> +		return PTR_ERR(group);

Maybe I overthought, but where the iommu_group is released? On successful
return you already have a ref, I didn't find an iommu_group_put after that.

--
Thanks,
Jike

> +
> +	ret = iommu_group_add_device(group, &mdev->dev);
> +	if (ret)
> +		goto attach_fail;
> +
> +	dev_info(&mdev->dev, "MDEV: group_id = %d\n",
> +				 iommu_group_id(group));
> +attach_fail:
> +	iommu_group_put(group);
> +	return ret;
> +}
> +
> +static void mdev_detach_iommu(struct mdev_device *mdev)
> +{
> +	iommu_group_remove_device(&mdev->dev);
> +	dev_info(&mdev->dev, "MDEV: detaching iommu\n");
> +}
> +

[toc] | [prev] | [next] | [standalone]

#1513073 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On Tue, 01 Nov 2016 11:08:15 +0800
Jike Song <jike.song@intel.com> wrote:

> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> > +static int mdev_attach_iommu(struct mdev_device *mdev)
> > +{
> > +	int ret;
> > +	struct iommu_group *group;
> > +
> > +	group = iommu_group_alloc();
> > +	if (IS_ERR(group))
> > +		return PTR_ERR(group);  
> 
> Maybe I overthought, but where the iommu_group is released? On successful
> return you already have a ref, I didn't find an iommu_group_put after that.
...
> > +
> > +	ret = iommu_group_add_device(group, &mdev->dev);
> > +	if (ret)
> > +		goto attach_fail;
> > +
> > +	dev_info(&mdev->dev, "MDEV: group_id = %d\n",
> > +				 iommu_group_id(group));
> > +attach_fail:
> > +	iommu_group_put(group);
> > +	return ret;
> > +}
> > +
> > +static void mdev_detach_iommu(struct mdev_device *mdev)
> > +{
> > +	iommu_group_remove_device(&mdev->dev);

Here.  Adding a device to the group takes a group reference so we can
'put' the group to release the reference from the alloc as soon as a
device is added.  When we remove the device, that reference is removed
and this should result in a release of the group if all other
references are balanced.  Thanks,

Alex

> > +	dev_info(&mdev->dev, "MDEV: detaching iommu\n");
> > +}
> > +  
> 

[toc] | [prev] | [next] | [standalone]

#1513100 — Re: [PATCH v10 01/19] vfio: Mediated device Core driver

On 11/01/2016 11:44 AM, Alex Williamson wrote:
> On Tue, 01 Nov 2016 11:08:15 +0800
> Jike Song <jike.song@intel.com> wrote:
>> On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
>>> +static int mdev_attach_iommu(struct mdev_device *mdev)
>>> +{
>>> +	int ret;
>>> +	struct iommu_group *group;
>>> +
>>> +	group = iommu_group_alloc();
>>> +	if (IS_ERR(group))
>>> +		return PTR_ERR(group);  
>>
>> Maybe I overthought, but where the iommu_group is released? On successful
>> return you already have a ref, I didn't find an iommu_group_put after that.
> ...
>>> +
>>> +	ret = iommu_group_add_device(group, &mdev->dev);
>>> +	if (ret)
>>> +		goto attach_fail;
>>> +
>>> +	dev_info(&mdev->dev, "MDEV: group_id = %d\n",
>>> +				 iommu_group_id(group));
>>> +attach_fail:
>>> +	iommu_group_put(group);
>>> +	return ret;
>>> +}
>>> +
>>> +static void mdev_detach_iommu(struct mdev_device *mdev)
>>> +{
>>> +	iommu_group_remove_device(&mdev->dev);
> 
> Here.  Adding a device to the group takes a group reference so we can
> 'put' the group to release the reference from the alloc as soon as a
> device is added.  When we remove the device, that reference is removed
> and this should result in a release of the group if all other
> references are balanced.  Thanks,

Now I understand, thanks :)

--
Thanks,
Jike

[toc] | [prev] | [next] | [standalone]

#1509849 — [PATCH v10 17/19] docs: Add Documentation for Mediated devices

Add file Documentation/vfio-mediated-device.txt that include details of
mediated device framework.

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Change-Id: I137dd646442936090d92008b115908b7b2c7bc5d
---
 Documentation/vfio-mdev/vfio-mediated-device.txt | 297 +++++++++++++++++++++++
 1 file changed, 297 insertions(+)
 create mode 100644 Documentation/vfio-mdev/vfio-mediated-device.txt

diff --git a/Documentation/vfio-mdev/vfio-mediated-device.txt b/Documentation/vfio-mdev/vfio-mediated-device.txt
new file mode 100644
index 000000000000..69660aff86e3
--- /dev/null
+++ b/Documentation/vfio-mdev/vfio-mediated-device.txt
@@ -0,0 +1,297 @@
+/*
+ * VFIO Mediated devices
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *     Author: Neo Jia <cjia@nvidia.com>
+ *             Kirti Wankhede <kwankhede@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+Virtual Function I/O (VFIO) Mediated devices[1]
+===============================================
+
+The number of use cases for virtualizing DMA devices that do not have built-in
+SR_IOV capability is increasing. Previously, to virtualize such devices,
+developers had to create their own management interfaces and APIs, and then
+integrate them with user space software. To simplify integration with user space
+software, we have identified common requirements and a unified management
+interface for such devices.
+
+The VFIO driver framework provides unified APIs for direct device access. It is
+an IOMMU/device-agnostic framework for exposing direct device access to user
+space in a secure, IOMMU-protected environment. This framework is used for
+multiple devices, such as GPUs, network adapters, and compute accelerators. With
+direct device access, virtual machines or user space applications have direct
+access to the physical device. This framework is reused for mediated devices.
+
+The mediated core driver provides a common interface for mediated device
+management that can be used by drivers of different devices. This module
+provides a generic interface to perform these operations:
+
+* Create and destroy a mediated device
+* Add a mediated device to and remove it from a mediated bus driver
+* Add a mediated device to and remove it from an IOMMU group
+
+The mediated core driver also provides an interface to register a bus driver.
+For example, the mediated VFIO mdev driver is designed for mediated devices and
+supports VFIO APIs. The mediated bus driver adds a mediated device to and
+removes it from a VFIO group.
+
+The following high-level block diagram shows the main components and interfaces
+in the VFIO mediated driver framework. The diagram shows NVIDIA, Intel, and IBM
+devices as examples, as these devices are the first devices to use this module.
+
+     +---------------+
+     |               |
+     | +-----------+ |  mdev_register_driver() +--------------+
+     | |           | +<------------------------+              |
+     | |  mdev     | |                         |              |
+     | |  bus      | +------------------------>+ vfio_mdev.ko |<-> VFIO user
+     | |  driver   | |     probe()/remove()    |              |    APIs
+     | |           | |                         +--------------+
+     | +-----------+ |
+     |               |
+     |  MDEV CORE    |
+     |   MODULE      |
+     |   mdev.ko     |
+     | +-----------+ |  mdev_register_device() +--------------+
+     | |           | +<------------------------+              |
+     | |           | |                         |  nvidia.ko   |<-> physical
+     | |           | +------------------------>+              |    device
+     | |           | |        callbacks        +--------------+
+     | | Physical  | |
+     | |  device   | |  mdev_register_device() +--------------+
+     | | interface | |<------------------------+              |
+     | |           | |                         |  i915.ko     |<-> physical
+     | |           | +------------------------>+              |    device
+     | |           | |        callbacks        +--------------+
+     | |           | |
+     | |           | |  mdev_register_device() +--------------+
+     | |           | +<------------------------+              |
+     | |           | |                         | ccw_device.ko|<-> physical
+     | |           | +------------------------>+              |    device
+     | |           | |        callbacks        +--------------+
+     | +-----------+ |
+     +---------------+
+
+
+Registration Interfaces
+=======================
+
+The mediated core driver provides the following types of registration
+interfaces:
+
+* Registration interface for a mediated bus driver
+* Physical device driver interface
+
+Registration Interface for a Mediated Bus Driver
+------------------------------------------------
+
+The registration interface for a mediated bus driver provides the following
+structure to represent a mediated device's driver:
+
+     /*
+      * struct mdev_driver [2] - Mediated device's driver
+      * @name: driver name
+      * @probe: called when new device created
+      * @remove: called when device removed
+      * @driver: device driver structure
+      */
+     struct mdev_driver {
+	     const char *name;
+	     int  (*probe)  (struct device *dev);
+	     void (*remove) (struct device *dev);
+	     struct device_driver    driver;
+     };
+
+A mediated bus driver for mdev should use this structure in the function calls
+to register and unregister itself with the core driver:
+
+* Register:
+
+  extern int  mdev_register_driver(struct mdev_driver *drv,
+				   struct module *owner);
+
+* Unregister:
+
+  extern void mdev_unregister_driver(struct mdev_driver *drv);
+
+The mediated bus driver is responsible for adding mediated devices to the VFIO
+group when devices are bound to the driver and removing mediated devices from
+the VFIO when devices are unbound from the driver.
+
+
+Physical Device Driver Interface
+--------------------------------
+
+The physical device driver interface provides the parent_ops[3] structure to
+define the APIs to manage work in the mediated core driver that is related to
+the physical device.
+
+The structures in the parent_ops structure are as follows:
+
+* dev_attr_groups: attributes of the parent device
+* mdev_attr_groups: attributes of the mediated device
+* supported_config: attributes to define supported configurations
+
+The functions in the parent_ops structure are as follows:
+
+* create: allocate basic resources in a driver for a mediated device
+* remove: free resources in a driver when a mediated device is destroyed
+
+The callbacks in the parent_ops structure are as follows:
+
+* open: open callback of mediated device
+* close: close callback of mediated device
+* ioctl: ioctl callback of mediated device
+* read : read emulation callback
+* write: write emulation callback
+* mmap: mmap emulation callback
+
+A driver should use the parent_ops structure in the function call to register
+itself with the mdev core driver:
+
+extern int  mdev_register_device(struct device *dev,
+                                 const struct parent_ops *ops);
+
+However, the parent_ops structure is not required in the function call that a
+driver should use to unregister itself with the mdev core driver:
+
+extern void mdev_unregister_device(struct device *dev);
+
+
+Mediated Device Management Interface Through sysfs
+==================================================
+
+The management interface through sysfs enables user space software, such as
+libvirt, to query and configure mediated devices in a hardware-agnostic fashion.
+This management interface provides flexibility to the underlying physical
+device's driver to support features such as:
+
+* Mediated device hot plug
+* Multiple mediated devices in a single virtual machine
+* Multiple mediated devices from different physical devices
+
+Links in the mdev_bus Class Directory
+-------------------------------------
+The /sys/class/mdev_bus/ directory contains links to devices that are registered
+with the mdev core driver.
+
+Directories and files under the sysfs for Each Physical Device
+--------------------------------------------------------------
+
+|- [parent physical device]
+|--- Vendor-specific-attributes [optional]
+|--- [mdev_supported_types]
+|     |--- [<type-id>]
+|     |   |--- create
+|     |   |--- name
+|     |   |--- available_instances
+|     |   |--- device_api
+|     |   |--- description
+|     |   |--- [devices]
+|     |--- [<type-id>]
+|     |   |--- create
+|     |   |--- name
+|     |   |--- available_instances
+|     |   |--- device_api
+|     |   |--- description
+|     |   |--- [devices]
+|     |--- [<type-id>]
+|          |--- create
+|          |--- name
+|          |--- available_instances
+|          |--- device_api
+|          |--- description
+|          |--- [devices]
+
+* [mdev_supported_types]
+
+  The list of currently supported mediated device types and their details.
+
+  [<type-id>], device_api, and available_instances are mandatory attributes
+  that should be provided by vendor driver.
+
+* [<type-id>]
+
+  The [<type-id>] name is created by adding the the device driver string as a
+  prefix to the string provided by the vendor driver. This format of this name
+  is as follows:
+
+	sprintf(buf, "%s-%s", dev_driver_string(parent->dev), group->name);
+
+* device_api
+
+  This attribute should show which device API is being created, for example,
+  "vfio-pci" for a PCI device.
+
+* available_instances
+
+  This attribute should show the number of devices of type <type-id> that can be
+  created.
+
+* [device]
+
+  This directory contains links to the devices of type <type-id> that have been
+created.
+
+* name
+
+  This attribute should show human readable name. This is optional attribute.
+
+* description
+
+  This attribute should show brief features/description of the type. This is
+  optional attribute.
+
+Directories and Files Under the sysfs for Each mdev Device
+----------------------------------------------------------
+
+|- [parent phy device]
+|--- [$MDEV_UUID]
+         |--- remove
+         |--- mdev_type {link to its type}
+         |--- vendor-specific-attributes [optional]
+
+* remove (write only)
+Writing '1' to the 'remove' file destroys the mdev device. The vendor driver can
+fail the remove() callback if that device is active and the vendor driver
+doesn't support hot unplug.
+
+Example:
+	# echo 1 > /sys/bus/mdev/devices/$mdev_UUID/remove
+
+Mediated device Hot plug:
+------------------------
+
+Mediated devices can be created and assigned at runtime. The procedure to hot
+plug a mediated device is the same as the procedure to hot plug a PCI device.
+
+Translation APIs for Mediated Devices
+=====================================
+
+The following APIs are provided for translating user pfn to host pfn in a VFIO
+driver:
+
+ extern long vfio_pin_pages(struct device *dev, unsigned long *user_pfn,
+                            long npage, int prot, unsigned long *phys_pfn);
+
+ extern long vfio_unpin_pages(struct device *dev, unsigned long *pfn,
+			      long npage);
+
+These functions call back into the back-end IOMMU module by using the pin_pages
+and unpin_pages callbacks of the struct vfio_iommu_driver_ops[4]. Currently
+these callbacks are supported in the TYPE1 IOMMU module. To enable them for
+other IOMMU backend modules, such as PPC64 sPAPR module, they need to provide
+these two callback functions.
+
+References
+----------
+
+[1] See Documentation/vfio.txt for more information on VFIO.
+[2] struct mdev_driver in include/linux/mdev.h
+[3] struct parent_ops in include/linux/mdev.h
+[4] struct vfio_iommu_driver_ops in include/linux/vfio.h
-- 
2.7.0

[toc] | [prev] | [next] | [standalone]

#1509850 — [PATCH v10 03/19] vfio: Rearrange functions to get vfio_group from dev

This patch rearranges functions to get vfio_group from device

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Change-Id: I1f93262bdbab75094bc24b087b29da35ba70c4c6
---
 drivers/vfio/vfio.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index d1d70e0b011b..23bc86c1d05d 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -480,6 +480,21 @@ static struct vfio_group *vfio_group_get_from_minor(int minor)
 	return group;
 }
 
+static struct vfio_group *vfio_group_get_from_dev(struct device *dev)
+{
+	struct iommu_group *iommu_group;
+	struct vfio_group *group;
+
+	iommu_group = iommu_group_get(dev);
+	if (!iommu_group)
+		return NULL;
+
+	group = vfio_group_get_from_iommu(iommu_group);
+	iommu_group_put(iommu_group);
+
+	return group;
+}
+
 /**
  * Device objects - create, release, get, put, search
  */
@@ -811,16 +826,10 @@ EXPORT_SYMBOL_GPL(vfio_add_group_dev);
  */
 struct vfio_device *vfio_device_get_from_dev(struct device *dev)
 {
-	struct iommu_group *iommu_group;
 	struct vfio_group *group;
 	struct vfio_device *device;
 
-	iommu_group = iommu_group_get(dev);
-	if (!iommu_group)
-		return NULL;
-
-	group = vfio_group_get_from_iommu(iommu_group);
-	iommu_group_put(iommu_group);
+	group = vfio_group_get_from_dev(dev);
 	if (!group)
 		return NULL;
 
-- 
2.7.0

[toc] | [prev] | [next] | [standalone]

#1513800 — Re: [PATCH v10 03/19] vfio: Rearrange functions to get vfio_group from dev

On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> This patch rearranges functions to get vfio_group from device
> 
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> Signed-off-by: Neo Jia <cjia@nvidia.com>
> Change-Id: I1f93262bdbab75094bc24b087b29da35ba70c4c6
> ---
>  drivers/vfio/vfio.c | 23 ++++++++++++++++-------
>  1 file changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index d1d70e0b011b..23bc86c1d05d 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -480,6 +480,21 @@ static struct vfio_group *vfio_group_get_from_minor(int minor)
>  	return group;
>  }
>  
> +static struct vfio_group *vfio_group_get_from_dev(struct device *dev)
> +{
> +	struct iommu_group *iommu_group;
> +	struct vfio_group *group;
> +
> +	iommu_group = iommu_group_get(dev);
> +	if (!iommu_group)
> +		return NULL;
> +
> +	group = vfio_group_get_from_iommu(iommu_group);
> +	iommu_group_put(iommu_group);
> +
> +	return group;
> +}
> +
>  /**
>   * Device objects - create, release, get, put, search
>   */
> @@ -811,16 +826,10 @@ EXPORT_SYMBOL_GPL(vfio_add_group_dev);
>   */
>  struct vfio_device *vfio_device_get_from_dev(struct device *dev)
>  {
> -	struct iommu_group *iommu_group;
>  	struct vfio_group *group;
>  	struct vfio_device *device;
>  
> -	iommu_group = iommu_group_get(dev);
> -	if (!iommu_group)
> -		return NULL;
> -
> -	group = vfio_group_get_from_iommu(iommu_group);
> -	iommu_group_put(iommu_group);
> +	group = vfio_group_get_from_dev(dev);
>  	if (!group)
>  		return NULL;
>  

Reviewed-by: Jike Song <jike.song@intel.com>

--
Thanks,
Jike

[toc] | [prev] | [next] | [standalone]

#1509854 — [PATCH v10 18/19] docs: Sysfs ABI for mediated device framework

Added details of sysfs ABI for mediated device framework

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Change-Id: Icb0fd4ed58a2fa793fbcb1c3d5009a4403c1f3ac
---
 Documentation/ABI/testing/sysfs-bus-vfio-mdev | 111 ++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev

diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev b/Documentation/ABI/testing/sysfs-bus-vfio-mdev
new file mode 100644
index 000000000000..452dbe39270e
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev
@@ -0,0 +1,111 @@
+What:           /sys/.../<device>/mdev_supported_types/
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+                This directory contains list of directories of currently
+		supported mediated device types and their details for
+		<device>. Supported type attributes are defined by the
+		vendor driver who registers with Mediated device framework.
+		Each supported type is a directory whose name is created
+		by adding the device driver string as a prefix to the
+		string provided by the vendor driver.
+
+What:           /sys/.../<device>/mdev_supported_types/<type-id>/
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+                This directory gives details of supported type, like name,
+		description, available_instances, device_api etc.
+		'device_api' and 'available_instances' are mandatory
+		attributes to be provided by vendor driver. 'name',
+		'description' and other vendor driver specific attributes
+		are optional.
+
+What:           /sys/.../mdev_supported_types/<type-id>/create
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Writing UUID to this file will create mediated device of
+		type <type-id> for parent device <device>. This is a
+		write-only file.
+		For example:
+		# echo "83b8f4f2-509f-382f-3c1e-e6bfe0fa1001" >	\
+		       /sys/devices/foo/mdev_supported_types/foo-1/create
+
+What:           /sys/.../mdev_supported_types/<type-id>/devices/
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		This directory contains symbolic links pointing to mdev
+		devices sysfs entries which are created of this <type-id>.
+
+What:           /sys/.../mdev_supported_types/<type-id>/available_instances
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Reading this attribute will show the number of mediated
+		devices of type <type-id> that can be created. This is a
+		readonly file.
+Users:
+		Userspace applications interested in creating mediated
+		device of that type. Userspace application should check
+		the number of available instances could be created before
+		creating mediated device of this type.
+
+What:           /sys/.../mdev_supported_types/<type-id>/device_api
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Reading this attribute will show VFIO device API supported
+		by this type. For example, "vfio-pci" for a PCI device,
+		"vfio-platform" for platform device.
+
+What:           /sys/.../mdev_supported_types/<type-id>/name
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Reading this attribute will show human readable name of the
+		mediated device that will get created of type <type-id>.
+		This is optional attribute. For example: "Grid M60-0Q"
+Users:
+		Userspace applications interested in knowing the name of
+		a particular <type-id> that can help in understanding the
+		type of mediated device.
+
+What:           /sys/.../mdev_supported_types/<type-id>/description
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Reading this attribute will show description of the type of
+		mediated device that will get created of type <type-id>.
+		This is optional attribute. For example:
+		"2 heads, 512M FB, 2560x1600 maximum resolution"
+Users:
+		Userspace applications interested in knowing the details of
+		a particular <type-id> that can help in understanding the
+		features provided by that type of mediated device.
+
+What:           /sys/.../<device>/<UUID>/
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		This directory represents device directory of mediated
+		device. It contains all the attributes related to mediated
+		device.
+
+What:           /sys/.../<device>/<UUID>/mdev_type
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		This is symbolic link pointing to supported type, <type-id>
+		directory of which this mediated device is created.
+
+What:           /sys/.../<device>/<UUID>/remove
+Date:           October 2016
+Contact:        Kirti Wankhede <kwankhede@nvidia.com>
+Description:
+		Writing '1' to this file destroys the mediated device. The
+		vendor driver can fail the remove() callback if that device
+		is active and the vendor driver doesn't support hot unplug.
+		Example:
+		# echo 1 > /sys/bus/mdev/devices/<UUID>/remove
-- 
2.7.0

[toc] | [prev] | [next] | [standalone]

#1512381 — Re: [PATCH v10 18/19] docs: Sysfs ABI for mediated device framework

On 10/27/2016 05:29 AM, Kirti Wankhede wrote:
> Added details of sysfs ABI for mediated device framework
> 
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> Signed-off-by: Neo Jia <cjia@nvidia.com>
> Change-Id: Icb0fd4ed58a2fa793fbcb1c3d5009a4403c1f3ac
> ---
>  Documentation/ABI/testing/sysfs-bus-vfio-mdev | 111 ++++++++++++++++++++++++++
>  1 file changed, 111 insertions(+)
>  create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev
> 
> diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev b/Documentation/ABI/testing/sysfs-bus-vfio-mdev
> new file mode 100644
> index 000000000000..452dbe39270e
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev
> @@ -0,0 +1,111 @@
> +What:           /sys/.../<device>/mdev_supported_types/
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +                This directory contains list of directories of currently
> +		supported mediated device types and their details for
> +		<device>. Supported type attributes are defined by the
> +		vendor driver who registers with Mediated device framework.
> +		Each supported type is a directory whose name is created
> +		by adding the device driver string as a prefix to the
> +		string provided by the vendor driver.
> +
> +What:           /sys/.../<device>/mdev_supported_types/<type-id>/
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +                This directory gives details of supported type, like name,
> +		description, available_instances, device_api etc.
> +		'device_api' and 'available_instances' are mandatory
> +		attributes to be provided by vendor driver. 'name',
> +		'description' and other vendor driver specific attributes
> +		are optional.
> +

Hi Kirti,

Is there any checking in the mdev framework that mandatory attributes
are actually provided?

--
Thanks,
Jike

> +What:           /sys/.../mdev_supported_types/<type-id>/create
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Writing UUID to this file will create mediated device of
> +		type <type-id> for parent device <device>. This is a
> +		write-only file.
> +		For example:
> +		# echo "83b8f4f2-509f-382f-3c1e-e6bfe0fa1001" >	\
> +		       /sys/devices/foo/mdev_supported_types/foo-1/create
> +
> +What:           /sys/.../mdev_supported_types/<type-id>/devices/
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		This directory contains symbolic links pointing to mdev
> +		devices sysfs entries which are created of this <type-id>.
> +
> +What:           /sys/.../mdev_supported_types/<type-id>/available_instances
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Reading this attribute will show the number of mediated
> +		devices of type <type-id> that can be created. This is a
> +		readonly file.
> +Users:
> +		Userspace applications interested in creating mediated
> +		device of that type. Userspace application should check
> +		the number of available instances could be created before
> +		creating mediated device of this type.
> +
> +What:           /sys/.../mdev_supported_types/<type-id>/device_api
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Reading this attribute will show VFIO device API supported
> +		by this type. For example, "vfio-pci" for a PCI device,
> +		"vfio-platform" for platform device.
> +
> +What:           /sys/.../mdev_supported_types/<type-id>/name
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Reading this attribute will show human readable name of the
> +		mediated device that will get created of type <type-id>.
> +		This is optional attribute. For example: "Grid M60-0Q"
> +Users:
> +		Userspace applications interested in knowing the name of
> +		a particular <type-id> that can help in understanding the
> +		type of mediated device.
> +
> +What:           /sys/.../mdev_supported_types/<type-id>/description
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Reading this attribute will show description of the type of
> +		mediated device that will get created of type <type-id>.
> +		This is optional attribute. For example:
> +		"2 heads, 512M FB, 2560x1600 maximum resolution"
> +Users:
> +		Userspace applications interested in knowing the details of
> +		a particular <type-id> that can help in understanding the
> +		features provided by that type of mediated device.
> +
> +What:           /sys/.../<device>/<UUID>/
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		This directory represents device directory of mediated
> +		device. It contains all the attributes related to mediated
> +		device.
> +
> +What:           /sys/.../<device>/<UUID>/mdev_type
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		This is symbolic link pointing to supported type, <type-id>
> +		directory of which this mediated device is created.
> +
> +What:           /sys/.../<device>/<UUID>/remove
> +Date:           October 2016
> +Contact:        Kirti Wankhede <kwankhede@nvidia.com>
> +Description:
> +		Writing '1' to this file destroys the mediated device. The
> +		vendor driver can fail the remove() callback if that device
> +		is active and the vendor driver doesn't support hot unplug.
> +		Example:
> +		# echo 1 > /sys/bus/mdev/devices/<UUID>/remove
> 

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.kernel

csiph-web