Re: Summary of LPC guest MSI discussion in Santa Fe

From	Will Deacon <will.deacon@arm.com>
Newsgroups	linux.kernel
Subject	Re: Summary of LPC guest MSI discussion in Santa Fe
Date	2016-11-10 00:40 +0100
Message-ID	<sBKDf-5OO-25@gated-at.bofh.it> (permalink)
References	(5 earlier) <sBGJj-34V-1@gated-at.bofh.it> <sBHP4-3Kz-31@gated-at.bofh.it> <sBJnP-50l-1@gated-at.bofh.it> <sBJxw-56q-23@gated-at.bofh.it> <sBKtz-5LS-13@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Wed, Nov 09, 2016 at 04:24:58PM -0700, Alex Williamson wrote:
> On Wed, 9 Nov 2016 22:25:22 +0000
> Will Deacon <will.deacon@arm.com> wrote:
> 
> > On Wed, Nov 09, 2016 at 03:17:09PM -0700, Alex Williamson wrote:
> > > On Wed, 9 Nov 2016 20:31:45 +0000
> > > Will Deacon <will.deacon@arm.com> wrote:  
> > > > On Wed, Nov 09, 2016 at 08:23:03PM +0100, Christoffer Dall wrote:  
> > > > > 
> > > > > (I suppose it's technically possible to get around this issue by letting
> > > > > QEMU place RAM wherever it wants but tell the guest to never use a
> > > > > particular subset of its RAM for DMA, because that would conflict with
> > > > > the doorbell IOVA or be seen as p2p transactions.  But I think we all
> > > > > probably agree that it's a disgusting idea.)    
> > > > 
> > > > Disgusting, yes, but Ben's idea of hotplugging on the host controller with
> > > > firmware tables describing the reserved regions is something that we could
> > > > do in the distant future. In the meantime, I don't think that VFIO should
> > > > explicitly reject overlapping mappings if userspace asks for them.  
> > > 
> > > I'm confused by the last sentence here, rejecting user mappings that
> > > overlap reserved ranges, such as MSI doorbell pages, is exactly how
> > > we'd reject hot-adding a device when we meet such a conflict.  If we
> > > don't reject such a mapping, we're knowingly creating a situation that
> > > potentially leads to data loss.  Minimally, QEMU would need to know
> > > about the reserved region, map around it through VFIO, and take
> > > responsibility (somehow) for making sure that region is never used for
> > > DMA.  Thanks,  
> > 
> > Yes, but my point is that it should be up to QEMU to abort the hotplug, not
> > the host kernel, since there may be ways in which a guest can tolerate the
> > overlapping region (e.g. by avoiding that range of memory for DMA).
> 
> The VFIO_IOMMU_MAP_DMA ioctl is a contract, the user ask to map a range
> of IOVAs to a range of virtual addresses for a given device.  If VFIO
> cannot reasonably fulfill that contract, it must fail.  It's up to QEMU
> how to manage the hotplug and what memory regions it asks VFIO to map
> for a device, but VFIO must reject mappings that it (or the SMMU by
> virtue of using the IOMMU API) know to overlap reserved ranges.  So I
> still disagree with the referenced statement.  Thanks,

I think that's a pity. Not only does it mean that both QEMU and the kernel
have more work to do (the former has to carve up its mapping requests,
whilst the latter has to check that it is indeed doing this), but it also
precludes the use of hugepage mappings on the IOMMU because of reserved
regions. For example, a 4k hole someplace may mean we can't put down 1GB
table entries for the guest memory in the SMMU.

All this seems to do is add complexity and decrease performance. For what?
QEMU has to go read the reserved regions from someplace anyway. It's also
the way that VFIO works *today* on arm64 wrt reserved regions, it just has
no way to identify those holes at present.

Will

Thread

[RFC 0/8] KVM PCIe/MSI passthrough on ARM/ARM64 (Alt II) Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 2/8] iommu/iova: fix __alloc_and_insert_iova_range Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 4/8] iommu: Add a list of iommu_reserved_region in iommu_domain Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 5/8] vfio/type1: Introduce RESV_IOVA_RANGE capability Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 3/8] iommu/dma: Allow MSI-only cookies Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 7/8] iommu/vt-d: Implement add_reserved_regions callback Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 1/8] vfio: fix vfio_info_cap_add/shift Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  [RFC 6/8] iommu: Handle the list of reserved regions Eric Auger <eric.auger@redhat.com> - 2016-11-03 22:50 +0100
  Re: [RFC 0/8] KVM PCIe/MSI passthrough on ARM/ARM64 (Alt II) Alex Williamson <alex.williamson@redhat.com> - 2016-11-04 05:10 +0100
    Summary of LPC guest MSI discussion in Santa Fe (was: Re: [RFC 0/8]  KVM PCIe/MSI passthrough on ARM/ARM64 (Alt II)) Will Deacon <will.deacon@arm.com> - 2016-11-08 03:50 +0100
      Re: Summary of LPC guest MSI discussion in Santa Fe Auger Eric <eric.auger@redhat.com> - 2016-11-08 15:30 +0100
        Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-08 19:00 +0100
          Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-08 20:10 +0100
            Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-08 20:20 +0100
            Re: Summary of LPC guest MSI discussion in Santa Fe Auger Eric <eric.auger@redhat.com> - 2016-11-09 08:50 +0100
      Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-08 17:10 +0100
      Re: Summary of LPC guest MSI discussion in Santa Fe (was: Re: [RFC  0/8] KVM PCIe/MSI passthrough on ARM/ARM64 (Alt II)) Christoffer Dall <christoffer.dall@linaro.org> - 2016-11-08 21:30 +0100
        Re: Summary of LPC guest MSI discussion in Santa Fe (was: Re: [RFC  0/8] KVM PCIe/MSI passthrough on ARM/ARM64 (Alt II)) Alex Williamson <alex.williamson@redhat.com> - 2016-11-09 00:40 +0100
          Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-09 04:00 +0100
            Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-09 18:10 +0100
              Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-09 20:00 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Christoffer Dall <christoffer.dall@linaro.org> - 2016-11-09 20:30 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-09 21:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Joerg Roedel <joro@8bytes.org> - 2016-11-10 15:50 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-10 18:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-09 21:40 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-09 23:20 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-09 23:30 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-10 00:30 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-10 00:40 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-10 01:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Auger Eric <eric.auger@redhat.com> - 2016-11-10 01:20 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-10 02:00 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Will Deacon <will.deacon@arm.com> - 2016-11-10 03:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Auger Eric <eric.auger@redhat.com> - 2016-11-10 12:20 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-10 18:50 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Joerg Roedel <joro@8bytes.org> - 2016-11-11 12:20 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-11 17:00 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Alex Williamson <alex.williamson@redhat.com> - 2016-11-11 17:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-11 17:30 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Don Dutile <ddutile@redhat.com> - 2016-11-11 17:10 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Joerg Roedel <joro@8bytes.org> - 2016-11-10 16:00 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Robin Murphy <robin.murphy@arm.com> - 2016-11-09 21:20 +0100
                Re: Summary of LPC guest MSI discussion in Santa Fe Joerg Roedel <joro@8bytes.org> - 2016-11-10 16:30 +0100

csiph-web