Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1367306 > unrolled thread

Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Eric Auger <eric.auger@linaro.org> - 2016-03-30 17:40 +0200
    Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Will Deacon <will.deacon@arm.com> - 2016-03-30 17:50 +0200
      Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Eric Auger <eric.auger@linaro.org> - 2016-03-30 18:00 +0200
        Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Will Deacon <will.deacon@arm.com> - 2016-03-30 19:30 +0200
          Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Eric Auger <eric.auger@linaro.org> - 2016-03-31 09:50 +0200
            Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node Will Deacon <will.deacon@arm.com> - 2016-03-31 19:40 +0200

#1367306 — Re: [PATCH 11/13] dtb: amd: Add PCIe SMMU device tree node

Dear all,
On 01/28/2016 03:27 PM, Will Deacon wrote:
> On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
>> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
>>>>
>>>> Will, Robin, thoughts?
>>>
>>> Any IDs specified here would only apply to DMA by the "platform device" 
>>> side of the host controller itself (as would an equivalent "iommus" 
>>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
>>> working on). In terms of PCI devices, the "mmu-masters" property is 
>>> overloaded such that only its existence matters, to identify that there 
>>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
>>> host controller.
>>
>> I wasn't aware that this was actually still specified. I had hoped
>> we were getting rid of mmu-masters before anyone actually started
>> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
>>
>> Does anyone know what happened to the plan to use the iommu DT binding
>> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
> 
> We always did -- Seattle used the mmu-masters binding before the generic
> binding even existed. Robin has been working on patches to get of_xlate
> up and running, but it got held up by Laurent's series which didn't end
> up going anywhere.
> 
> Will
Up to now I have used the PCI smmu description as described in Suravee's
patch and this does not work anymore with 4.6-rc1 since the default
domain was introduced. So now I see 2 SMRs matching a single streamid
(in my case 256, one steming from the "platform device" side of the host
controller and one steming from the PCI device) and this causes SMCF
(stream match conflict fault). So PCIe PF does not work.

I observe the fault only when I override the PCIe ACS property in my
case which was pretty confusing (each PF/VF is put in a separate group).

What is the correct syntax then: mmu-masters = <&pcie0>?

instead of

+			 mmu-masters = <&pcie0
+				/* 1:00:[0,3] */ 256 257 258 259
+				/* 2:00:[0,3] */ 512 513 514 515
+				/* 3:00:[0,3] */ 768 769 770 771
+				/* 4:00:[0,3] */ 1024 1025 1026 1027>

Is there a plan to update/upstream the dt description for PCIe smmu.

Thank you in advance

Best Regards

Eric


> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

[toc] | [next] | [standalone]

#1367312

Hi Eric,

On Wed, Mar 30, 2016 at 05:37:27PM +0200, Eric Auger wrote:
> On 01/28/2016 03:27 PM, Will Deacon wrote:
> > On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
> >> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
> >>>>
> >>> Any IDs specified here would only apply to DMA by the "platform device" 
> >>> side of the host controller itself (as would an equivalent "iommus" 
> >>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
> >>> working on). In terms of PCI devices, the "mmu-masters" property is 
> >>> overloaded such that only its existence matters, to identify that there 
> >>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
> >>> host controller.
> >>
> >> I wasn't aware that this was actually still specified. I had hoped
> >> we were getting rid of mmu-masters before anyone actually started
> >> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
> >>
> >> Does anyone know what happened to the plan to use the iommu DT binding
> >> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
> > 
> > We always did -- Seattle used the mmu-masters binding before the generic
> > binding even existed. Robin has been working on patches to get of_xlate
> > up and running, but it got held up by Laurent's series which didn't end
> > up going anywhere.
> > 
> Up to now I have used the PCI smmu description as described in Suravee's
> patch and this does not work anymore with 4.6-rc1 since the default
> domain was introduced. So now I see 2 SMRs matching a single streamid
> (in my case 256, one steming from the "platform device" side of the host
> controller and one steming from the PCI device) and this causes SMCF
> (stream match conflict fault). So PCIe PF does not work.

Sorry about that, it wasn't intentional. In fact, I wrote commit
cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now")
specifically to avoid this breakage, after seeing it myself with VFIO
and an S2CR-based configuration. It looks like the check just needs moving
higher up (i.e. before we initialise the SMRs).

Does that fix it for you?

Will

[toc] | [prev] | [next] | [standalone]

#1367320

Hi Will,
On 03/30/2016 05:45 PM, Will Deacon wrote:
> Hi Eric,
> 
> On Wed, Mar 30, 2016 at 05:37:27PM +0200, Eric Auger wrote:
>> On 01/28/2016 03:27 PM, Will Deacon wrote:
>>> On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
>>>> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
>>>>>>
>>>>> Any IDs specified here would only apply to DMA by the "platform device" 
>>>>> side of the host controller itself (as would an equivalent "iommus" 
>>>>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
>>>>> working on). In terms of PCI devices, the "mmu-masters" property is 
>>>>> overloaded such that only its existence matters, to identify that there 
>>>>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
>>>>> host controller.
>>>>
>>>> I wasn't aware that this was actually still specified. I had hoped
>>>> we were getting rid of mmu-masters before anyone actually started
>>>> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
>>>>
>>>> Does anyone know what happened to the plan to use the iommu DT binding
>>>> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
>>>
>>> We always did -- Seattle used the mmu-masters binding before the generic
>>> binding even existed. Robin has been working on patches to get of_xlate
>>> up and running, but it got held up by Laurent's series which didn't end
>>> up going anywhere.
>>>
>> Up to now I have used the PCI smmu description as described in Suravee's
>> patch and this does not work anymore with 4.6-rc1 since the default
>> domain was introduced. So now I see 2 SMRs matching a single streamid
>> (in my case 256, one steming from the "platform device" side of the host
>> controller and one steming from the PCI device) and this causes SMCF
>> (stream match conflict fault). So PCIe PF does not work.
> 
> Sorry about that, it wasn't intentional. In fact, I wrote commit
> cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now")
> specifically to avoid this breakage, after seeing it myself with VFIO
> and an S2CR-based configuration. It looks like the check just needs moving
> higher up (i.e. before we initialise the SMRs).
> 
> Does that fix it for you?
Yes this fixes the issue for me, thanks! I guess you will send that patch?

So eventually what is the right way to describe the smmu-masters (~
future of that patch)?

Best Regards

Eric
> 
> Will
> 

[toc] | [prev] | [next] | [standalone]

#1367480

On Wed, Mar 30, 2016 at 05:57:08PM +0200, Eric Auger wrote:
> On 03/30/2016 05:45 PM, Will Deacon wrote:
> > On Wed, Mar 30, 2016 at 05:37:27PM +0200, Eric Auger wrote:
> >> On 01/28/2016 03:27 PM, Will Deacon wrote:
> >>> On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
> >>>> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
> >>>>>>
> >>>>> Any IDs specified here would only apply to DMA by the "platform device" 
> >>>>> side of the host controller itself (as would an equivalent "iommus" 
> >>>>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
> >>>>> working on). In terms of PCI devices, the "mmu-masters" property is 
> >>>>> overloaded such that only its existence matters, to identify that there 
> >>>>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
> >>>>> host controller.
> >>>>
> >>>> I wasn't aware that this was actually still specified. I had hoped
> >>>> we were getting rid of mmu-masters before anyone actually started
> >>>> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
> >>>>
> >>>> Does anyone know what happened to the plan to use the iommu DT binding
> >>>> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
> >>>
> >>> We always did -- Seattle used the mmu-masters binding before the generic
> >>> binding even existed. Robin has been working on patches to get of_xlate
> >>> up and running, but it got held up by Laurent's series which didn't end
> >>> up going anywhere.
> >>>
> >> Up to now I have used the PCI smmu description as described in Suravee's
> >> patch and this does not work anymore with 4.6-rc1 since the default
> >> domain was introduced. So now I see 2 SMRs matching a single streamid
> >> (in my case 256, one steming from the "platform device" side of the host
> >> controller and one steming from the PCI device) and this causes SMCF
> >> (stream match conflict fault). So PCIe PF does not work.
> > 
> > Sorry about that, it wasn't intentional. In fact, I wrote commit
> > cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now")
> > specifically to avoid this breakage, after seeing it myself with VFIO
> > and an S2CR-based configuration. It looks like the check just needs moving
> > higher up (i.e. before we initialise the SMRs).
> > 
> > Does that fix it for you?
> Yes this fixes the issue for me, thanks! I guess you will send that patch?

I need to check that it doesn't break rebinding to the host after VFIO
has been used for passthrough, first.

Does your devicetree explicitly assign a StreamID to the platform device
for the host controller? We should probably be handling this, since it
will crop us as an issue again once we decide to enable the default
domain properly.

> So eventually what is the right way to describe the smmu-masters (~
> future of that patch)?

Using the generic iommu binding
(Documentation/devicetree/bindings/iommu/iommu.txt), which will be required
for of_xlate-based probing. The old binding should still continue to
function as it always has, though.

Will

[toc] | [prev] | [next] | [standalone]

#1367919

Hi Will,
On 03/30/2016 07:24 PM, Will Deacon wrote:
> On Wed, Mar 30, 2016 at 05:57:08PM +0200, Eric Auger wrote:
>> On 03/30/2016 05:45 PM, Will Deacon wrote:
>>> On Wed, Mar 30, 2016 at 05:37:27PM +0200, Eric Auger wrote:
>>>> On 01/28/2016 03:27 PM, Will Deacon wrote:
>>>>> On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
>>>>>> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
>>>>>>>>
>>>>>>> Any IDs specified here would only apply to DMA by the "platform device" 
>>>>>>> side of the host controller itself (as would an equivalent "iommus" 
>>>>>>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
>>>>>>> working on). In terms of PCI devices, the "mmu-masters" property is 
>>>>>>> overloaded such that only its existence matters, to identify that there 
>>>>>>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
>>>>>>> host controller.
>>>>>>
>>>>>> I wasn't aware that this was actually still specified. I had hoped
>>>>>> we were getting rid of mmu-masters before anyone actually started
>>>>>> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
>>>>>>
>>>>>> Does anyone know what happened to the plan to use the iommu DT binding
>>>>>> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
>>>>>
>>>>> We always did -- Seattle used the mmu-masters binding before the generic
>>>>> binding even existed. Robin has been working on patches to get of_xlate
>>>>> up and running, but it got held up by Laurent's series which didn't end
>>>>> up going anywhere.
>>>>>
>>>> Up to now I have used the PCI smmu description as described in Suravee's
>>>> patch and this does not work anymore with 4.6-rc1 since the default
>>>> domain was introduced. So now I see 2 SMRs matching a single streamid
>>>> (in my case 256, one steming from the "platform device" side of the host
>>>> controller and one steming from the PCI device) and this causes SMCF
>>>> (stream match conflict fault). So PCIe PF does not work.
>>>
>>> Sorry about that, it wasn't intentional. In fact, I wrote commit
>>> cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now")
>>> specifically to avoid this breakage, after seeing it myself with VFIO
>>> and an S2CR-based configuration. It looks like the check just needs moving
>>> higher up (i.e. before we initialise the SMRs).
>>>
>>> Does that fix it for you?
>> Yes this fixes the issue for me, thanks! I guess you will send that patch?
> 
> I need to check that it doesn't break rebinding to the host after VFIO
> has been used for passthrough, first.

OK. I can help testing too since I am currently working on PCIe
passthrough respin.
> 
> Does your devicetree explicitly assign a StreamID to the platform device
> for the host controller? We should probably be handling this, since it
> will crop us as an issue again once we decide to enable the default
> domain properly.

Yes it does. My dt description currently matches the one found in this
patch from Suravee (old binding style). Hence my question. Shall I
remove this and let PCIe register their RID=SID automatically?
> 
>> So eventually what is the right way to describe the smmu-masters (~
>> future of that patch)?
> 
> Using the generic iommu binding
> (Documentation/devicetree/bindings/iommu/iommu.txt), which will be required
> for of_xlate-based probing. The old binding should still continue to
> function as it always has, though.
OK thanks

Eric
> 
> Will
> 

[toc] | [prev] | [next] | [standalone]

#1368487

On Thu, Mar 31, 2016 at 09:39:38AM +0200, Eric Auger wrote:
> On 03/30/2016 07:24 PM, Will Deacon wrote:
> > On Wed, Mar 30, 2016 at 05:57:08PM +0200, Eric Auger wrote:
> >> On 03/30/2016 05:45 PM, Will Deacon wrote:
> >>> On Wed, Mar 30, 2016 at 05:37:27PM +0200, Eric Auger wrote:
> >>>> On 01/28/2016 03:27 PM, Will Deacon wrote:
> >>>>> On Thu, Jan 28, 2016 at 03:17:33PM +0100, Arnd Bergmann wrote:
> >>>>>> On Thursday 28 January 2016 12:20:58 Robin Murphy wrote:
> >>>>>>>>
> >>>>>>> Any IDs specified here would only apply to DMA by the "platform device" 
> >>>>>>> side of the host controller itself (as would an equivalent "iommus" 
> >>>>>>> property on pcie0 once I finish the SMMUv2 generic binding support I'm 
> >>>>>>> working on). In terms of PCI devices, the "mmu-masters" property is 
> >>>>>>> overloaded such that only its existence matters, to identify that there 
> >>>>>>> _is_ a relationship between the SMMU and the PCI bus(es) behind that 
> >>>>>>> host controller.
> >>>>>>
> >>>>>> I wasn't aware that this was actually still specified. I had hoped
> >>>>>> we were getting rid of mmu-masters before anyone actually started
> >>>>>> using it, but now I see it in ns2.dtsi and fsl-ls2080a.dtsi.
> >>>>>>
> >>>>>> Does anyone know what happened to the plan to use the iommu DT binding
> >>>>>> for the ARM SMMU instead? Do we now have to support both ways indefinitely?
> >>>>>
> >>>>> We always did -- Seattle used the mmu-masters binding before the generic
> >>>>> binding even existed. Robin has been working on patches to get of_xlate
> >>>>> up and running, but it got held up by Laurent's series which didn't end
> >>>>> up going anywhere.
> >>>>>
> >>>> Up to now I have used the PCI smmu description as described in Suravee's
> >>>> patch and this does not work anymore with 4.6-rc1 since the default
> >>>> domain was introduced. So now I see 2 SMRs matching a single streamid
> >>>> (in my case 256, one steming from the "platform device" side of the host
> >>>> controller and one steming from the PCI device) and this causes SMCF
> >>>> (stream match conflict fault). So PCIe PF does not work.
> >>>
> >>> Sorry about that, it wasn't intentional. In fact, I wrote commit
> >>> cbf8277ef456 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now")
> >>> specifically to avoid this breakage, after seeing it myself with VFIO
> >>> and an S2CR-based configuration. It looks like the check just needs moving
> >>> higher up (i.e. before we initialise the SMRs).
> >>>
> >>> Does that fix it for you?
> >> Yes this fixes the issue for me, thanks! I guess you will send that patch?
> > 
> > I need to check that it doesn't break rebinding to the host after VFIO
> > has been used for passthrough, first.
> 
> OK. I can help testing too since I am currently working on PCIe
> passthrough respin.

Thanks, that would be really helpful if you have the time.

Will

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web