Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1404127 > unrolled thread

Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Jon Masters <jcm@redhat.com> - 2016-05-20 06:50 +0200
    Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-05-20 09:40 +0200
      Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Jon Masters <jcm@redhat.com> - 2016-05-20 10:10 +0200
        Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-05-20 10:30 +0200
          RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-20 10:50 +0200
            Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-05-20 11:20 +0200
              Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> - 2016-05-23 13:00 +0200
                RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-23 17:20 +0200
                  Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Bjorn Helgaas <helgaas@kernel.org> - 2016-05-24 01:40 +0200
                    Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Jon Masters <jcm@redhat.com> - 2016-05-24 03:20 +0200
                      Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Jon Masters <jcm@redhat.com> - 2016-05-24 03:50 +0200
                      RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-24 16:40 +0200
                    RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-24 09:30 +0200
                      Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Jon Masters <jcm@redhat.com> - 2016-05-24 16:40 +0200
                    Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> - 2016-05-24 19:30 +0200
                      Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller Jon Masters <jcm@redhat.com> - 2016-05-24 19:40 +0200
                      Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Bjorn Helgaas <helgaas@kernel.org> - 2016-05-24 21:10 +0200
                        RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-26 12:00 +0200
                      RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-25 08:40 +0200
                Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Jon Masters <jcm@redhat.com> - 2016-05-24 06:30 +0200
      RE: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Gabriele Paoloni <gabriele.paoloni@huawei.com> - 2016-05-20 10:20 +0200
        Re: [PATCH V7 00/11] Support for generic ACPI based PCI host  controller Jon Masters <jcm@redhat.com> - 2016-05-20 10:30 +0200

Page 1 of 2  [1] 2  Next page →

#1404127 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

Hi Tomasz, all,

On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:

> On 11.05.2016 12:41, Gabriele Paoloni wrote:

>>> v6 -> v7
>>> - drop quirks handling
>>
>> Maybe I missed something in the v6 discussion thread; when was it
>> decided to drop quirk handling?
> 
> I had such requests in previous series.

A quick note on quirk handling. This, I believe, applies post-merge of
the base infrastructure, which I realize will likely not have quirks.

We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like) due
to people repurposing existing embedded PCIe IP blocks or using them for
the first time (especially in servers), and those being involved in the
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.

Anyway. We missed this one in our SBSA requirements. They say (words to
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that actually
is in reality. That is being fixed. A lot of things are happening behind
the scenes - especially with third party IP block providers (all of whom
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't have
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.

Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches, especially
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't expect
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.

So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there to
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the small
number of existing offenders, we undermine efforts to force vendors to
ensure that their IP blocks are compliant going forward.

Side note: if you're a third party IP vendor and we didn't already speak
about this one, drop me a line, and let's collaborate also on your test
frameworks to make sure you're covered as well.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

[toc] | [next] | [standalone]

#1404198 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

On 20 May 2016 at 06:41, Jon Masters <jcm@redhat.com> wrote:
> Hi Tomasz, all,
>
> On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
>
>> On 11.05.2016 12:41, Gabriele Paoloni wrote:
>
>>>> v6 -> v7
>>>> - drop quirks handling
>>>
>>> Maybe I missed something in the v6 discussion thread; when was it
>>> decided to drop quirk handling?
>>
>> I had such requests in previous series.
>
> A quick note on quirk handling. This, I believe, applies post-merge of
> the base infrastructure, which I realize will likely not have quirks.
>
> We've some "gen1" ARMv8 server platforms where we end up doing quirks
> (for things like forcing 32-bit config space accessors and the like) due
> to people repurposing existing embedded PCIe IP blocks or using them for
> the first time (especially in servers), and those being involved in the
> design not necessarily seeing this problem ahead of time, or not
> realizing that it would be an issue for servers. In the early days of
> ARM server designs 3-4 years ago, many of us had never really played
> with ECAM or realized how modern topologies are built.
>
> Anyway. We missed this one in our SBSA requirements. They say (words to
> the effect of) "thou shalt do PCIe the way it is done on servers" but
> they aren't prescriptive, and they don't tell people how that actually
> is in reality. That is being fixed. A lot of things are happening behind
> the scenes - especially with third party IP block providers (all of whom
> myself and others are speaking with directly about this) - to ensure
> that the next wave of designs won't repeat these mistakes. We don't have
> a time machine, but we can contain this from becoming an ongoing mess
> for upstream, and we will do so. It won't be a zoo.
>
> Various proposals have arisen for how to handle quirks in the longer
> term, including elaborate frameworks and tables to describe them
> generically. I would like to caution against such approaches, especially
> in the case that they deviate from practice on x86, or prior to being
> standardized fully with other Operating System vendors. I don't expect
> there to be too many more than the existing initial set of quirks we
> have seen posted. A number of "future" server SoCs have already been
> fixed prior to silicon, and new design starts are being warned not to
> make this a problem for us to have to clean up later.
>
> So, I would like to suggest that the eventual framework mirror the
> existing approach on x86 systems (matching DMI, etc.) and not be made
> into some kind of generic, utopia. This is a case where we want there to
> be pain involved (and upstream patches required) when people screw up,
> so that they have a level of pain in response to ever making this
> mistake in the future. If we try to create too grand a generic scheme
> and make it too easy to handle this kind of situation beyond the small
> number of existing offenders, we undermine efforts to force vendors to
> ensure that their IP blocks are compliant going forward.
>

I understand that there is a desire from the RedHat side to mimic x86
as closely as possible, but I never saw any technical justification
for that. DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the kernel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only thing
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage this
first gen hardware to misrepresent non-compliant hardware as compliant
as well.

Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic? This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far as
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small pool
of third party IP could potentially be identified directly rather than
based on some divination of the SoC we may or may not be running on. I
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had to
make because they did not have any choice to begin with.

If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.

-- 
Ard.

[toc] | [prev] | [next] | [standalone]

#1404213

Hi Ard,

On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
> On 20 May 2016 at 06:41, Jon Masters <jcm@redhat.com> wrote:
>> Hi Tomasz, all,
>>
>> On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
>>
>>> On 11.05.2016 12:41, Gabriele Paoloni wrote:
>>
>>>>> v6 -> v7
>>>>> - drop quirks handling
>>>>
>>>> Maybe I missed something in the v6 discussion thread; when was it
>>>> decided to drop quirk handling?
>>>
>>> I had such requests in previous series.
>>
>> A quick note on quirk handling. This, I believe, applies post-merge of
>> the base infrastructure, which I realize will likely not have quirks.
>>
>> We've some "gen1" ARMv8 server platforms where we end up doing quirks
>> (for things like forcing 32-bit config space accessors and the like) due
>> to people repurposing existing embedded PCIe IP blocks or using them for
>> the first time (especially in servers), and those being involved in the
>> design not necessarily seeing this problem ahead of time, or not
>> realizing that it would be an issue for servers. In the early days of
>> ARM server designs 3-4 years ago, many of us had never really played
>> with ECAM or realized how modern topologies are built.
>>
>> Anyway. We missed this one in our SBSA requirements. They say (words to
>> the effect of) "thou shalt do PCIe the way it is done on servers" but
>> they aren't prescriptive, and they don't tell people how that actually
>> is in reality. That is being fixed. A lot of things are happening behind
>> the scenes - especially with third party IP block providers (all of whom
>> myself and others are speaking with directly about this) - to ensure
>> that the next wave of designs won't repeat these mistakes. We don't have
>> a time machine, but we can contain this from becoming an ongoing mess
>> for upstream, and we will do so. It won't be a zoo.
>>
>> Various proposals have arisen for how to handle quirks in the longer
>> term, including elaborate frameworks and tables to describe them
>> generically. I would like to caution against such approaches, especially
>> in the case that they deviate from practice on x86, or prior to being
>> standardized fully with other Operating System vendors. I don't expect
>> there to be too many more than the existing initial set of quirks we
>> have seen posted. A number of "future" server SoCs have already been
>> fixed prior to silicon, and new design starts are being warned not to
>> make this a problem for us to have to clean up later.
>>
>> So, I would like to suggest that the eventual framework mirror the
>> existing approach on x86 systems (matching DMI, etc.) and not be made
>> into some kind of generic, utopia. This is a case where we want there to
>> be pain involved (and upstream patches required) when people screw up,
>> so that they have a level of pain in response to ever making this
>> mistake in the future. If we try to create too grand a generic scheme
>> and make it too easy to handle this kind of situation beyond the small
>> number of existing offenders, we undermine efforts to force vendors to
>> ensure that their IP blocks are compliant going forward.

> I understand that there is a desire from the RedHat side to mimic x86
> as closely as possible, but I never saw any technical justification
> for that.

Understood. My own motivation is always to make the experience as
familiar as possible, both for end users, as well as for ODMs and the
entire ecosystem. There are very many ODMs currently working on v8
server designs and they're already expecting this to be "just like x88".
Intentionally. But as to the specifics of using DMI...

> DMI contains strings that are visible to userland, and you
> effectively lock those down to certain values just so that the kernel
> can distinguish a broken PCIe root complex from a working one. Linux
> on x86 had no choice, since the overwhelming majority of existing
> hardware misrepresented itself as generic, and DMI was the only thing
> available to actually distinguish these broken implementations from
> one another. This does not mean we should allow and/or encourage this
> first gen hardware to misrepresent non-compliant hardware as compliant
> as well.

That's a very reasonable argument. I don't disagree that it would be
nice to have nicer ways to distinguish the non-compliant IP than
treating the whole platform with an ASCII matching sledgehammer.

> Since you are talking to all the people involved, how about you
> convince them to put something in the ACPI tables that allows the
> kernel to distinguish those non-standard PCIe implementations from
> hardware that is really generic?

I'm open to this *BUT* it has to be something that will be adopted
beyond Linux. I have reached out to some non-Linux folks about this. If
there's buy-in, and if there's agreement to go standardize it through
the ASWG, then we should do so. What we should not do is treat ARM as
special in a way that the others aren't involved with. I'll admit DMI
ended up part of the SBBR in part because I wrote that piece in with the
assumption that exactly the same matches as on x86 would happen.

> This way, we can sidestep the quirks
> debate entirely, since it will simply be a different device as far as
> the kernel is concerned. This is no worse than a quirk from a
> practical point of view, since an older OS will be equally unable to
> run on newer hardware, but it is arguably more true to the standards
> compliance you tend to preach about, especially since this small pool
> of third party IP could potentially be identified directly rather than
> based on some divination of the SoC we may or may not be running on. I
> am also convinced that adding support for an additional HID() to the
> ACPI ECAM driver with some special config space handling wired in is
> an easier sell upstream than making the same ugly mess x86 has had to
> make because they did not have any choice to begin with.

Again, open to it. I just don't want to do something that's Linux
specific. So it'll take time. It would be awesome if an interim quirk
solution existed that got platforms that are shipping in production
(e.g. HP Moonshot) actually booting upstream kernels this year. We
/really/ want F25 to be able to run on these without needing to carry an
out-of-tree quirk patch or just not support them. That's much worse.

> If we do need a quirks handling mechanism, I still don't see how the
> x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
> SoC vendor to identify the make and particular revision, and quirks
> could be keyed off of that.

Fair enough. Is there any traction for an interim solution for these
initial platforms do you think? If we wait to add a new table it's
probably going to be the end of the year before we get this done.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

[toc] | [prev] | [next] | [standalone]

#1404233 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

On 20 May 2016 at 10:01, Jon Masters <jcm@redhat.com> wrote:
> Hi Ard,
>
> On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
>> On 20 May 2016 at 06:41, Jon Masters <jcm@redhat.com> wrote:
>>> Hi Tomasz, all,
>>>
>>> On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
>>>
>>>> On 11.05.2016 12:41, Gabriele Paoloni wrote:
>>>
>>>>>> v6 -> v7
>>>>>> - drop quirks handling
>>>>>
>>>>> Maybe I missed something in the v6 discussion thread; when was it
>>>>> decided to drop quirk handling?
>>>>
>>>> I had such requests in previous series.
>>>
>>> A quick note on quirk handling. This, I believe, applies post-merge of
>>> the base infrastructure, which I realize will likely not have quirks.
>>>
>>> We've some "gen1" ARMv8 server platforms where we end up doing quirks
>>> (for things like forcing 32-bit config space accessors and the like) due
>>> to people repurposing existing embedded PCIe IP blocks or using them for
>>> the first time (especially in servers), and those being involved in the
>>> design not necessarily seeing this problem ahead of time, or not
>>> realizing that it would be an issue for servers. In the early days of
>>> ARM server designs 3-4 years ago, many of us had never really played
>>> with ECAM or realized how modern topologies are built.
>>>
>>> Anyway. We missed this one in our SBSA requirements. They say (words to
>>> the effect of) "thou shalt do PCIe the way it is done on servers" but
>>> they aren't prescriptive, and they don't tell people how that actually
>>> is in reality. That is being fixed. A lot of things are happening behind
>>> the scenes - especially with third party IP block providers (all of whom
>>> myself and others are speaking with directly about this) - to ensure
>>> that the next wave of designs won't repeat these mistakes. We don't have
>>> a time machine, but we can contain this from becoming an ongoing mess
>>> for upstream, and we will do so. It won't be a zoo.
>>>
>>> Various proposals have arisen for how to handle quirks in the longer
>>> term, including elaborate frameworks and tables to describe them
>>> generically. I would like to caution against such approaches, especially
>>> in the case that they deviate from practice on x86, or prior to being
>>> standardized fully with other Operating System vendors. I don't expect
>>> there to be too many more than the existing initial set of quirks we
>>> have seen posted. A number of "future" server SoCs have already been
>>> fixed prior to silicon, and new design starts are being warned not to
>>> make this a problem for us to have to clean up later.
>>>
>>> So, I would like to suggest that the eventual framework mirror the
>>> existing approach on x86 systems (matching DMI, etc.) and not be made
>>> into some kind of generic, utopia. This is a case where we want there to
>>> be pain involved (and upstream patches required) when people screw up,
>>> so that they have a level of pain in response to ever making this
>>> mistake in the future. If we try to create too grand a generic scheme
>>> and make it too easy to handle this kind of situation beyond the small
>>> number of existing offenders, we undermine efforts to force vendors to
>>> ensure that their IP blocks are compliant going forward.
>
>> I understand that there is a desire from the RedHat side to mimic x86
>> as closely as possible, but I never saw any technical justification
>> for that.
>
> Understood. My own motivation is always to make the experience as
> familiar as possible, both for end users, as well as for ODMs and the
> entire ecosystem. There are very many ODMs currently working on v8
> server designs and they're already expecting this to be "just like x88".
> Intentionally. But as to the specifics of using DMI...
>
>> DMI contains strings that are visible to userland, and you
>> effectively lock those down to certain values just so that the kernel
>> can distinguish a broken PCIe root complex from a working one. Linux
>> on x86 had no choice, since the overwhelming majority of existing
>> hardware misrepresented itself as generic, and DMI was the only thing
>> available to actually distinguish these broken implementations from
>> one another. This does not mean we should allow and/or encourage this
>> first gen hardware to misrepresent non-compliant hardware as compliant
>> as well.
>
> That's a very reasonable argument. I don't disagree that it would be
> nice to have nicer ways to distinguish the non-compliant IP than
> treating the whole platform with an ASCII matching sledgehammer.
>
>> Since you are talking to all the people involved, how about you
>> convince them to put something in the ACPI tables that allows the
>> kernel to distinguish those non-standard PCIe implementations from
>> hardware that is really generic?
>
> I'm open to this *BUT* it has to be something that will be adopted
> beyond Linux. I have reached out to some non-Linux folks about this. If
> there's buy-in, and if there's agreement to go standardize it through
> the ASWG, then we should do so. What we should not do is treat ARM as
> special in a way that the others aren't involved with. I'll admit DMI
> ended up part of the SBBR in part because I wrote that piece in with the
> assumption that exactly the same matches as on x86 would happen.
>

Is the PCIe root complex so special that you cannot simply describe an
implementation that is not PNP0408 compatible as something else, under
its own unique HID? If everybody is onboard with using ACPI, how is
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already deviated
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.

>> This way, we can sidestep the quirks
>> debate entirely, since it will simply be a different device as far as
>> the kernel is concerned. This is no worse than a quirk from a
>> practical point of view, since an older OS will be equally unable to
>> run on newer hardware, but it is arguably more true to the standards
>> compliance you tend to preach about, especially since this small pool
>> of third party IP could potentially be identified directly rather than
>> based on some divination of the SoC we may or may not be running on. I
>> am also convinced that adding support for an additional HID() to the
>> ACPI ECAM driver with some special config space handling wired in is
>> an easier sell upstream than making the same ugly mess x86 has had to
>> make because they did not have any choice to begin with.
>
> Again, open to it. I just don't want to do something that's Linux
> specific. So it'll take time. It would be awesome if an interim quirk
> solution existed that got platforms that are shipping in production
> (e.g. HP Moonshot) actually booting upstream kernels this year. We
> /really/ want F25 to be able to run on these without needing to carry an
> out-of-tree quirk patch or just not support them. That's much worse.
>

My whole point is that we don't need quirks in the first place if
non-compliant devices are not being misrepresented as compliant ones.
This is fine for other platform devices, i.e., SATA, network, so
again, why is PCIe so special that we *must* use a generic ID + quirks
rather than a specific ID?

>> If we do need a quirks handling mechanism, I still don't see how the
>> x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
>> SoC vendor to identify the make and particular revision, and quirks
>> could be keyed off of that.
>
> Fair enough. Is there any traction for an interim solution for these
> initial platforms do you think? If we wait to add a new table it's
> probably going to be the end of the year before we get this done.
>

The 'interim solution' is to come to terms with the fact that these
initial platforms are not SBSA compliant, contain a PCIe root complex
that is not PNP0408 but can be identified by its own HID, and we make
the software work with that. This means our message from the beginning
is that, yes, you can have non-compliant hardware and the burden is on
you to get it supported upstream, and no, you don't get to hang out
with the cool SBSA kids if you decide to go that route

[toc] | [prev] | [next] | [standalone]

#1404260

Hi Ard

> -----Original Message-----
> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> Sent: 20 May 2016 09:29
> To: Jon Masters
> Cc: Tomasz Nowicki; Gabriele Paoloni; helgaas@kernel.org;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com;
> okaya@codeaurora.org; jchandra@broadcom.com; linaro-
> acpi@lists.linaro.org; linux-pci@vger.kernel.org; dhdang@apm.com;
> Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com;
> linux-kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> On 20 May 2016 at 10:01, Jon Masters <jcm@redhat.com> wrote:
> > Hi Ard,
> >
> > On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
> >> On 20 May 2016 at 06:41, Jon Masters <jcm@redhat.com> wrote:
> >>> Hi Tomasz, all,
> >>>
> >>> On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
> >>>
> >>>> On 11.05.2016 12:41, Gabriele Paoloni wrote:
> >>>
> >>>>>> v6 -> v7
> >>>>>> - drop quirks handling
> >>>>>
> >>>>> Maybe I missed something in the v6 discussion thread; when was it
> >>>>> decided to drop quirk handling?
> >>>>
> >>>> I had such requests in previous series.
> >>>
> >>> A quick note on quirk handling. This, I believe, applies post-merge
> of
> >>> the base infrastructure, which I realize will likely not have
> quirks.
> >>>
> >>> We've some "gen1" ARMv8 server platforms where we end up doing
> quirks
> >>> (for things like forcing 32-bit config space accessors and the
> like) due
> >>> to people repurposing existing embedded PCIe IP blocks or using
> them for
> >>> the first time (especially in servers), and those being involved in
> the
> >>> design not necessarily seeing this problem ahead of time, or not
> >>> realizing that it would be an issue for servers. In the early days
> of
> >>> ARM server designs 3-4 years ago, many of us had never really
> played
> >>> with ECAM or realized how modern topologies are built.
> >>>
> >>> Anyway. We missed this one in our SBSA requirements. They say
> (words to
> >>> the effect of) "thou shalt do PCIe the way it is done on servers"
> but
> >>> they aren't prescriptive, and they don't tell people how that
> actually
> >>> is in reality. That is being fixed. A lot of things are happening
> behind
> >>> the scenes - especially with third party IP block providers (all of
> whom
> >>> myself and others are speaking with directly about this) - to
> ensure
> >>> that the next wave of designs won't repeat these mistakes. We don't
> have
> >>> a time machine, but we can contain this from becoming an ongoing
> mess
> >>> for upstream, and we will do so. It won't be a zoo.
> >>>
> >>> Various proposals have arisen for how to handle quirks in the
> longer
> >>> term, including elaborate frameworks and tables to describe them
> >>> generically. I would like to caution against such approaches,
> especially
> >>> in the case that they deviate from practice on x86, or prior to
> being
> >>> standardized fully with other Operating System vendors. I don't
> expect
> >>> there to be too many more than the existing initial set of quirks
> we
> >>> have seen posted. A number of "future" server SoCs have already
> been
> >>> fixed prior to silicon, and new design starts are being warned not
> to
> >>> make this a problem for us to have to clean up later.
> >>>
> >>> So, I would like to suggest that the eventual framework mirror the
> >>> existing approach on x86 systems (matching DMI, etc.) and not be
> made
> >>> into some kind of generic, utopia. This is a case where we want
> there to
> >>> be pain involved (and upstream patches required) when people screw
> up,
> >>> so that they have a level of pain in response to ever making this
> >>> mistake in the future. If we try to create too grand a generic
> scheme
> >>> and make it too easy to handle this kind of situation beyond the
> small
> >>> number of existing offenders, we undermine efforts to force vendors
> to
> >>> ensure that their IP blocks are compliant going forward.
> >
> >> I understand that there is a desire from the RedHat side to mimic
> x86
> >> as closely as possible, but I never saw any technical justification
> >> for that.
> >
> > Understood. My own motivation is always to make the experience as
> > familiar as possible, both for end users, as well as for ODMs and the
> > entire ecosystem. There are very many ODMs currently working on v8
> > server designs and they're already expecting this to be "just like
> x88".
> > Intentionally. But as to the specifics of using DMI...
> >
> >> DMI contains strings that are visible to userland, and you
> >> effectively lock those down to certain values just so that the
> kernel
> >> can distinguish a broken PCIe root complex from a working one. Linux
> >> on x86 had no choice, since the overwhelming majority of existing
> >> hardware misrepresented itself as generic, and DMI was the only
> thing
> >> available to actually distinguish these broken implementations from
> >> one another. This does not mean we should allow and/or encourage
> this
> >> first gen hardware to misrepresent non-compliant hardware as
> compliant
> >> as well.
> >
> > That's a very reasonable argument. I don't disagree that it would be
> > nice to have nicer ways to distinguish the non-compliant IP than
> > treating the whole platform with an ASCII matching sledgehammer.
> >
> >> Since you are talking to all the people involved, how about you
> >> convince them to put something in the ACPI tables that allows the
> >> kernel to distinguish those non-standard PCIe implementations from
> >> hardware that is really generic?
> >
> > I'm open to this *BUT* it has to be something that will be adopted
> > beyond Linux. I have reached out to some non-Linux folks about this.
> If
> > there's buy-in, and if there's agreement to go standardize it through
> > the ASWG, then we should do so. What we should not do is treat ARM as
> > special in a way that the others aren't involved with. I'll admit DMI
> > ended up part of the SBBR in part because I wrote that piece in with
> the
> > assumption that exactly the same matches as on x86 would happen.
> >
> 
> Is the PCIe root complex so special that you cannot simply describe an
> implementation that is not PNP0408 compatible as something else, under
> its own unique HID? If everybody is onboard with using ACPI, how is
> this any different from describing other parts of the platform
> topology? Even if the SBSA mandates generic PCI, they already deviated
> from that when they built the hardware, so pretending that it is a
> PNP0408 with quirks really does not buy us anything.

From my understanding we want to avoid this as this would allow each
vendor to come up with his own code and it would be much more effort
for the PCI maintainer to rework the PCI framework to accommodate X86
and "all" ARM64 Host Controllers...

I guess this approach is too risky and we want to avoid this. Through
standardization we can more easily maintain the code and scale it to
multiple SoCs...

So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...

Thanks

Gab


> 
> >> This way, we can sidestep the quirks
> >> debate entirely, since it will simply be a different device as far
> as
> >> the kernel is concerned. This is no worse than a quirk from a
> >> practical point of view, since an older OS will be equally unable to
> >> run on newer hardware, but it is arguably more true to the standards
> >> compliance you tend to preach about, especially since this small
> pool
> >> of third party IP could potentially be identified directly rather
> than
> >> based on some divination of the SoC we may or may not be running on.
> I
> >> am also convinced that adding support for an additional HID() to the
> >> ACPI ECAM driver with some special config space handling wired in is
> >> an easier sell upstream than making the same ugly mess x86 has had
> to
> >> make because they did not have any choice to begin with.
> >
> > Again, open to it. I just don't want to do something that's Linux
> > specific. So it'll take time. It would be awesome if an interim quirk
> > solution existed that got platforms that are shipping in production
> > (e.g. HP Moonshot) actually booting upstream kernels this year. We
> > /really/ want F25 to be able to run on these without needing to carry
> an
> > out-of-tree quirk patch or just not support them. That's much worse.
> >
> 
> My whole point is that we don't need quirks in the first place if
> non-compliant devices are not being misrepresented as compliant ones.
> This is fine for other platform devices, i.e., SATA, network, so
> again, why is PCIe so special that we *must* use a generic ID + quirks
> rather than a specific ID?
> 
> >> If we do need a quirks handling mechanism, I still don't see how the
> >> x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
> >> SoC vendor to identify the make and particular revision, and quirks
> >> could be keyed off of that.
> >
> > Fair enough. Is there any traction for an interim solution for these
> > initial platforms do you think? If we wait to add a new table it's
> > probably going to be the end of the year before we get this done.
> >
> 
> The 'interim solution' is to come to terms with the fact that these
> initial platforms are not SBSA compliant, contain a PCIe root complex
> that is not PNP0408 but can be identified by its own HID, and we make
> the software work with that. This means our message from the beginning
> is that, yes, you can have non-compliant hardware and the burden is on
> you to get it supported upstream, and no, you don't get to hang out
> with the cool SBSA kids if you decide to go that route

[toc] | [prev] | [next] | [standalone]

#1404273 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

On 20 May 2016 at 10:40, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> Hi Ard
>
>> -----Original Message-----
>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
>>
>> Is the PCIe root complex so special that you cannot simply describe an
>> implementation that is not PNP0408 compatible as something else, under
>> its own unique HID? If everybody is onboard with using ACPI, how is
>> this any different from describing other parts of the platform
>> topology? Even if the SBSA mandates generic PCI, they already deviated
>> from that when they built the hardware, so pretending that it is a
>> PNP0408 with quirks really does not buy us anything.
>
> From my understanding we want to avoid this as this would allow each
> vendor to come up with his own code and it would be much more effort
> for the PCI maintainer to rework the PCI framework to accommodate X86
> and "all" ARM64 Host Controllers...
>
> I guess this approach is too risky and we want to avoid this. Through
> standardization we can more easily maintain the code and scale it to
> multiple SoCs...
>
> So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
> a bit more explanation...
>

OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.

We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess which
exact hardware they might be running on. For arm64, it is the opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires a
priori knowledge inside the OS that a certain 'generic' device must be
driven in a special way.

So can anyone comment on the feasibility of adding support for devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?

[toc] | [prev] | [next] | [standalone]

#1405241

On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
> On 20 May 2016 at 10:40, Gabriele Paoloni <gabriele.paoloni@huawei.com> wrote:
> > Hi Ard
> >
> >> -----Original Message-----
> >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> [...]
> >>
> >> Is the PCIe root complex so special that you cannot simply describe an
> >> implementation that is not PNP0408 compatible as something else, under
> >> its own unique HID? If everybody is onboard with using ACPI, how is
> >> this any different from describing other parts of the platform
> >> topology? Even if the SBSA mandates generic PCI, they already deviated
> >> from that when they built the hardware, so pretending that it is a
> >> PNP0408 with quirks really does not buy us anything.
> >
> > From my understanding we want to avoid this as this would allow each
> > vendor to come up with his own code and it would be much more effort
> > for the PCI maintainer to rework the PCI framework to accommodate X86
> > and "all" ARM64 Host Controllers...
> >
> > I guess this approach is too risky and we want to avoid this. Through
> > standardization we can more easily maintain the code and scale it to
> > multiple SoCs...
> >
> > So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
> > a bit more explanation...
> >
> 
> OK, so that boils down to recommending to vendors to represent known
> non-compliant hardware as compliant, just so that we don't have to
> change the code to support additional flavors of ECAM ? It's fine to
> be pragmatic, but that sucks.
> 
> We keep confusing the x86 case with the ARM case here: for x86, they
> needed to deal with broken hardware *after* the fact, and all they
> could do is find /some/ distinguishing feature in order to guess which
> exact hardware they might be running on. For arm64, it is the opposite
> case. We are currently in a position where we can demand vendors to
> comply with the standards they endorsed themselves, and (ab)using ACPI
> + DMI as a de facto platform description rather than plain ACPI makes
> me think the DT crowd were actually right from the beginning. It
> *directly* violates the standardization principle, since it requires a
> priori knowledge inside the OS that a certain 'generic' device must be
> driven in a special way.
> 
> So can anyone comment on the feasibility of adding support for devices
> with vendor specific HIDs (and no generic CIDs) to the current ACPI
> ECAM driver in Linux?

Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.

X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).

Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?

No.

So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.

The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.

Lorenzo

[toc] | [prev] | [next] | [standalone]

#1405432

Hi Lorenzo

> -----Original Message-----
> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> Sent: 23 May 2016 11:57
> To: Ard Biesheuvel
> Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
> > On 20 May 2016 at 10:40, Gabriele Paoloni
> <gabriele.paoloni@huawei.com> wrote:
> > > Hi Ard
> > >
> > >> -----Original Message-----
> > >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> > [...]
> > >>
> > >> Is the PCIe root complex so special that you cannot simply
> describe an
> > >> implementation that is not PNP0408 compatible as something else,
> under
> > >> its own unique HID? If everybody is onboard with using ACPI, how
> is
> > >> this any different from describing other parts of the platform
> > >> topology? Even if the SBSA mandates generic PCI, they already
> deviated
> > >> from that when they built the hardware, so pretending that it is a
> > >> PNP0408 with quirks really does not buy us anything.
> > >
> > > From my understanding we want to avoid this as this would allow
> each
> > > vendor to come up with his own code and it would be much more
> effort
> > > for the PCI maintainer to rework the PCI framework to accommodate
> X86
> > > and "all" ARM64 Host Controllers...
> > >
> > > I guess this approach is too risky and we want to avoid this.
> Through
> > > standardization we can more easily maintain the code and scale it
> to
> > > multiple SoCs...
> > >
> > > So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
> > > a bit more explanation...
> > >
> >
> > OK, so that boils down to recommending to vendors to represent known
> > non-compliant hardware as compliant, just so that we don't have to
> > change the code to support additional flavors of ECAM ? It's fine to
> > be pragmatic, but that sucks.
> >
> > We keep confusing the x86 case with the ARM case here: for x86, they
> > needed to deal with broken hardware *after* the fact, and all they
> > could do is find /some/ distinguishing feature in order to guess
> which
> > exact hardware they might be running on. For arm64, it is the
> opposite
> > case. We are currently in a position where we can demand vendors to
> > comply with the standards they endorsed themselves, and (ab)using
> ACPI
> > + DMI as a de facto platform description rather than plain ACPI makes
> > me think the DT crowd were actually right from the beginning. It
> > *directly* violates the standardization principle, since it requires
> a
> > priori knowledge inside the OS that a certain 'generic' device must
> be
> > driven in a special way.
> >
> > So can anyone comment on the feasibility of adding support for
> devices
> > with vendor specific HIDs (and no generic CIDs) to the current ACPI
> > ECAM driver in Linux?
> 
> Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
> most of the arch specific code is handled in the respective arch
> directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG
> for
> PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
> through
> ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
> sets-up resources AND config space on an arch specific basis.
> 
> X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
> on a platform specific basis (and it is not nice, but it works because
> as you all know the number of platforms in X86 world is contained).
> 
> Will this happen for ARM64 in arch/arm64 based on vendor specific
> HIDs ?
> 
> No.
> 
> So given the current state of play (we were requested to move the
> arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
> with arch/arm64 code requiring code in /drivers to set-up pci_ops
> in a platform specific way, it is horrible, if feasible at all.
> 
> The only way this can be implemented is by pretending that the
> ACPI/PCI arch/arm64 implementation is generic code (that's what this
> series does), move it to /drivers (where it is in this series), and
> implement _DSD vendor specific bindings (per HID) to set-up the pci
> operations; whether this solution should go upstream, given that it
> is just a short-term solution for early platforms bugs, it is another
> story and my personal answer is no.

I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.

i.e.:

extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];

static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
{
        int bus_num = root->secondary.start;
        int domain = root->segment;
        struct pci_cfg_fixup *f;

        /*
         * Match against platform specific quirks and return corresponding
         * CAM ops.
         *
         * First match against PCI topology <domain:bus> then use DMI or
         * custom match handler.
         */
        for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
                if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
                    (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
                    (f->system ? dmi_check_system(f->system) : 1) &&
                    (f->match ? f->match(f, root) : 1))
                        return f->ops;
        }
        /* No quirks, use ECAM */
        return &pci_generic_ecam_ops;
}

Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.

e.g. in HiSilicon case we would have

DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
			PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);

in "drivers/pci/host/pcie-hisi-acpi.c "

Thanks

Gab

> 
> Lorenzo

[toc] | [prev] | [next] | [standalone]

#1405721

On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> Hi Lorenzo
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > Sent: 23 May 2016 11:57
> > To: Ard Biesheuvel
> > Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org;
> > arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> > rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> > jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> > pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> > ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> > kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> > robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> > msalter@redhat.com; Wangyijing; mw@semihalf.com;
> > andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> > Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> > controller
> > 
> > On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
> > > On 20 May 2016 at 10:40, Gabriele Paoloni
> > <gabriele.paoloni@huawei.com> wrote:
> > > > Hi Ard
> > > >
> > > >> -----Original Message-----
> > > >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> > > [...]
> > > >>
> > > >> Is the PCIe root complex so special that you cannot simply
> > describe an
> > > >> implementation that is not PNP0408 compatible as something else,
> > under
> > > >> its own unique HID? If everybody is onboard with using ACPI, how
> > is
> > > >> this any different from describing other parts of the platform
> > > >> topology? Even if the SBSA mandates generic PCI, they already
> > deviated
> > > >> from that when they built the hardware, so pretending that it is a
> > > >> PNP0408 with quirks really does not buy us anything.
> > > >
> > > > From my understanding we want to avoid this as this would allow
> > each
> > > > vendor to come up with his own code and it would be much more
> > effort
> > > > for the PCI maintainer to rework the PCI framework to accommodate
> > X86
> > > > and "all" ARM64 Host Controllers...
> > > >
> > > > I guess this approach is too risky and we want to avoid this.
> > Through
> > > > standardization we can more easily maintain the code and scale it
> > to
> > > > multiple SoCs...
> > > >
> > > > So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
> > > > a bit more explanation...
> > > >
> > >
> > > OK, so that boils down to recommending to vendors to represent known
> > > non-compliant hardware as compliant, just so that we don't have to
> > > change the code to support additional flavors of ECAM ? It's fine to
> > > be pragmatic, but that sucks.
> > >
> > > We keep confusing the x86 case with the ARM case here: for x86, they
> > > needed to deal with broken hardware *after* the fact, and all they
> > > could do is find /some/ distinguishing feature in order to guess
> > which
> > > exact hardware they might be running on. For arm64, it is the
> > opposite
> > > case. We are currently in a position where we can demand vendors to
> > > comply with the standards they endorsed themselves, and (ab)using
> > ACPI
> > > + DMI as a de facto platform description rather than plain ACPI makes
> > > me think the DT crowd were actually right from the beginning. It
> > > *directly* violates the standardization principle, since it requires
> > a
> > > priori knowledge inside the OS that a certain 'generic' device must
> > be
> > > driven in a special way.
> > >
> > > So can anyone comment on the feasibility of adding support for
> > devices
> > > with vendor specific HIDs (and no generic CIDs) to the current ACPI
> > > ECAM driver in Linux?

I don't think of ECAM support itself as a "driver".  It's just a
service available to drivers, similar to OF resource parsing.

Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space.  It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.

"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
r3.2, sec 4.1) means:

  (a) a memory-mapped model for config space access, and
  (b) a specific mapping of address bits to bus/device/function/
      register

MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.

If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.

Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways.  Is there a mechanism in use already?  How
does Windows handle this?  DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.

A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.

There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case.  I don't know how important
that is.  Defining an MCFG extension seems like the most obvious
solution.

If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.

> > Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
> > most of the arch specific code is handled in the respective arch
> > directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG
> > for
> > PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
> > through
> > ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
> > sets-up resources AND config space on an arch specific basis.
> > 
> > X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
> > on a platform specific basis (and it is not nice, but it works because
> > as you all know the number of platforms in X86 world is contained).
> > 
> > Will this happen for ARM64 in arch/arm64 based on vendor specific
> > HIDs ?
> > 
> > No.
> > 
> > So given the current state of play (we were requested to move the
> > arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
> > with arch/arm64 code requiring code in /drivers to set-up pci_ops
> > in a platform specific way, it is horrible, if feasible at all.
> > 
> > The only way this can be implemented is by pretending that the
> > ACPI/PCI arch/arm64 implementation is generic code (that's what this
> > series does), move it to /drivers (where it is in this series), and
> > implement _DSD vendor specific bindings (per HID) to set-up the pci
> > operations; whether this solution should go upstream, given that it
> > is just a short-term solution for early platforms bugs, it is another
> > story and my personal answer is no.

It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.

Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code.  I think it would be nicer if the
lookup and the use were together.  Then it would be easier to override
it because the mapping assumptions would all be in one place.

> I think it shouldn't be too bad to move quirk handling mechanism to
> arch/arm64. Effectively we would not move platform specific code into
> arch/arm64 but just the mechanism checking if there is any quirk that
> is defined.
> 
> i.e.:
> 
> extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
> extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
> 
> static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
> {
>         int bus_num = root->secondary.start;
>         int domain = root->segment;
>         struct pci_cfg_fixup *f;
> 
>         /*
>          * Match against platform specific quirks and return corresponding
>          * CAM ops.
>          *
>          * First match against PCI topology <domain:bus> then use DMI or
>          * custom match handler.
>          */
>         for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
>                 if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
>                     (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
>                     (f->system ? dmi_check_system(f->system) : 1) &&
>                     (f->match ? f->match(f, root) : 1))
>                         return f->ops;
>         }
>         /* No quirks, use ECAM */
>         return &pci_generic_ecam_ops;
> }
> 
> Such quirks will be defined anyway in drivers/pci/host/ in the vendor
> specific quirk implementations.
> 
> e.g. in HiSilicon case we would have
> 
> DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
> 			PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
> 
> in "drivers/pci/host/pcie-hisi-acpi.c "
> 
> Thanks
> 
> Gab

Sorry Gab, I guess I was really responding to earlier messages :)

I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
I suppose there's already firmware in the field that does that,
though?

Bjorn

[toc] | [prev] | [next] | [standalone]

#1405755 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

Bjorn,

Out walking so sorry about top posting. Quick reply though:

1. I checked with the Windows team. They usually avoid quirks entirely but when it has happened, it has been done via the MCFG/FADT not DSDT.

2. They would be ok if we were to key off the OEM  name and revision for the IP in the MCFG table.

3. I have already verified existing shipping ARMv8 systems provide enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.

4. I have requested (and Linaro are investigating) that Linaro (with ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.

5. I will drive a clarification to the SBBR that does not encourage or endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.

More later.

Jon.

-- 
Computer Architect | Sent from my 64-bit #ARM Powered phone

> On May 23, 2016, at 19:39, Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
>> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
>> Hi Lorenzo
>> 
>>> -----Original Message-----
>>> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
>>> Sent: 23 May 2016 11:57
>>> To: Ard Biesheuvel
>>> Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org;
>>> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
>>> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
>>> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
>>> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
>>> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
>>> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
>>> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
>>> msalter@redhat.com; Wangyijing; mw@semihalf.com;
>>> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
>>> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
>>> controller
>>> 
>>>> On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
>>>> On 20 May 2016 at 10:40, Gabriele Paoloni
>>> <gabriele.paoloni@huawei.com> wrote:
>>>>> Hi Ard
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
>>>> [...]
>>>>>> 
>>>>>> Is the PCIe root complex so special that you cannot simply
>>> describe an
>>>>>> implementation that is not PNP0408 compatible as something else,
>>> under
>>>>>> its own unique HID? If everybody is onboard with using ACPI, how
>>> is
>>>>>> this any different from describing other parts of the platform
>>>>>> topology? Even if the SBSA mandates generic PCI, they already
>>> deviated
>>>>>> from that when they built the hardware, so pretending that it is a
>>>>>> PNP0408 with quirks really does not buy us anything.
>>>>> 
>>>>> From my understanding we want to avoid this as this would allow
>>> each
>>>>> vendor to come up with his own code and it would be much more
>>> effort
>>>>> for the PCI maintainer to rework the PCI framework to accommodate
>>> X86
>>>>> and "all" ARM64 Host Controllers...
>>>>> 
>>>>> I guess this approach is too risky and we want to avoid this.
>>> Through
>>>>> standardization we can more easily maintain the code and scale it
>>> to
>>>>> multiple SoCs...
>>>>> 
>>>>> So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
>>>>> a bit more explanation...
>>>> 
>>>> OK, so that boils down to recommending to vendors to represent known
>>>> non-compliant hardware as compliant, just so that we don't have to
>>>> change the code to support additional flavors of ECAM ? It's fine to
>>>> be pragmatic, but that sucks.
>>>> 
>>>> We keep confusing the x86 case with the ARM case here: for x86, they
>>>> needed to deal with broken hardware *after* the fact, and all they
>>>> could do is find /some/ distinguishing feature in order to guess
>>> which
>>>> exact hardware they might be running on. For arm64, it is the
>>> opposite
>>>> case. We are currently in a position where we can demand vendors to
>>>> comply with the standards they endorsed themselves, and (ab)using
>>> ACPI
>>>> + DMI as a de facto platform description rather than plain ACPI makes
>>>> me think the DT crowd were actually right from the beginning. It
>>>> *directly* violates the standardization principle, since it requires
>>> a
>>>> priori knowledge inside the OS that a certain 'generic' device must
>>> be
>>>> driven in a special way.
>>>> 
>>>> So can anyone comment on the feasibility of adding support for
>>> devices
>>>> with vendor specific HIDs (and no generic CIDs) to the current ACPI
>>>> ECAM driver in Linux?
> 
> I don't think of ECAM support itself as a "driver".  It's just a
> service available to drivers, similar to OF resource parsing.
> 
> Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> supports extended config space.  It doesn't specify how we access that
> config space, so I think hardware with non-standard ECAM should still
> have PNP0A03 and PNP0A08 in _CID or _HID.
> 
> "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> r3.2, sec 4.1) means:
> 
>  (a) a memory-mapped model for config space access, and
>  (b) a specific mapping of address bits to bus/device/function/
>      register
> 
> MCFG and _CBA assume both (a) and (b), so I think a device with
> non-standard ECAM mappings should not be described in MCFG or _CBA.
> 
> If a bridge has ECAM with non-standard mappings, I think either a
> vendor-specific _HID or a device-specific method, e.g., _DSM, could
> communicate that.
> 
> Jon, I agree that we should avoid describing non-standardized hardware
> in Linux-specific ways.  Is there a mechanism in use already?  How
> does Windows handle this?  DMI is a poor long-term solution because it
> requires ongoing maintenance for new platforms, but I think it's OK
> for getting started with platforms already shipping.
> 
> A _DSM has the advantage that once it is defined and supported, OEMs
> can ship new platforms without requiring a new quirk or a new _HID to
> be added to a driver.
> 
> There would still be the problem of config access before the namespace
> is available, i.e., the MCFG use case.  I don't know how important
> that is.  Defining an MCFG extension seems like the most obvious
> solution.
> 
> If we only expect a few non-standard devices, maybe it's enough to
> have DMI quirks to statically set up ECAM and just live with the
> inconvenience of requiring a kernel change for every new non-standard
> device.
> 
>>> Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
>>> most of the arch specific code is handled in the respective arch
>>> directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG
>>> for
>>> PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
>>> through
>>> ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
>>> sets-up resources AND config space on an arch specific basis.
>>> 
>>> X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
>>> on a platform specific basis (and it is not nice, but it works because
>>> as you all know the number of platforms in X86 world is contained).
>>> 
>>> Will this happen for ARM64 in arch/arm64 based on vendor specific
>>> HIDs ?
>>> 
>>> No.
>>> 
>>> So given the current state of play (we were requested to move the
>>> arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
>>> with arch/arm64 code requiring code in /drivers to set-up pci_ops
>>> in a platform specific way, it is horrible, if feasible at all.
>>> 
>>> The only way this can be implemented is by pretending that the
>>> ACPI/PCI arch/arm64 implementation is generic code (that's what this
>>> series does), move it to /drivers (where it is in this series), and
>>> implement _DSD vendor specific bindings (per HID) to set-up the pci
>>> operations; whether this solution should go upstream, given that it
>>> is just a short-term solution for early platforms bugs, it is another
>>> story and my personal answer is no.
> 
> It seems like there should be a way to look for a _DSM before we call
> acpi_pci_root_get_mcfg_addr() to look for _CBA.
> 
> Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
> the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
> only used in x86-specific code.  I think it would be nicer if the
> lookup and the use were together.  Then it would be easier to override
> it because the mapping assumptions would all be in one place.
> 
>> I think it shouldn't be too bad to move quirk handling mechanism to
>> arch/arm64. Effectively we would not move platform specific code into
>> arch/arm64 but just the mechanism checking if there is any quirk that
>> is defined.
>> 
>> i.e.:
>> 
>> extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
>> extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
>> 
>> static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
>> {
>>        int bus_num = root->secondary.start;
>>        int domain = root->segment;
>>        struct pci_cfg_fixup *f;
>> 
>>        /*
>>         * Match against platform specific quirks and return corresponding
>>         * CAM ops.
>>         *
>>         * First match against PCI topology <domain:bus> then use DMI or
>>         * custom match handler.
>>         */
>>        for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
>>                if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
>>                    (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
>>                    (f->system ? dmi_check_system(f->system) : 1) &&
>>                    (f->match ? f->match(f, root) : 1))
>>                        return f->ops;
>>        }
>>        /* No quirks, use ECAM */
>>        return &pci_generic_ecam_ops;
>> }
>> 
>> Such quirks will be defined anyway in drivers/pci/host/ in the vendor
>> specific quirk implementations.
>> 
>> e.g. in HiSilicon case we would have
>> 
>> DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
>>            PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
>> 
>> in "drivers/pci/host/pcie-hisi-acpi.c "
>> 
>> Thanks
>> 
>> Gab
> 
> Sorry Gab, I guess I was really responding to earlier messages :)
> 
> I don't really have much to say here, except that it doesn't seem
> right to have an MCFG that describes a non-standard ECAM mapping.
> I suppose there's already firmware in the field that does that,
> though?
> 
> Bjorn

[toc] | [prev] | [next] | [standalone]

#1405769 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

Additional: I would like to thank Ard for suggesting this approach. It turns out (apparently) that Mark Salter's initial X-Gene quirks internal to RH did it this way as well. You great minds think alike. If this works for folks then I hope it leads to upstream kernel support in F25 (we have a bunch of Moonshot hardware we would like to deeply in Fedora but can't without the PCIe network...). I rant only because I care :)

Jon.

-- 
Computer Architect | Sent from my 64-bit #ARM Powered phone

> On May 23, 2016, at 21:11, Jon Masters <jcm@redhat.com> wrote:
> 
> Bjorn,
> 
> Out walking so sorry about top posting. Quick reply though:
> 
> 1. I checked with the Windows team. They usually avoid quirks entirely but when it has happened, it has been done via the MCFG/FADT not DSDT.
> 
> 2. They would be ok if we were to key off the OEM  name and revision for the IP in the MCFG table.
> 
> 3. I have already verified existing shipping ARMv8 systems provide enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.
> 
> 4. I have requested (and Linaro are investigating) that Linaro (with ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.
> 
> 5. I will drive a clarification to the SBBR that does not encourage or endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.
> 
> More later.
> 
> Jon.
> 
> -- 
> Computer Architect | Sent from my 64-bit #ARM Powered phone
> 
>>> On May 23, 2016, at 19:39, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>> 
>>> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
>>> Hi Lorenzo
>>> 
>>>> -----Original Message-----
>>>> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
>>>> Sent: 23 May 2016 11:57
>>>> To: Ard Biesheuvel
>>>> Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org;
>>>> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
>>>> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
>>>> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
>>>> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
>>>> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
>>>> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
>>>> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
>>>> msalter@redhat.com; Wangyijing; mw@semihalf.com;
>>>> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
>>>> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
>>>> controller
>>>> 
>>>>> On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
>>>>> On 20 May 2016 at 10:40, Gabriele Paoloni
>>>> <gabriele.paoloni@huawei.com> wrote:
>>>>>> Hi Ard
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
>>>>> [...]
>>>>>>> 
>>>>>>> Is the PCIe root complex so special that you cannot simply
>>>> describe an
>>>>>>> implementation that is not PNP0408 compatible as something else,
>>>> under
>>>>>>> its own unique HID? If everybody is onboard with using ACPI, how
>>>> is
>>>>>>> this any different from describing other parts of the platform
>>>>>>> topology? Even if the SBSA mandates generic PCI, they already
>>>> deviated
>>>>>>> from that when they built the hardware, so pretending that it is a
>>>>>>> PNP0408 with quirks really does not buy us anything.
>>>>>> 
>>>>>> From my understanding we want to avoid this as this would allow
>>>> each
>>>>>> vendor to come up with his own code and it would be much more
>>>> effort
>>>>>> for the PCI maintainer to rework the PCI framework to accommodate
>>>> X86
>>>>>> and "all" ARM64 Host Controllers...
>>>>>> 
>>>>>> I guess this approach is too risky and we want to avoid this.
>>>> Through
>>>>>> standardization we can more easily maintain the code and scale it
>>>> to
>>>>>> multiple SoCs...
>>>>>> 
>>>>>> So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
>>>>>> a bit more explanation...
>>>>> 
>>>>> OK, so that boils down to recommending to vendors to represent known
>>>>> non-compliant hardware as compliant, just so that we don't have to
>>>>> change the code to support additional flavors of ECAM ? It's fine to
>>>>> be pragmatic, but that sucks.
>>>>> 
>>>>> We keep confusing the x86 case with the ARM case here: for x86, they
>>>>> needed to deal with broken hardware *after* the fact, and all they
>>>>> could do is find /some/ distinguishing feature in order to guess
>>>> which
>>>>> exact hardware they might be running on. For arm64, it is the
>>>> opposite
>>>>> case. We are currently in a position where we can demand vendors to
>>>>> comply with the standards they endorsed themselves, and (ab)using
>>>> ACPI
>>>>> + DMI as a de facto platform description rather than plain ACPI makes
>>>>> me think the DT crowd were actually right from the beginning. It
>>>>> *directly* violates the standardization principle, since it requires
>>>> a
>>>>> priori knowledge inside the OS that a certain 'generic' device must
>>>> be
>>>>> driven in a special way.
>>>>> 
>>>>> So can anyone comment on the feasibility of adding support for
>>>> devices
>>>>> with vendor specific HIDs (and no generic CIDs) to the current ACPI
>>>>> ECAM driver in Linux?
>> 
>> I don't think of ECAM support itself as a "driver".  It's just a
>> service available to drivers, similar to OF resource parsing.
>> 
>> Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
>> host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
>> supports extended config space.  It doesn't specify how we access that
>> config space, so I think hardware with non-standard ECAM should still
>> have PNP0A03 and PNP0A08 in _CID or _HID.
>> 
>> "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
>> r3.2, sec 4.1) means:
>> 
>> (a) a memory-mapped model for config space access, and
>> (b) a specific mapping of address bits to bus/device/function/
>>     register
>> 
>> MCFG and _CBA assume both (a) and (b), so I think a device with
>> non-standard ECAM mappings should not be described in MCFG or _CBA.
>> 
>> If a bridge has ECAM with non-standard mappings, I think either a
>> vendor-specific _HID or a device-specific method, e.g., _DSM, could
>> communicate that.
>> 
>> Jon, I agree that we should avoid describing non-standardized hardware
>> in Linux-specific ways.  Is there a mechanism in use already?  How
>> does Windows handle this?  DMI is a poor long-term solution because it
>> requires ongoing maintenance for new platforms, but I think it's OK
>> for getting started with platforms already shipping.
>> 
>> A _DSM has the advantage that once it is defined and supported, OEMs
>> can ship new platforms without requiring a new quirk or a new _HID to
>> be added to a driver.
>> 
>> There would still be the problem of config access before the namespace
>> is available, i.e., the MCFG use case.  I don't know how important
>> that is.  Defining an MCFG extension seems like the most obvious
>> solution.
>> 
>> If we only expect a few non-standard devices, maybe it's enough to
>> have DMI quirks to statically set up ECAM and just live with the
>> inconvenience of requiring a kernel change for every new non-standard
>> device.
>> 
>>>> Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
>>>> most of the arch specific code is handled in the respective arch
>>>> directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG
>>>> for
>>>> PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
>>>> through
>>>> ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
>>>> sets-up resources AND config space on an arch specific basis.
>>>> 
>>>> X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
>>>> on a platform specific basis (and it is not nice, but it works because
>>>> as you all know the number of platforms in X86 world is contained).
>>>> 
>>>> Will this happen for ARM64 in arch/arm64 based on vendor specific
>>>> HIDs ?
>>>> 
>>>> No.
>>>> 
>>>> So given the current state of play (we were requested to move the
>>>> arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
>>>> with arch/arm64 code requiring code in /drivers to set-up pci_ops
>>>> in a platform specific way, it is horrible, if feasible at all.
>>>> 
>>>> The only way this can be implemented is by pretending that the
>>>> ACPI/PCI arch/arm64 implementation is generic code (that's what this
>>>> series does), move it to /drivers (where it is in this series), and
>>>> implement _DSD vendor specific bindings (per HID) to set-up the pci
>>>> operations; whether this solution should go upstream, given that it
>>>> is just a short-term solution for early platforms bugs, it is another
>>>> story and my personal answer is no.
>> 
>> It seems like there should be a way to look for a _DSM before we call
>> acpi_pci_root_get_mcfg_addr() to look for _CBA.
>> 
>> Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
>> the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
>> only used in x86-specific code.  I think it would be nicer if the
>> lookup and the use were together.  Then it would be easier to override
>> it because the mapping assumptions would all be in one place.
>> 
>>> I think it shouldn't be too bad to move quirk handling mechanism to
>>> arch/arm64. Effectively we would not move platform specific code into
>>> arch/arm64 but just the mechanism checking if there is any quirk that
>>> is defined.
>>> 
>>> i.e.:
>>> 
>>> extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
>>> extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
>>> 
>>> static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
>>> {
>>>       int bus_num = root->secondary.start;
>>>       int domain = root->segment;
>>>       struct pci_cfg_fixup *f;
>>> 
>>>       /*
>>>        * Match against platform specific quirks and return corresponding
>>>        * CAM ops.
>>>        *
>>>        * First match against PCI topology <domain:bus> then use DMI or
>>>        * custom match handler.
>>>        */
>>>       for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
>>>               if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
>>>                   (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
>>>                   (f->system ? dmi_check_system(f->system) : 1) &&
>>>                   (f->match ? f->match(f, root) : 1))
>>>                       return f->ops;
>>>       }
>>>       /* No quirks, use ECAM */
>>>       return &pci_generic_ecam_ops;
>>> }
>>> 
>>> Such quirks will be defined anyway in drivers/pci/host/ in the vendor
>>> specific quirk implementations.
>>> 
>>> e.g. in HiSilicon case we would have
>>> 
>>> DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
>>>           PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
>>> 
>>> in "drivers/pci/host/pcie-hisi-acpi.c "
>>> 
>>> Thanks
>>> 
>>> Gab
>> 
>> Sorry Gab, I guess I was really responding to earlier messages :)
>> 
>> I don't really have much to say here, except that it doesn't seem
>> right to have an MCFG that describes a non-standard ECAM mapping.
>> I suppose there's already firmware in the field that does that,
>> though?
>> 
>> Bjorn

[toc] | [prev] | [next] | [standalone]

#1406217

Hi Jon

> -----Original Message-----
> From: Jon Masters [mailto:jcm@redhat.com]
> Sent: 24 May 2016 02:11
> To: Bjorn Helgaas
> Cc: Gabriele Paoloni; Lorenzo Pieralisi; Ard Biesheuvel; Tomasz
> Nowicki; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> Bjorn,
> 
> Out walking so sorry about top posting. Quick reply though:
> 
> 1. I checked with the Windows team. They usually avoid quirks entirely
> but when it has happened, it has been done via the MCFG/FADT not DSDT.
> 
> 2. They would be ok if we were to key off the OEM  name and revision
> for the IP in the MCFG table.

I see some problems with this approach:

1) We would need to modify the ACPI specs to accommodate quirks in the MCFG,
   correct?

2) Just adding OEM info would not fit some other people (like us for Designware
   based solutions). In our case for example the addresses defined in the MCFG
   are not compatible with the ones used by the Designware IP, therefore we would
   also need specific quirk data

I think that we can use an approach where we use MCFG entries for the ECAM
address spaces and motherboard reserved resources for those BUSes that are
outside the MCFG table and therefore are non ECAM.

I think it is a more generic approach that would suit anybody and there
is no need to redefine the ACPI specs for MCFG... ?

Thanks

Gab

> 
> 3. I have already verified existing shipping ARMv8 systems provide
> enough unique data in that entry, and have asked that vendors guarantee
> to rev it in future IP (which I will verify on models pre tapeout and
> certainly in early firmware builds). One vendor has a platform that
> isn't public yet that uses a non-public name in the MCFG but I spoke
> with them on Friday and they will shortly update their firmware so that
> a quirk could be posted.
> 
> 4. I have requested (and Linaro are investigating) that Linaro (with
> ARM's assistance) begin to drive a separate thread around upstreaming
> (independent of this core effort) quirks that use the OEM fields in the
> MCFG as a more scalable approach than one per platform via DMI.
> 
> 5. I will drive a clarification to the SBBR that does not encourage or
> endorse quirks but does merely reinforce that data must be unique in
> such tables. I am driving a separate series of conversations with
> vendors to ensure that this is the case on all future platforms -
> though just generally, there is no more high end top shelf "Xeon class"
> silicon needing common quirks in the pipeline.
> 
> More later.
> 
> Jon.
> 
> --
> Computer Architect | Sent from my 64-bit #ARM Powered phone
> 
> > On May 23, 2016, at 19:39, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> >> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> >> Hi Lorenzo
> >>
> >>> -----Original Message-----
> >>> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> >>> Sent: 23 May 2016 11:57
> >>> To: Ard Biesheuvel
> >>> Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
> helgaas@kernel.org;
> >>> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> >>> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> >>> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> >>> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> >>> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> >>> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> >>> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> >>> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> >>> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> >>> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
> host
> >>> controller
> >>>
> >>>> On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
> >>>> On 20 May 2016 at 10:40, Gabriele Paoloni
> >>> <gabriele.paoloni@huawei.com> wrote:
> >>>>> Hi Ard
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> >>>> [...]
> >>>>>>
> >>>>>> Is the PCIe root complex so special that you cannot simply
> >>> describe an
> >>>>>> implementation that is not PNP0408 compatible as something else,
> >>> under
> >>>>>> its own unique HID? If everybody is onboard with using ACPI, how
> >>> is
> >>>>>> this any different from describing other parts of the platform
> >>>>>> topology? Even if the SBSA mandates generic PCI, they already
> >>> deviated
> >>>>>> from that when they built the hardware, so pretending that it is
> a
> >>>>>> PNP0408 with quirks really does not buy us anything.
> >>>>>
> >>>>> From my understanding we want to avoid this as this would allow
> >>> each
> >>>>> vendor to come up with his own code and it would be much more
> >>> effort
> >>>>> for the PCI maintainer to rework the PCI framework to accommodate
> >>> X86
> >>>>> and "all" ARM64 Host Controllers...
> >>>>>
> >>>>> I guess this approach is too risky and we want to avoid this.
> >>> Through
> >>>>> standardization we can more easily maintain the code and scale it
> >>> to
> >>>>> multiple SoCs...
> >>>>>
> >>>>> So this is my understanding; maybe Jon, Tomasz or Lorenzo can
> give
> >>>>> a bit more explanation...
> >>>>
> >>>> OK, so that boils down to recommending to vendors to represent
> known
> >>>> non-compliant hardware as compliant, just so that we don't have to
> >>>> change the code to support additional flavors of ECAM ? It's fine
> to
> >>>> be pragmatic, but that sucks.
> >>>>
> >>>> We keep confusing the x86 case with the ARM case here: for x86,
> they
> >>>> needed to deal with broken hardware *after* the fact, and all they
> >>>> could do is find /some/ distinguishing feature in order to guess
> >>> which
> >>>> exact hardware they might be running on. For arm64, it is the
> >>> opposite
> >>>> case. We are currently in a position where we can demand vendors
> to
> >>>> comply with the standards they endorsed themselves, and (ab)using
> >>> ACPI
> >>>> + DMI as a de facto platform description rather than plain ACPI
> makes
> >>>> me think the DT crowd were actually right from the beginning. It
> >>>> *directly* violates the standardization principle, since it
> requires
> >>> a
> >>>> priori knowledge inside the OS that a certain 'generic' device
> must
> >>> be
> >>>> driven in a special way.
> >>>>
> >>>> So can anyone comment on the feasibility of adding support for
> >>> devices
> >>>> with vendor specific HIDs (and no generic CIDs) to the current
> ACPI
> >>>> ECAM driver in Linux?
> >
> > I don't think of ECAM support itself as a "driver".  It's just a
> > service available to drivers, similar to OF resource parsing.
> >
> > Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> > host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> > supports extended config space.  It doesn't specify how we access
> that
> > config space, so I think hardware with non-standard ECAM should still
> > have PNP0A03 and PNP0A08 in _CID or _HID.
> >
> > "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> > r3.2, sec 4.1) means:
> >
> >  (a) a memory-mapped model for config space access, and
> >  (b) a specific mapping of address bits to bus/device/function/
> >      register
> >
> > MCFG and _CBA assume both (a) and (b), so I think a device with
> > non-standard ECAM mappings should not be described in MCFG or _CBA.
> >
> > If a bridge has ECAM with non-standard mappings, I think either a
> > vendor-specific _HID or a device-specific method, e.g., _DSM, could
> > communicate that.
> >
> > Jon, I agree that we should avoid describing non-standardized
> hardware
> > in Linux-specific ways.  Is there a mechanism in use already?  How
> > does Windows handle this?  DMI is a poor long-term solution because
> it
> > requires ongoing maintenance for new platforms, but I think it's OK
> > for getting started with platforms already shipping.
> >
> > A _DSM has the advantage that once it is defined and supported, OEMs
> > can ship new platforms without requiring a new quirk or a new _HID to
> > be added to a driver.
> >
> > There would still be the problem of config access before the
> namespace
> > is available, i.e., the MCFG use case.  I don't know how important
> > that is.  Defining an MCFG extension seems like the most obvious
> > solution.
> >
> > If we only expect a few non-standard devices, maybe it's enough to
> > have DMI quirks to statically set up ECAM and just live with the
> > inconvenience of requiring a kernel change for every new non-standard
> > device.
> >
> >>> Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
> >>> most of the arch specific code is handled in the respective arch
> >>> directories (X86 and IA64, even though IA64 does not rely on
> ECAM/MCFG
> >>> for
> >>> PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
> >>> through
> >>> ACPI scan handlers and the respective arch code (ie
> pci_acpi_scan_root)
> >>> sets-up resources AND config space on an arch specific basis.
> >>>
> >>> X86 deals with that with code in arch/x86 that sets-up the
> pci_raw_ops
> >>> on a platform specific basis (and it is not nice, but it works
> because
> >>> as you all know the number of platforms in X86 world is contained).
> >>>
> >>> Will this happen for ARM64 in arch/arm64 based on vendor specific
> >>> HIDs ?
> >>>
> >>> No.
> >>>
> >>> So given the current state of play (we were requested to move the
> >>> arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
> >>> with arch/arm64 code requiring code in /drivers to set-up pci_ops
> >>> in a platform specific way, it is horrible, if feasible at all.
> >>>
> >>> The only way this can be implemented is by pretending that the
> >>> ACPI/PCI arch/arm64 implementation is generic code (that's what
> this
> >>> series does), move it to /drivers (where it is in this series), and
> >>> implement _DSD vendor specific bindings (per HID) to set-up the pci
> >>> operations; whether this solution should go upstream, given that it
> >>> is just a short-term solution for early platforms bugs, it is
> another
> >>> story and my personal answer is no.
> >
> > It seems like there should be a way to look for a _DSM before we call
> > acpi_pci_root_get_mcfg_addr() to look for _CBA.
> >
> > Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
> > the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
> > only used in x86-specific code.  I think it would be nicer if the
> > lookup and the use were together.  Then it would be easier to
> override
> > it because the mapping assumptions would all be in one place.
> >
> >> I think it shouldn't be too bad to move quirk handling mechanism to
> >> arch/arm64. Effectively we would not move platform specific code
> into
> >> arch/arm64 but just the mechanism checking if there is any quirk
> that
> >> is defined.
> >>
> >> i.e.:
> >>
> >> extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
> >> extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
> >>
> >> static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
> *root)
> >> {
> >>        int bus_num = root->secondary.start;
> >>        int domain = root->segment;
> >>        struct pci_cfg_fixup *f;
> >>
> >>        /*
> >>         * Match against platform specific quirks and return
> corresponding
> >>         * CAM ops.
> >>         *
> >>         * First match against PCI topology <domain:bus> then use DMI
> or
> >>         * custom match handler.
> >>         */
> >>        for (f = __start_acpi_mcfg_fixups; f <
> __end_acpi_mcfg_fixups; f++) {
> >>                if ((f->domain == domain || f->domain ==
> PCI_MCFG_DOMAIN_ANY) &&
> >>                    (f->bus_num == bus_num || f->bus_num ==
> PCI_MCFG_BUS_ANY) &&
> >>                    (f->system ? dmi_check_system(f->system) : 1) &&
> >>                    (f->match ? f->match(f, root) : 1))
> >>                        return f->ops;
> >>        }
> >>        /* No quirks, use ECAM */
> >>        return &pci_generic_ecam_ops;
> >> }
> >>
> >> Such quirks will be defined anyway in drivers/pci/host/ in the
> vendor
> >> specific quirk implementations.
> >>
> >> e.g. in HiSilicon case we would have
> >>
> >> DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
> >>            PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
> >>
> >> in "drivers/pci/host/pcie-hisi-acpi.c "
> >>
> >> Thanks
> >>
> >> Gab
> >
> > Sorry Gab, I guess I was really responding to earlier messages :)
> >
> > I don't really have much to say here, except that it doesn't seem
> > right to have an MCFG that describes a non-standard ECAM mapping.
> > I suppose there's already firmware in the field that does that,
> > though?
> >
> > Bjorn

[toc] | [prev] | [next] | [standalone]

#1405911

Hi Bjorn

> -----Original Message-----
> From: Bjorn Helgaas [mailto:helgaas@kernel.org]
> Sent: 24 May 2016 00:39
> To: Gabriele Paoloni
> Cc: Lorenzo Pieralisi; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> > Hi Lorenzo
> >
> > > -----Original Message-----
> > > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > > Sent: 23 May 2016 11:57
> > > To: Ard Biesheuvel
> > > Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
> helgaas@kernel.org;
> > > arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> > > rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> > > jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> > > pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> > > ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> > > kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> > > robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> > > msalter@redhat.com; Wangyijing; mw@semihalf.com;
> > > andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> > > Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
> host
> > > controller
> > >
> > > On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
> > > > On 20 May 2016 at 10:40, Gabriele Paoloni
> > > <gabriele.paoloni@huawei.com> wrote:
> > > > > Hi Ard
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
> > > > [...]
> > > > >>
> > > > >> Is the PCIe root complex so special that you cannot simply
> > > describe an
> > > > >> implementation that is not PNP0408 compatible as something
> else,
> > > under
> > > > >> its own unique HID? If everybody is onboard with using ACPI,
> how
> > > is
> > > > >> this any different from describing other parts of the platform
> > > > >> topology? Even if the SBSA mandates generic PCI, they already
> > > deviated
> > > > >> from that when they built the hardware, so pretending that it
> is a
> > > > >> PNP0408 with quirks really does not buy us anything.
> > > > >
> > > > > From my understanding we want to avoid this as this would allow
> > > each
> > > > > vendor to come up with his own code and it would be much more
> > > effort
> > > > > for the PCI maintainer to rework the PCI framework to
> accommodate
> > > X86
> > > > > and "all" ARM64 Host Controllers...
> > > > >
> > > > > I guess this approach is too risky and we want to avoid this.
> > > Through
> > > > > standardization we can more easily maintain the code and scale
> it
> > > to
> > > > > multiple SoCs...
> > > > >
> > > > > So this is my understanding; maybe Jon, Tomasz or Lorenzo can
> give
> > > > > a bit more explanation...
> > > > >
> > > >
> > > > OK, so that boils down to recommending to vendors to represent
> known
> > > > non-compliant hardware as compliant, just so that we don't have
> to
> > > > change the code to support additional flavors of ECAM ? It's fine
> to
> > > > be pragmatic, but that sucks.
> > > >
> > > > We keep confusing the x86 case with the ARM case here: for x86,
> they
> > > > needed to deal with broken hardware *after* the fact, and all
> they
> > > > could do is find /some/ distinguishing feature in order to guess
> > > which
> > > > exact hardware they might be running on. For arm64, it is the
> > > opposite
> > > > case. We are currently in a position where we can demand vendors
> to
> > > > comply with the standards they endorsed themselves, and (ab)using
> > > ACPI
> > > > + DMI as a de facto platform description rather than plain ACPI
> makes
> > > > me think the DT crowd were actually right from the beginning. It
> > > > *directly* violates the standardization principle, since it
> requires
> > > a
> > > > priori knowledge inside the OS that a certain 'generic' device
> must
> > > be
> > > > driven in a special way.
> > > >
> > > > So can anyone comment on the feasibility of adding support for
> > > devices
> > > > with vendor specific HIDs (and no generic CIDs) to the current
> ACPI
> > > > ECAM driver in Linux?
> 
> I don't think of ECAM support itself as a "driver".  It's just a
> service available to drivers, similar to OF resource parsing.
> 
> Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> supports extended config space.  It doesn't specify how we access that
> config space, so I think hardware with non-standard ECAM should still
> have PNP0A03 and PNP0A08 in _CID or _HID.
> 
> "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> r3.2, sec 4.1) means:
> 
>   (a) a memory-mapped model for config space access, and
>   (b) a specific mapping of address bits to bus/device/function/
>       register
> 
> MCFG and _CBA assume both (a) and (b), so I think a device with
> non-standard ECAM mappings should not be described in MCFG or _CBA.
> 
> If a bridge has ECAM with non-standard mappings, I think either a
> vendor-specific _HID or a device-specific method, e.g., _DSM, could
> communicate that.
> 
> Jon, I agree that we should avoid describing non-standardized hardware
> in Linux-specific ways.  Is there a mechanism in use already?  How
> does Windows handle this?  DMI is a poor long-term solution because it
> requires ongoing maintenance for new platforms, but I think it's OK
> for getting started with platforms already shipping.
> 
> A _DSM has the advantage that once it is defined and supported, OEMs
> can ship new platforms without requiring a new quirk or a new _HID to
> be added to a driver.
> 
> There would still be the problem of config access before the namespace
> is available, i.e., the MCFG use case.  I don't know how important
> that is.  Defining an MCFG extension seems like the most obvious
> solution.
> 
> If we only expect a few non-standard devices, maybe it's enough to
> have DMI quirks to statically set up ECAM and just live with the
> inconvenience of requiring a kernel change for every new non-standard
> device.
> 
> > > Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
> > > most of the arch specific code is handled in the respective arch
> > > directories (X86 and IA64, even though IA64 does not rely on
> ECAM/MCFG
> > > for
> > > PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected
> > > through
> > > ACPI scan handlers and the respective arch code (ie
> pci_acpi_scan_root)
> > > sets-up resources AND config space on an arch specific basis.
> > >
> > > X86 deals with that with code in arch/x86 that sets-up the
> pci_raw_ops
> > > on a platform specific basis (and it is not nice, but it works
> because
> > > as you all know the number of platforms in X86 world is contained).
> > >
> > > Will this happen for ARM64 in arch/arm64 based on vendor specific
> > > HIDs ?
> > >
> > > No.
> > >
> > > So given the current state of play (we were requested to move the
> > > arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
> > > with arch/arm64 code requiring code in /drivers to set-up pci_ops
> > > in a platform specific way, it is horrible, if feasible at all.
> > >
> > > The only way this can be implemented is by pretending that the
> > > ACPI/PCI arch/arm64 implementation is generic code (that's what
> this
> > > series does), move it to /drivers (where it is in this series), and
> > > implement _DSD vendor specific bindings (per HID) to set-up the pci
> > > operations; whether this solution should go upstream, given that it
> > > is just a short-term solution for early platforms bugs, it is
> another
> > > story and my personal answer is no.
> 
> It seems like there should be a way to look for a _DSM before we call
> acpi_pci_root_get_mcfg_addr() to look for _CBA.
> 
> Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
> the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
> only used in x86-specific code.  I think it would be nicer if the
> lookup and the use were together.  Then it would be easier to override
> it because the mapping assumptions would all be in one place.
> 
> > I think it shouldn't be too bad to move quirk handling mechanism to
> > arch/arm64. Effectively we would not move platform specific code into
> > arch/arm64 but just the mechanism checking if there is any quirk that
> > is defined.
> >
> > i.e.:
> >
> > extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
> > extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
> >
> > static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
> *root)
> > {
> >         int bus_num = root->secondary.start;
> >         int domain = root->segment;
> >         struct pci_cfg_fixup *f;
> >
> >         /*
> >          * Match against platform specific quirks and return
> corresponding
> >          * CAM ops.
> >          *
> >          * First match against PCI topology <domain:bus> then use DMI
> or
> >          * custom match handler.
> >          */
> >         for (f = __start_acpi_mcfg_fixups; f <
> __end_acpi_mcfg_fixups; f++) {
> >                 if ((f->domain == domain || f->domain ==
> PCI_MCFG_DOMAIN_ANY) &&
> >                     (f->bus_num == bus_num || f->bus_num ==
> PCI_MCFG_BUS_ANY) &&
> >                     (f->system ? dmi_check_system(f->system) : 1) &&
> >                     (f->match ? f->match(f, root) : 1))
> >                         return f->ops;
> >         }
> >         /* No quirks, use ECAM */
> >         return &pci_generic_ecam_ops;
> > }
> >
> > Such quirks will be defined anyway in drivers/pci/host/ in the vendor
> > specific quirk implementations.
> >
> > e.g. in HiSilicon case we would have
> >
> > DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
> > 			PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
> >
> > in "drivers/pci/host/pcie-hisi-acpi.c "
> >
> > Thanks
> >
> > Gab
> 
> Sorry Gab, I guess I was really responding to earlier messages :)
> 
> I don't really have much to say here, except that it doesn't seem
> right to have an MCFG that describes a non-standard ECAM mapping.

The ACPI table that this mechanism relies upon is the one discussed
in:
https://lkml.org/lkml/2016/3/9/91

As you can see MCFG describes ECAM mappings, but we have a motherboard
reserved resource outside the MCFG:
Device (RES0)
{
	Name (_HID, "HISI0081") // HiSi PCIe RC config base address
	Name (_CID, "PNP0C02") // Motherboard reserved resource
	Name (_CRS, ResourceTemplate (){
	Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000)
	})
}

This allows us to retrieve the address we need for accessing
the config space on the RC (that is non-ECAM).

I was thinking that such mechanism could fit different vendors
and allow them define their own quirks without spoiling the 
official and standard MCFG; also from the thread discussion
you seemed quite ok with such solution...?

> I suppose there's already firmware in the field that does that,
> though?

We have "experimental" firmware that is based on the ACPI table
described above, however it is not widely distributed and, obviously,
it is not supported by Linux mainline (so we have room to rework
if we decide that another solution is more appropriate)

Thanks and Regards

Gab

> 
> Bjorn

[toc] | [prev] | [next] | [standalone]

#1406218

Hi Gabriele, all,

On 05/24/2016 03:23 AM, Gabriele Paoloni wrote:

>> Sorry Gab, I guess I was really responding to earlier messages :)
>>
>> I don't really have much to say here, except that it doesn't seem
>> right to have an MCFG that describes a non-standard ECAM mapping.
> 
> The ACPI table that this mechanism relies upon is the one discussed
> in:
> https://lkml.org/lkml/2016/3/9/91
> 
> As you can see MCFG describes ECAM mappings, but we have a motherboard
> reserved resource outside the MCFG:
> Device (RES0)
> {
> 	Name (_HID, "HISI0081") // HiSi PCIe RC config base address
> 	Name (_CID, "PNP0C02") // Motherboard reserved resource
> 	Name (_CRS, ResourceTemplate (){
> 	Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000)
> 	})
> }
> 
> This allows us to retrieve the address we need for accessing
> the config space on the RC (that is non-ECAM).
> 
> I was thinking that such mechanism could fit different vendors
> and allow them define their own quirks without spoiling the 
> official and standard MCFG; also from the thread discussion
> you seemed quite ok with such solution...?

This could have been useful 2-3 years ago (when myself and others first
pulled the fire alarm concerning the lack of upstreaming of the ACPI
enablement for PCIe - which should have been fully upstream before the
first platforms ever even shipped) but at this time we have shipping
platforms that don't have tables built in this way. While we can go back
around to vendors and try to get them to rebuild firmware, it would be
by far preferable to adopt a solution that works with what is already
being deployed in the field today. Such as OEM match in MCFG.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

[toc] | [prev] | [next] | [standalone]

#1406300

Hi Bjorn,

On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:

[...]

> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> I don't think of ECAM support itself as a "driver".  It's just a
> service available to drivers, similar to OF resource parsing.
> 
> Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> supports extended config space.  It doesn't specify how we access that
> config space, so I think hardware with non-standard ECAM should still
> have PNP0A03 and PNP0A08 in _CID or _HID.
> 
> "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> r3.2, sec 4.1) means:
> 
>   (a) a memory-mapped model for config space access, and
>   (b) a specific mapping of address bits to bus/device/function/
>       register
> 
> MCFG and _CBA assume both (a) and (b), so I think a device with
> non-standard ECAM mappings should not be described in MCFG or _CBA.
> 
> If a bridge has ECAM with non-standard mappings, I think either a
> vendor-specific _HID or a device-specific method, e.g., _DSM, could
> communicate that.
> 
> Jon, I agree that we should avoid describing non-standardized hardware
> in Linux-specific ways.  Is there a mechanism in use already?  How
> does Windows handle this?  DMI is a poor long-term solution because it
> requires ongoing maintenance for new platforms, but I think it's OK
> for getting started with platforms already shipping.
> 
> A _DSM has the advantage that once it is defined and supported, OEMs
> can ship new platforms without requiring a new quirk or a new _HID to
> be added to a driver.
> 
> There would still be the problem of config access before the namespace
> is available, i.e., the MCFG use case.  I don't know how important
> that is.  Defining an MCFG extension seems like the most obvious
> solution.

Your summary above is a perfect representation of the situation.

We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.

1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
   ECAM for config space is basically ready (Tomasz and JC addressed
   Rafael's concerns in relation to ARM64 specific code, and managed
   to find a way to allocate domain numbers in preparation for Arnd
   pci_create_root_bus() clean-up, v8 to be posted shortly and should
   be final). This provides support for de-facto ACPI/PCI ECAM base
   standard for ARM64 (with a clean-split between generic code and ARM64
   bits, where ARM64, like X86 and IA64, manages in arch code IO space and
   PCI resources, to be further consolidated in the near future).
   I do not think anyone can complain about the generality of what we
   achieved, for systems that are PCI standard (yes, PCI STANDARD) this
   would just be sufficient.
2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
   ECAM compliant, already shipped with the corresponding firmware that
   we can't update. HW has ECAM quirks and to work around it in the kernel
   we put forward many solutions to the problem, it is time we found a
   solution (when, of course, (1) is completed and upstream).
   Using the MCFG table OEMID matching floated around in this thread
   would work fine for most of the platforms (and cross-OS) that have
   shipped with HW ECAM quirks, so I think that's the starting point for
   our solution and that's how we can sort this out, _today_.

   The solution is a trivial look-up table:
   MCFG OEMID <-> PCI config space ops

3) (2) does not just work on some platforms (and we can't predict the
   future either - actually I can, it is three letters, ECAM), simply
   because MCFG OEMID matching does not provide a way to attach further
   data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
   ECAM compliant, the config region can't be handled and must not be
   handled through a corresponding MCFG region.
   That's the problem Gabriele is facing and wants to solve through
   something like:

   https://lkml.org/lkml/2016/3/9/91

   in the respective ACPI tables-bindings. It may be an idea worth
   pursuing, it does not solve (2) simply because that FW has shipped,
   we can't patch it any longer.

Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
following steps, in order:

- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows
  us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that
  can we can accomodate further platforms that can't rely on (2) but
  have more leeway in terms of FW updates.

I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.

Thank you,
Lorenzo

[toc] | [prev] | [next] | [standalone]

#1406303 — Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller

A big +1 to the below :) :) :)

-- 
Computer Architect | Sent from my 64-bit #ARM Powered phone

> On May 24, 2016, at 13:24, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> 
> Hi Bjorn,
> 
> On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
> 
> [...]
> 
>> On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
>> I don't think of ECAM support itself as a "driver".  It's just a
>> service available to drivers, similar to OF resource parsing.
>> 
>> Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
>> host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
>> supports extended config space.  It doesn't specify how we access that
>> config space, so I think hardware with non-standard ECAM should still
>> have PNP0A03 and PNP0A08 in _CID or _HID.
>> 
>> "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
>> r3.2, sec 4.1) means:
>> 
>>  (a) a memory-mapped model for config space access, and
>>  (b) a specific mapping of address bits to bus/device/function/
>>      register
>> 
>> MCFG and _CBA assume both (a) and (b), so I think a device with
>> non-standard ECAM mappings should not be described in MCFG or _CBA.
>> 
>> If a bridge has ECAM with non-standard mappings, I think either a
>> vendor-specific _HID or a device-specific method, e.g., _DSM, could
>> communicate that.
>> 
>> Jon, I agree that we should avoid describing non-standardized hardware
>> in Linux-specific ways.  Is there a mechanism in use already?  How
>> does Windows handle this?  DMI is a poor long-term solution because it
>> requires ongoing maintenance for new platforms, but I think it's OK
>> for getting started with platforms already shipping.
>> 
>> A _DSM has the advantage that once it is defined and supported, OEMs
>> can ship new platforms without requiring a new quirk or a new _HID to
>> be added to a driver.
>> 
>> There would still be the problem of config access before the namespace
>> is available, i.e., the MCFG use case.  I don't know how important
>> that is.  Defining an MCFG extension seems like the most obvious
>> solution.
> 
> Your summary above is a perfect representation of the situation.
> 
> We had an opportunity to sync-up on the current status of ACPI PCI
> for ARM64 (and talked about a way forward for this series, which
> includes quirks handling), let me summarize it here for everyone
> involved so that we can agree on a way forward.
> 
> 1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
>   ECAM for config space is basically ready (Tomasz and JC addressed
>   Rafael's concerns in relation to ARM64 specific code, and managed
>   to find a way to allocate domain numbers in preparation for Arnd
>   pci_create_root_bus() clean-up, v8 to be posted shortly and should
>   be final). This provides support for de-facto ACPI/PCI ECAM base
>   standard for ARM64 (with a clean-split between generic code and ARM64
>   bits, where ARM64, like X86 and IA64, manages in arch code IO space and
>   PCI resources, to be further consolidated in the near future).
>   I do not think anyone can complain about the generality of what we
>   achieved, for systems that are PCI standard (yes, PCI STANDARD) this
>   would just be sufficient.
> 2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
>   ECAM compliant, already shipped with the corresponding firmware that
>   we can't update. HW has ECAM quirks and to work around it in the kernel
>   we put forward many solutions to the problem, it is time we found a
>   solution (when, of course, (1) is completed and upstream).
>   Using the MCFG table OEMID matching floated around in this thread
>   would work fine for most of the platforms (and cross-OS) that have
>   shipped with HW ECAM quirks, so I think that's the starting point for
>   our solution and that's how we can sort this out, _today_.
> 
>   The solution is a trivial look-up table:
>   MCFG OEMID <-> PCI config space ops
> 
> 3) (2) does not just work on some platforms (and we can't predict the
>   future either - actually I can, it is three letters, ECAM), simply
>   because MCFG OEMID matching does not provide a way to attach further
>   data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
>   ECAM compliant, the config region can't be handled and must not be
>   handled through a corresponding MCFG region.
>   That's the problem Gabriele is facing and wants to solve through
>   something like:
> 
>   https://lkml.org/lkml/2016/3/9/91
> 
>   in the respective ACPI tables-bindings. It may be an idea worth
>   pursuing, it does not solve (2) simply because that FW has shipped,
>   we can't patch it any longer.
> 
> Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
> following steps, in order:
> 
> - Let's complete/merge (1), that's fundamental to this whole thread
> - On top of (1) we apply a quirking mechanism based on (2) that allows
>  us to boot mainline with boxes shipping today with no FW update required.
> - We devise a way to handle quirks that is more generic than (2) so that
>  can we can accomodate further platforms that can't rely on (2) but
>  have more leeway in terms of FW updates.
> 
> I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
> 
> Thank you,
> Lorenzo

[toc] | [prev] | [next] | [standalone]

#1406397

On Tue, May 24, 2016 at 06:24:23PM +0100, Lorenzo Pieralisi wrote:
> Hi Bjorn,
> 
> On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
> 
> [...]
> 
> > On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> > I don't think of ECAM support itself as a "driver".  It's just a
> > service available to drivers, similar to OF resource parsing.
> > 
> > Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> > host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> > supports extended config space.  It doesn't specify how we access that
> > config space, so I think hardware with non-standard ECAM should still
> > have PNP0A03 and PNP0A08 in _CID or _HID.
> > 
> > "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> > r3.2, sec 4.1) means:
> > 
> >   (a) a memory-mapped model for config space access, and
> >   (b) a specific mapping of address bits to bus/device/function/
> >       register
> > 
> > MCFG and _CBA assume both (a) and (b), so I think a device with
> > non-standard ECAM mappings should not be described in MCFG or _CBA.
> > 
> > If a bridge has ECAM with non-standard mappings, I think either a
> > vendor-specific _HID or a device-specific method, e.g., _DSM, could
> > communicate that.
> > 
> > Jon, I agree that we should avoid describing non-standardized hardware
> > in Linux-specific ways.  Is there a mechanism in use already?  How
> > does Windows handle this?  DMI is a poor long-term solution because it
> > requires ongoing maintenance for new platforms, but I think it's OK
> > for getting started with platforms already shipping.
> > 
> > A _DSM has the advantage that once it is defined and supported, OEMs
> > can ship new platforms without requiring a new quirk or a new _HID to
> > be added to a driver.
> > 
> > There would still be the problem of config access before the namespace
> > is available, i.e., the MCFG use case.  I don't know how important
> > that is.  Defining an MCFG extension seems like the most obvious
> > solution.
> 
> Your summary above is a perfect representation of the situation.
> 
> We had an opportunity to sync-up on the current status of ACPI PCI
> for ARM64 (and talked about a way forward for this series, which
> includes quirks handling), let me summarize it here for everyone
> involved so that we can agree on a way forward.
> 
> 1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
>    ECAM for config space is basically ready (Tomasz and JC addressed
>    Rafael's concerns in relation to ARM64 specific code, and managed
>    to find a way to allocate domain numbers in preparation for Arnd
>    pci_create_root_bus() clean-up, v8 to be posted shortly and should
>    be final). This provides support for de-facto ACPI/PCI ECAM base
>    standard for ARM64 (with a clean-split between generic code and ARM64
>    bits, where ARM64, like X86 and IA64, manages in arch code IO space and
>    PCI resources, to be further consolidated in the near future).
>    I do not think anyone can complain about the generality of what we
>    achieved, for systems that are PCI standard (yes, PCI STANDARD) this
>    would just be sufficient.

Sounds good to me.

> 2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
>    ECAM compliant, already shipped with the corresponding firmware that
>    we can't update. HW has ECAM quirks and to work around it in the kernel
>    we put forward many solutions to the problem, it is time we found a
>    solution (when, of course, (1) is completed and upstream).
>    Using the MCFG table OEMID matching floated around in this thread
>    would work fine for most of the platforms (and cross-OS) that have
>    shipped with HW ECAM quirks, so I think that's the starting point for
>    our solution and that's how we can sort this out, _today_.
> 
>    The solution is a trivial look-up table:
>    MCFG OEMID <-> PCI config space ops

Sounds reasonable to me.

> 3) (2) does not just work on some platforms (and we can't predict the
>    future either - actually I can, it is three letters, ECAM), simply
>    because MCFG OEMID matching does not provide a way to attach further
>    data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
>    ECAM compliant, the config region can't be handled and must not be
>    handled through a corresponding MCFG region.

Couldn't this be handled by custom pci_ops that do something special
for bus 0 domain 0, and default to some different pci_ops for the
rest?

>    That's the problem Gabriele is facing and wants to solve through
>    something like:
> 
>    https://lkml.org/lkml/2016/3/9/91
> 
>    in the respective ACPI tables-bindings. It may be an idea worth
>    pursuing, it does not solve (2) simply because that FW has shipped,
>    we can't patch it any longer.

(2) is for quirks to deal with MCFG.  Gabriele's post is a proposal
for ACPI namespace.  We can't use anything in the namespace to
implement MCFG quirks because MCFG is needed before the namespace is
available.

I think Gabriele's post is a good proposal for the namespace, but I
would propose the following modifications:

  - Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge
  - Add a PCI1 _DSM describing the ECAM space
  - Remove the HISI0081 _HID

The result would be:

  Device (PCI1) {
    Name(_HID, "HISI0080")
    Name(_CID, "PNP0A03,PNP0A08")
    Method(_CRS) { ... }
    Method(_DSM) { <describe ECAM base and mapping function> }
  }
  Device (RES0) {
    Name(_HID, "PNP0C02")
    Name(_CRS) { <describe ECAM base and size> }
  }

RES0 could also be contained within PCI1, as Gabriele suggested.  I
don't really care whether it's contained or not, and making it
contained might make it easier for firmware, because addition/removal
of PCI1 and RES0 should always happen together.

I think the _DSM is important because it is really ugly if the
HISI0080 driver has to look for a separate HISI0081 device to learn
about the ECAM space.  There are several PCI drivers that do something
similar, using for_each_pci_dev(), and I cringe every time I see them,
because this totally screws up the driver model.  A driver should
claim a device via a .probe() method called by the core, and it
shouldn't look at devices it hasn't claimed.  This is required to make
hotplug work correctly.

> Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
> following steps, in order:
> 
> - Let's complete/merge (1), that's fundamental to this whole thread
> - On top of (1) we apply a quirking mechanism based on (2) that allows
>   us to boot mainline with boxes shipping today with no FW update required.
> - We devise a way to handle quirks that is more generic than (2) so that
>   can we can accomodate further platforms that can't rely on (2) but
>   have more leeway in terms of FW updates.
> 
> I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.

Sounds very good to me; I'm looking forward to v8.

Bjorn

[toc] | [prev] | [next] | [standalone]

#1407488

Hi Bjorn many thanks for your suggestions

> -----Original Message-----
> From: Bjorn Helgaas [mailto:helgaas@kernel.org]
> Sent: 24 May 2016 20:01
> To: Lorenzo Pieralisi
> Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> On Tue, May 24, 2016 at 06:24:23PM +0100, Lorenzo Pieralisi wrote:
> > Hi Bjorn,
> >
> > On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
> >
> > [...]
> >
> > > On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> > > I don't think of ECAM support itself as a "driver".  It's just a
> > > service available to drivers, similar to OF resource parsing.
> > >
> > > Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> > > host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> > > supports extended config space.  It doesn't specify how we access
> that
> > > config space, so I think hardware with non-standard ECAM should
> still
> > > have PNP0A03 and PNP0A08 in _CID or _HID.
> > >
> > > "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> > > r3.2, sec 4.1) means:
> > >
> > >   (a) a memory-mapped model for config space access, and
> > >   (b) a specific mapping of address bits to bus/device/function/
> > >       register
> > >
> > > MCFG and _CBA assume both (a) and (b), so I think a device with
> > > non-standard ECAM mappings should not be described in MCFG or _CBA.
> > >
> > > If a bridge has ECAM with non-standard mappings, I think either a
> > > vendor-specific _HID or a device-specific method, e.g., _DSM, could
> > > communicate that.
> > >
> > > Jon, I agree that we should avoid describing non-standardized
> hardware
> > > in Linux-specific ways.  Is there a mechanism in use already?  How
> > > does Windows handle this?  DMI is a poor long-term solution because
> it
> > > requires ongoing maintenance for new platforms, but I think it's OK
> > > for getting started with platforms already shipping.
> > >
> > > A _DSM has the advantage that once it is defined and supported,
> OEMs
> > > can ship new platforms without requiring a new quirk or a new _HID
> to
> > > be added to a driver.
> > >
> > > There would still be the problem of config access before the
> namespace
> > > is available, i.e., the MCFG use case.  I don't know how important
> > > that is.  Defining an MCFG extension seems like the most obvious
> > > solution.
> >
> > Your summary above is a perfect representation of the situation.
> >
> > We had an opportunity to sync-up on the current status of ACPI PCI
> > for ARM64 (and talked about a way forward for this series, which
> > includes quirks handling), let me summarize it here for everyone
> > involved so that we can agree on a way forward.
> >
> > 1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
> >    ECAM for config space is basically ready (Tomasz and JC addressed
> >    Rafael's concerns in relation to ARM64 specific code, and managed
> >    to find a way to allocate domain numbers in preparation for Arnd
> >    pci_create_root_bus() clean-up, v8 to be posted shortly and should
> >    be final). This provides support for de-facto ACPI/PCI ECAM base
> >    standard for ARM64 (with a clean-split between generic code and
> ARM64
> >    bits, where ARM64, like X86 and IA64, manages in arch code IO
> space and
> >    PCI resources, to be further consolidated in the near future).
> >    I do not think anyone can complain about the generality of what we
> >    achieved, for systems that are PCI standard (yes, PCI STANDARD)
> this
> >    would just be sufficient.
> 
> Sounds good to me.
> 
> > 2) In a real world (1) is not enough. Some ARM64 platforms, not
> entirely
> >    ECAM compliant, already shipped with the corresponding firmware
> that
> >    we can't update. HW has ECAM quirks and to work around it in the
> kernel
> >    we put forward many solutions to the problem, it is time we found
> a
> >    solution (when, of course, (1) is completed and upstream).
> >    Using the MCFG table OEMID matching floated around in this thread
> >    would work fine for most of the platforms (and cross-OS) that have
> >    shipped with HW ECAM quirks, so I think that's the starting point
> for
> >    our solution and that's how we can sort this out, _today_.
> >
> >    The solution is a trivial look-up table:
> >    MCFG OEMID <-> PCI config space ops
> 
> Sounds reasonable to me.
> 
> > 3) (2) does not just work on some platforms (and we can't predict the
> >    future either - actually I can, it is three letters, ECAM), simply
> >    because MCFG OEMID matching does not provide a way to attach
> further
> >    data to the MCFG (eg if config space for, say, bus 0 domain 0, is
> not
> >    ECAM compliant, the config region can't be handled and must not be
> >    handled through a corresponding MCFG region.
> 
> Couldn't this be handled by custom pci_ops that do something special
> for bus 0 domain 0, and default to some different pci_ops for the
> rest?

My idea was to remove bus 0 from domain 0 MCFG (and also the other
buses corresponding to the root complexes ports)

So the driver would use the ECAM access with addresses retrieved from
MCFG for any devices except the RCs.

For the RC we could retrieve special addresses from the "PNP0C02"
reserved resource... 


> 
> >    That's the problem Gabriele is facing and wants to solve through
> >    something like:
> >
> >    https://lkml.org/lkml/2016/3/9/91
> >
> >    in the respective ACPI tables-bindings. It may be an idea worth
> >    pursuing, it does not solve (2) simply because that FW has
> shipped,
> >    we can't patch it any longer.
> 
> (2) is for quirks to deal with MCFG.  Gabriele's post is a proposal
> for ACPI namespace.  We can't use anything in the namespace to
> implement MCFG quirks because MCFG is needed before the namespace is
> available.

Honestly I have to dig better into this (and I will once V8 is out),
However from my current understanding so far we have look-up where
MCFG OEMID is going to tell which specific pci-ops are going to be used.

Now from my quirk idea I need to retrieve the "special address" from the
ACPI namespace before cfg rd/wr take place...if this is not doable then
I need to find a different solution...than can be the one you proposed
below (many thanks for this).

I will look into details later on once v8 is out.

Thanks

Gab

> 
> I think Gabriele's post is a good proposal for the namespace, but I
> would propose the following modifications:
> 
>   - Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge
>   - Add a PCI1 _DSM describing the ECAM space
>   - Remove the HISI0081 _HID
> 
> The result would be:
> 
>   Device (PCI1) {
>     Name(_HID, "HISI0080")
>     Name(_CID, "PNP0A03,PNP0A08")
>     Method(_CRS) { ... }
>     Method(_DSM) { <describe ECAM base and mapping function> }
>   }
>   Device (RES0) {
>     Name(_HID, "PNP0C02")
>     Name(_CRS) { <describe ECAM base and size> }
>   }
> 
> RES0 could also be contained within PCI1, as Gabriele suggested.  I
> don't really care whether it's contained or not, and making it
> contained might make it easier for firmware, because addition/removal
> of PCI1 and RES0 should always happen together.
> 
> I think the _DSM is important because it is really ugly if the
> HISI0080 driver has to look for a separate HISI0081 device to learn
> about the ECAM space.  There are several PCI drivers that do something
> similar, using for_each_pci_dev(), and I cringe every time I see them,
> because this totally screws up the driver model.  A driver should
> claim a device via a .probe() method called by the core, and it
> shouldn't look at devices it hasn't claimed.  This is required to make
> hotplug work correctly.
> 
> > Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
> > following steps, in order:
> >
> > - Let's complete/merge (1), that's fundamental to this whole thread
> > - On top of (1) we apply a quirking mechanism based on (2) that
> allows
> >   us to boot mainline with boxes shipping today with no FW update
> required.
> > - We devise a way to handle quirks that is more generic than (2) so
> that
> >   can we can accomodate further platforms that can't rely on (2) but
> >   have more leeway in terms of FW updates.
> >
> > I hope that's a reasonable plan, Tomasz's v8 series coming to kick it
> off.
> 
> Sounds very good to me; I'm looking forward to v8.
> 
> Bjorn

[toc] | [prev] | [next] | [standalone]

#1406680

Hi Lorenzo

> -----Original Message-----
> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> Sent: 24 May 2016 18:24
> To: Bjorn Helgaas
> Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org;
> jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux-
> pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com;
> ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-
> kernel@vger.kernel.org; linux-acpi@vger.kernel.org;
> robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com;
> msalter@redhat.com; Wangyijing; mw@semihalf.com;
> andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
> controller
> 
> Hi Bjorn,
> 
> On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
> 
> [...]
> 
> > On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
> > I don't think of ECAM support itself as a "driver".  It's just a
> > service available to drivers, similar to OF resource parsing.
> >
> > Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
> > host bridge.  "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
> > supports extended config space.  It doesn't specify how we access
> that
> > config space, so I think hardware with non-standard ECAM should still
> > have PNP0A03 and PNP0A08 in _CID or _HID.
> >
> > "ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
> > r3.2, sec 4.1) means:
> >
> >   (a) a memory-mapped model for config space access, and
> >   (b) a specific mapping of address bits to bus/device/function/
> >       register
> >
> > MCFG and _CBA assume both (a) and (b), so I think a device with
> > non-standard ECAM mappings should not be described in MCFG or _CBA.
> >
> > If a bridge has ECAM with non-standard mappings, I think either a
> > vendor-specific _HID or a device-specific method, e.g., _DSM, could
> > communicate that.
> >
> > Jon, I agree that we should avoid describing non-standardized
> hardware
> > in Linux-specific ways.  Is there a mechanism in use already?  How
> > does Windows handle this?  DMI is a poor long-term solution because
> it
> > requires ongoing maintenance for new platforms, but I think it's OK
> > for getting started with platforms already shipping.
> >
> > A _DSM has the advantage that once it is defined and supported, OEMs
> > can ship new platforms without requiring a new quirk or a new _HID to
> > be added to a driver.
> >
> > There would still be the problem of config access before the
> namespace
> > is available, i.e., the MCFG use case.  I don't know how important
> > that is.  Defining an MCFG extension seems like the most obvious
> > solution.
> 
> Your summary above is a perfect representation of the situation.
> 
> We had an opportunity to sync-up on the current status of ACPI PCI
> for ARM64 (and talked about a way forward for this series, which
> includes quirks handling), let me summarize it here for everyone
> involved so that we can agree on a way forward.
> 
> 1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
>    ECAM for config space is basically ready (Tomasz and JC addressed
>    Rafael's concerns in relation to ARM64 specific code, and managed
>    to find a way to allocate domain numbers in preparation for Arnd
>    pci_create_root_bus() clean-up, v8 to be posted shortly and should
>    be final). This provides support for de-facto ACPI/PCI ECAM base
>    standard for ARM64 (with a clean-split between generic code and
> ARM64
>    bits, where ARM64, like X86 and IA64, manages in arch code IO space
> and
>    PCI resources, to be further consolidated in the near future).
>    I do not think anyone can complain about the generality of what we
>    achieved, for systems that are PCI standard (yes, PCI STANDARD) this
>    would just be sufficient.
> 2) In a real world (1) is not enough. Some ARM64 platforms, not
> entirely
>    ECAM compliant, already shipped with the corresponding firmware that
>    we can't update. HW has ECAM quirks and to work around it in the
> kernel
>    we put forward many solutions to the problem, it is time we found a
>    solution (when, of course, (1) is completed and upstream).
>    Using the MCFG table OEMID matching floated around in this thread
>    would work fine for most of the platforms (and cross-OS) that have
>    shipped with HW ECAM quirks, so I think that's the starting point
> for
>    our solution and that's how we can sort this out, _today_.
> 
>    The solution is a trivial look-up table:
>    MCFG OEMID <-> PCI config space ops
> 
> 3) (2) does not just work on some platforms (and we can't predict the
>    future either - actually I can, it is three letters, ECAM), simply
>    because MCFG OEMID matching does not provide a way to attach further
>    data to the MCFG (eg if config space for, say, bus 0 domain 0, is
> not
>    ECAM compliant, the config region can't be handled and must not be
>    handled through a corresponding MCFG region.
>    That's the problem Gabriele is facing and wants to solve through
>    something like:
> 
>    https://lkml.org/lkml/2016/3/9/91
> 
>    in the respective ACPI tables-bindings. It may be an idea worth
>    pursuing, it does not solve (2) simply because that FW has shipped,
>    we can't patch it any longer.
> 
> Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
> following steps, in order:
> 
> - Let's complete/merge (1), that's fundamental to this whole thread
> - On top of (1) we apply a quirking mechanism based on (2) that allows
>   us to boot mainline with boxes shipping today with no FW update
> required.
> - We devise a way to handle quirks that is more generic than (2) so
> that
>   can we can accomodate further platforms that can't rely on (2) but
>   have more leeway in terms of FW updates.
> 
> I hope that's a reasonable plan, Tomasz's v8 series coming to kick it
> off.

Thanks for summarizing.

100% agree on the summary and next steps.

Cheers

Gab


> 
> Thank you,
> Lorenzo

[toc] | [prev] | [next] | [standalone]

#1405823

On 05/23/2016 06:56 AM, Lorenzo Pieralisi wrote:

> The only way this can be implemented is by pretending that the
> ACPI/PCI arch/arm64 implementation is generic code (that's what this
> series does), move it to /drivers (where it is in this series), and
> implement _DSD vendor specific bindings (per HID) to set-up the pci
> operations; whether this solution should go upstream, given that it
> is just a short-term solution for early platforms bugs, it is another
> story and my personal answer is no.

Just for completeness, let me also followup to this one.

We have real, shipping, systems in the field based on ARMv8. For
example, HPE Moonshot ProLiant m400. Not everyone loves the first
generation of anything (Applied get a lot of stick, but the reality is
that someone had to go first, and whoever that was was going to learn
all of the lessons that others don't need to) but it is out there and we
need an upstream kernel solution that includes support for that.

In the server world, we (speaking as a major distro vendor here) are not
going to entertain a situation in which non-upstream patches are needed
to even boot a platform. That simply won't do. We need to separate out
the issue of getting the core in place from the quirks, but then we need
quirks that include support for all early ARMv8 platforms that are out
there today. If we can't get to a point where a Moonshot[0] cartridge
boots out of the box with an upstream kernel, let's just give up and do
something else instead :) (joke)

Jon.

[0] HPE have been *amazingly* patient with this stuff. They've reworked
the firmware when someone (cough) pointed out that the early stuff
they'd been fed was not built according to the standards (U-Boot). They
have *really good* UEFI and ACPI enabled firmware that is running
RHEL(SA) great. But that's not good enough. We don't ship a distro with
hacks. We ship a distro derived from upstream code (although we might
have to backport a lot of stuff later). There's wiggle room, but there
is not wiggle room for core platforms. On ARM, users and developers
*will* be able to take an upstream kernel and boot it on their RHEL
install. And they *will* be able to reproduce problems against upstream,
and help develop against upstream, and further the upstream first
mentality in the ARM ecosystem. There will not be "oh well, it runs RHEL
so that's good enough for the early generation...".

-- 
Computer Architect | Sent from my Fedora powered laptop

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web