Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1364668 > unrolled thread
| Started by | khalasa@piap.pl (Krzysztof Hałasa) |
|---|---|
| First post | 2016-03-25 14:40 +0100 |
| Last post | 2016-03-30 18:20 +0200 |
| Articles | 20 on this page of 41 — 9 participants |
Back to article view | Back to linux.kernel
[PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-25 14:40 +0100
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-27 16:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-28 02:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-28 22:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-28 22:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-28 22:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-28 23:40 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 00:10 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-29 00:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-29 07:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-29 07:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-29 07:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Lucas Stach <l.stach@pengutronix.de> - 2016-03-29 11:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-29 12:40 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Lucas Stach <l.stach@pengutronix.de> - 2016-03-29 13:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-29 15:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 15:40 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-29 16:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 16:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-29 17:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 17:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-29 17:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 19:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-29 21:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Marc Zyngier <marc.zyngier@arm.com> - 2016-03-29 20:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Roberto Fichera <kernel@tekno-soft.it> - 2016-03-29 18:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Roberto Fichera <kernel@tekno-soft.it> - 2016-03-29 18:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 19:40 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Roberto Fichera <kernel@tekno-soft.it> - 2016-03-30 10:10 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Arnd Bergmann <arnd@arndb.de> - 2016-03-30 12:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Roberto Fichera <kernel@tekno-soft.it> - 2016-03-30 15:00 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-30 15:40 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Roberto Fichera <kernel@tekno-soft.it> - 2016-03-30 17:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-29 18:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-30 10:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Tim Harvey <tharvey@gateworks.com> - 2016-03-31 18:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-29 16:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity khalasa@piap.pl (Krzysztof Hałasa) - 2016-03-29 07:30 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Petr Štetiar <ynezz@true.cz> - 2016-03-30 14:20 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Fabio Estevam <festevam@gmail.com> - 2016-03-30 14:50 +0200
Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity Marcel Ziswiler <marcel.ziswiler@toradex.com> - 2016-03-30 18:20 +0200
Page 2 of 3 — ← Prev page 1 [2] 3 Next page →
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-29 17:20 +0200 |
| Message-ID | <ri3AZ-86I-3@gated-at.bofh.it> |
| In reply to | #1366361 |
On Tue, Mar 29, 2016 at 7:50 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Tuesday 29 March 2016 07:29:34 Tim Harvey wrote:
>> On Tue, Mar 29, 2016 at 6:52 AM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Tuesday 29 March 2016 06:32:29 Tim Harvey wrote:
>
>> >> I think 31e98e0d24cd2537a63e06e235e050a06b175df7 "ARM:
>> >> imx_v6_v7_defconfig: enable PCI_MSI" should be reverted as well until
>> >> we figure this out.
>> >
>> > That doesn't sound like a helpful solution, multi_v7_defconfig for
>> > instance will still be broken because it enables PCI_MSI, and so
>> > will be all major distros.
>>
>> Arnd,
>>
>> True, but keep in mind that as far as I can tell this behavior has
>> been around since MSI was added to the IMX6 (breakage of devices that
>> use legacy irq's if MSI is enabled) which was in 3.16.
>
> I see. This part wasn't clear to me.
>
>> > What happens if we patch the pci-imx6 driver to not make use of
>> > its MSI support even when that is enabled in the kernel? Does that
>> > get both devices in your GW5xxx to work with legacy interrupts or
>> > is one or both of them still broken?
>> >
>> > Arnd
>> >
>> > diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
>> > index eb5a2755a164..d7607b2695c6 100644
>> > --- a/drivers/pci/host/pci-imx6.c
>> > +++ b/drivers/pci/host/pci-imx6.c
>> > @@ -470,7 +470,7 @@ static void imx6_pcie_host_init(struct pcie_port *pp)
>> >
>> > imx6_pcie_establish_link(pp);
>> >
>> > - if (IS_ENABLED(CONFIG_PCI_MSI))
>> > + if (0 && IS_ENABLED(CONFIG_PCI_MSI))
>> > dw_pcie_msi_init(pp);
>> > }
>> >
>> > @@ -490,7 +490,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
>> > {
>> > int ret;
>> >
>> > - if (IS_ENABLED(CONFIG_PCI_MSI)) {
>> > + if (0 && IS_ENABLED(CONFIG_PCI_MSI)) {
>> > pp->msi_irq = platform_get_irq_byname(pdev, "msi");
>> > if (pp->msi_irq <= 0) {
>> > dev_err(&pdev->dev, "failed to get MSI irq\n");
>> >
>>
>> That is not enough - we would also need to disable a couple more in
>> the designware core that imx6 uses, which is also used by several
>> other SoC's. We should probably get some feedback from people with
>> those SoC's regarding MSI breaking legacy irqs.
>
> Good point. I really just meant this as an experiment, trying to
> figure out what causes it to break. I'd be surprised if the
> MSI support in the generic pcie-designware driver caused the same
> problem on the other SoCs.
>
>> PCI_MSI was enabled in imx_v6_v7_defconfig in 4.5 and enabled in
>> multi_v7_defconfig back in 3.16. PCI MSI was first introduced for the
>> IMX6 host controller in 3.16 as well. I verified that the same issue
>> exists all the way back to 3.16.
>
> Thanks for doing that research.
>
>> I don't know if its worse to disable PCI MSI for IMX6/designware all
>> the way back to 3.16 or to disable it just back to 4.5 where imx_v6_v7
>> enabled it, or perhaps just figure out the actual issue and get that
>> backported?
>
> I'd like to see the problem understood better before we talk about
> backports.
>
> One thing I just noticed is that the same GPC interrupt line (GIC_SPI
> 120) is used for MSI and for IntD. Maybe there is something going on with
> sharing an interrupt line between a nested irqchip and a device?
>
> Could this be a bug in the generic IRQ handling code? Can you check
> which interrupt the broken device(s) on your machine are using? Is
> it always the 120 (IntD) line, or are the other three lines broken
> as well? I don't actually know how to look it up, but the 'lspci -t'
> output should let us reconstruct the swizzling manually.
>
> Arnd
Arnd,
Right, on the IMX the MSI interrupt is GIC-120 which is also the
legacy INTD and I do see that if I happen to put a radio in a slot
where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt
does fire and the device works. Any other slot using GIC-123 (INTA),
GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible
that something in the designware core is masking out the legacy irqs.
I would also think this was something IMX specific, but I really don't
see any codepaths in pci-imx6.c that would cause that: a driver
requesting a legacy PCI would get a GIC interrupt which is handled by
the IMX6 gpc interrupt controller.
Any dra7xxx, exynos, spear13xx, keystone, layerscape, hisi, qcom SoC
users of designware PCIe core out there that can verify PCI MSI and
legacy are both working at the same time?
Lucas is the expert here and I believe he has the documentation for
the designware core that Freescale doens't provide with the IMX6
documentation so hopefully he can provide some insight. He's the one
that has authored all the MSI support and has been using it.
I typically advise our users to 'not' enable MSI because
architecturally you can spread 4 distinct legacy irq's across CPU's
better than a single shared irq.
Tim
[toc] | [prev] | [next] | [standalone]
| From | Arnd Bergmann <arnd@arndb.de> |
|---|---|
| Date | 2016-03-29 17:30 +0200 |
| Message-ID | <ri3KG-8bX-1@gated-at.bofh.it> |
| In reply to | #1366373 |
On Tuesday 29 March 2016 08:10:08 Tim Harvey wrote: > Arnd, > > Right, on the IMX the MSI interrupt is GIC-120 which is also the > legacy INTD and I do see that if I happen to put a radio in a slot > where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt > does fire and the device works. Any other slot using GIC-123 (INTA), > GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible > that something in the designware core is masking out the legacy irqs. Interesting. I was actually expecting the opposite here, having the IRQs only work if they are not IntD. > I typically advise our users to 'not' enable MSI because > architecturally you can spread 4 distinct legacy irq's across CPU's > better than a single shared irq. That is a very good point, I never understood why we want to enable MSI support on any PCI host bridge that just forwards all MSIs to a single IRQ line. Originally MSI was meant as a performance feature, but there is nothing in this setup that makes things go faster, and several things that make it go slower. I would still hope that with disabling MSI support in just the i.MX driver (as in the trivial patch I suggested trying, or by reverting Lucas' d1dc9749a5b8 patch) will make things just work. If not, we may need to change the pcie-designware driver as well, so it doesn't try to enable MSI on its own. Arnd
[toc] | [prev] | [next] | [standalone]
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-29 19:50 +0200 |
| Message-ID | <ri5Wa-18l-7@gated-at.bofh.it> |
| In reply to | #1366383 |
On Tue, Mar 29, 2016 at 8:24 AM, Arnd Bergmann <arnd@arndb.de> wrote: > On Tuesday 29 March 2016 08:10:08 Tim Harvey wrote: >> Arnd, >> >> Right, on the IMX the MSI interrupt is GIC-120 which is also the >> legacy INTD and I do see that if I happen to put a radio in a slot >> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt >> does fire and the device works. Any other slot using GIC-123 (INTA), >> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible >> that something in the designware core is masking out the legacy irqs. > > Interesting. I was actually expecting the opposite here, having the > IRQs only work if they are not IntD. > > >> I typically advise our users to 'not' enable MSI because >> architecturally you can spread 4 distinct legacy irq's across CPU's >> better than a single shared irq. > > That is a very good point, I never understood why we want to enable > MSI support on any PCI host bridge that just forwards all MSIs > to a single IRQ line. Originally MSI was meant as a performance > feature, but there is nothing in this setup that makes things go > faster, and several things that make it go slower. I had a conversation once with Lucas about implementing the shared MSI interrupt in such a way that its smp affinity could be set to other CPU's to gain a performance benefit in certain multi-device cases. While this is technically possible it would involve creating a softirq glue between the different handlers but that would add overhead of a softirq plus potentially waking up another CPU to every IRQ which would end up adding some overhead to even the simple single-device case. Without any hard data it wasn't clear if this was worth it or if there was a clean way to provide this as build-time or run-time option. Tim
[toc] | [prev] | [next] | [standalone]
| From | Arnd Bergmann <arnd@arndb.de> |
|---|---|
| Date | 2016-03-29 21:50 +0200 |
| Message-ID | <ri7Oi-2sv-27@gated-at.bofh.it> |
| In reply to | #1366515 |
On Tuesday 29 March 2016 10:38:16 Tim Harvey wrote: > On Tue, Mar 29, 2016 at 8:24 AM, Arnd Bergmann <arnd@arndb.de> wrote: > > On Tuesday 29 March 2016 08:10:08 Tim Harvey wrote: > >> Arnd, > >> > >> Right, on the IMX the MSI interrupt is GIC-120 which is also the > >> legacy INTD and I do see that if I happen to put a radio in a slot > >> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt > >> does fire and the device works. Any other slot using GIC-123 (INTA), > >> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible > >> that something in the designware core is masking out the legacy irqs. > > > > Interesting. I was actually expecting the opposite here, having the > > IRQs only work if they are not IntD. > > > > > >> I typically advise our users to 'not' enable MSI because > >> architecturally you can spread 4 distinct legacy irq's across CPU's > >> better than a single shared irq. > > > > That is a very good point, I never understood why we want to enable > > MSI support on any PCI host bridge that just forwards all MSIs > > to a single IRQ line. Originally MSI was meant as a performance > > feature, but there is nothing in this setup that makes things go > > faster, and several things that make it go slower. > > I had a conversation once with Lucas about implementing the shared MSI > interrupt in such a way that its smp affinity could be set to other > CPU's to gain a performance benefit in certain multi-device cases. > > While this is technically possible it would involve creating a softirq > glue between the different handlers but that would add overhead of a > softirq plus potentially waking up another CPU to every IRQ which > would end up adding some overhead to even the simple single-device > case. > > Without any hard data it wasn't clear if this was worth it or if there > was a clean way to provide this as build-time or run-time option. I think it's pretty clear that this would take things from 'somewhat silly' to 'completely bonkers' ;-) Arnd
[toc] | [prev] | [next] | [standalone]
| From | Marc Zyngier <marc.zyngier@arm.com> |
|---|---|
| Date | 2016-03-29 20:00 +0200 |
| Message-ID | <ri65Q-1dr-11@gated-at.bofh.it> |
| In reply to | #1366383 |
On 29/03/16 16:24, Arnd Bergmann wrote: > On Tuesday 29 March 2016 08:10:08 Tim Harvey wrote: >> Arnd, >> >> Right, on the IMX the MSI interrupt is GIC-120 which is also the >> legacy INTD and I do see that if I happen to put a radio in a slot >> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt >> does fire and the device works. Any other slot using GIC-123 (INTA), >> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible >> that something in the designware core is masking out the legacy irqs. > > Interesting. I was actually expecting the opposite here, having the > IRQs only work if they are not IntD. > > >> I typically advise our users to 'not' enable MSI because >> architecturally you can spread 4 distinct legacy irq's across CPU's >> better than a single shared irq. > > That is a very good point, I never understood why we want to enable > MSI support on any PCI host bridge that just forwards all MSIs > to a single IRQ line. Originally MSI was meant as a performance > feature, but there is nothing in this setup that makes things go > faster, and several things that make it go slower. Feature-ticking exercise. "We support MSI", never mind if that negating the benefits of the mechanism and ending up with disastrous impacts on interrupt affinity, and a set of open questions regarding the effect of the MSI as a DMA fence. /me stops ranting for the day... M. -- Jazz is not dead. It just smells funny...
[toc] | [prev] | [next] | [standalone]
| From | Roberto Fichera <kernel@tekno-soft.it> |
|---|---|
| Date | 2016-03-29 18:30 +0200 |
| Message-ID | <ri4GL-pQ-41@gated-at.bofh.it> |
| In reply to | #1366373 |
On 03/29/2016 05:10 PM, Tim Harvey wrote: > Arnd, > > Right, on the IMX the MSI interrupt is GIC-120 which is also the > legacy INTD and I do see that if I happen to put a radio in a slot > where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt > does fire and the device works. Any other slot using GIC-123 (INTA), > GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible > that something in the designware core is masking out the legacy irqs. > I would also think this was something IMX specific, but I really don't > see any codepaths in pci-imx6.c that would cause that: a driver > requesting a legacy PCI would get a GIC interrupt which is handled by > the IMX6 gpc interrupt controller. > > Any dra7xxx, exynos, spear13xx, keystone, layerscape, hisi, qcom SoC > users of designware PCIe core out there that can verify PCI MSI and > legacy are both working at the same time? > > Lucas is the expert here and I believe he has the documentation for > the designware core that Freescale doens't provide with the IMX6 > documentation so hopefully he can provide some insight. He's the one > that has authored all the MSI support and has been using it. > > I typically advise our users to 'not' enable MSI because > architecturally you can spread 4 distinct legacy irq's across CPU's > better than a single shared irq. Don't know if I'm facing similar problem, however devices connected in miniPCI slot behind a PCIe-to-PCI bridge (MSI is disabled) using INTA all is working ok, including shared IRQ. In case of INTB will not work, and the GIC irq quite often get stuck.
[toc] | [prev] | [next] | [standalone]
| From | Roberto Fichera <kernel@tekno-soft.it> |
|---|---|
| Date | 2016-03-29 18:50 +0200 |
| Message-ID | <ri506-wp-9@gated-at.bofh.it> |
| In reply to | #1366446 |
On 03/29/2016 06:40 PM, Tim Harvey wrote:
> On Tue, Mar 29, 2016 at 9:13 AM, Roberto Fichera <kernel@tekno-soft.it> wrote:
>> On 03/29/2016 05:10 PM, Tim Harvey wrote:
>>> Arnd,
>>>
>>> Right, on the IMX the MSI interrupt is GIC-120 which is also the
>>> legacy INTD and I do see that if I happen to put a radio in a slot
>>> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt
>>> does fire and the device works. Any other slot using GIC-123 (INTA),
>>> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible
>>> that something in the designware core is masking out the legacy irqs.
>>> I would also think this was something IMX specific, but I really don't
>>> see any codepaths in pci-imx6.c that would cause that: a driver
>>> requesting a legacy PCI would get a GIC interrupt which is handled by
>>> the IMX6 gpc interrupt controller.
>>>
>>> Any dra7xxx, exynos, spear13xx, keystone, layerscape, hisi, qcom SoC
>>> users of designware PCIe core out there that can verify PCI MSI and
>>> legacy are both working at the same time?
>>>
>>> Lucas is the expert here and I believe he has the documentation for
>>> the designware core that Freescale doens't provide with the IMX6
>>> documentation so hopefully he can provide some insight. He's the one
>>> that has authored all the MSI support and has been using it.
>>>
>>> I typically advise our users to 'not' enable MSI because
>>> architecturally you can spread 4 distinct legacy irq's across CPU's
>>> better than a single shared irq.
>> Don't know if I'm facing similar problem, however devices connected in miniPCI slot behind
>> a PCIe-to-PCI bridge (MSI is disabled) using INTA all is working ok, including shared IRQ.
>> In case of INTB will not work, and the GIC irq quite often get stuck.
>>
> Roberto,
>
> What board/platform is this and what does /proc/interrupts look like?
It's a custom board
root@voneus-janas-imx6q:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 936 637 2057 938 GIC 29 Edge twd
17: 0 0 0 0 GPC 55 Level i.MX Timer Tick
22: 247 0 0 0 GPC 26 Level 2020000.serial
34: 0 0 0 0 gpio-mxc 6 Edge Factory Reset Button
267: 0 0 0 0 GPC 49 Level imx_thermal
272: 0 0 0 0 GPC 19 Level rtc alarm
278: 0 0 0 0 GPC 2 Level sdma
281: 361 0 0 0 GIC 150 Level 2188000.ethernet
282: 0 0 0 0 GIC 151 Level 2188000.ethernet
283: 2882 0 0 0 GPC 25 Level mmc0
284: 95 0 0 0 GPC 37 Level 21a4000.i2c
290: 36546 0 0 0 GPC 123 Level PCIe PME, b4xxp
291: 2 0 0 0 GIC 137 Level 2101000.jr0
292: 0 0 0 0 GIC 138 Level 2102000.jr1
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 0 0 0 0 Timer broadcast interrupts
IPI2: 1642 1038 1626 1781 Rescheduling interrupts
IPI3: 95 95 122 119 Function call interrupts
IPI4: 3 0 2 0 Single function call interrupts
IPI5: 0 0 0 0 CPU stop interrupts
IPI6: 0 0 0 0 IRQ work interrupts
IPI7: 0 0 0 0 completion interrupts
Err: 0
>
> This sounds like what would happen if the downstream interrupts on the
> PCIe-to-PCI bridge are not mapped properly as was the case with a
> board I support (in which case I had to work out a bootloader fixup
> that placed a non-standard interrupt-map in the device-tree for the
> bridge). What bridge are you using?
PCIe-to-PCI bridge is a Ti XIO2001 where we are using INTA/B only wired 1:1
>
> Tim
>
[toc] | [prev] | [next] | [standalone]
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-29 19:40 +0200 |
| Message-ID | <ri5Mu-14H-11@gated-at.bofh.it> |
| In reply to | #1366461 |
On Tue, Mar 29, 2016 at 9:44 AM, Roberto Fichera <kernel@tekno-soft.it> wrote: > On 03/29/2016 06:40 PM, Tim Harvey wrote: >> On Tue, Mar 29, 2016 at 9:13 AM, Roberto Fichera <kernel@tekno-soft.it> wrote: >>> On 03/29/2016 05:10 PM, Tim Harvey wrote: >>>> Arnd, >>>> >>>> Right, on the IMX the MSI interrupt is GIC-120 which is also the >>>> legacy INTD and I do see that if I happen to put a radio in a slot >>>> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt >>>> does fire and the device works. Any other slot using GIC-123 (INTA), >>>> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible >>>> that something in the designware core is masking out the legacy irqs. >>>> I would also think this was something IMX specific, but I really don't >>>> see any codepaths in pci-imx6.c that would cause that: a driver >>>> requesting a legacy PCI would get a GIC interrupt which is handled by >>>> the IMX6 gpc interrupt controller. >>>> >>>> Any dra7xxx, exynos, spear13xx, keystone, layerscape, hisi, qcom SoC >>>> users of designware PCIe core out there that can verify PCI MSI and >>>> legacy are both working at the same time? >>>> >>>> Lucas is the expert here and I believe he has the documentation for >>>> the designware core that Freescale doens't provide with the IMX6 >>>> documentation so hopefully he can provide some insight. He's the one >>>> that has authored all the MSI support and has been using it. >>>> >>>> I typically advise our users to 'not' enable MSI because >>>> architecturally you can spread 4 distinct legacy irq's across CPU's >>>> better than a single shared irq. >>> Don't know if I'm facing similar problem, however devices connected in miniPCI slot behind >>> a PCIe-to-PCI bridge (MSI is disabled) using INTA all is working ok, including shared IRQ. >>> In case of INTB will not work, and the GIC irq quite often get stuck. >>> >> Roberto, >> >> What board/platform is this and what does /proc/interrupts look like? > > It's a custom board > > root@voneus-janas-imx6q:~# cat /proc/interrupts > CPU0 CPU1 CPU2 CPU3 > 16: 936 637 2057 938 GIC 29 Edge twd > 17: 0 0 0 0 GPC 55 Level i.MX Timer Tick > 22: 247 0 0 0 GPC 26 Level 2020000.serial > 34: 0 0 0 0 gpio-mxc 6 Edge Factory Reset Button > 267: 0 0 0 0 GPC 49 Level imx_thermal > 272: 0 0 0 0 GPC 19 Level rtc alarm > 278: 0 0 0 0 GPC 2 Level sdma > 281: 361 0 0 0 GIC 150 Level 2188000.ethernet > 282: 0 0 0 0 GIC 151 Level 2188000.ethernet > 283: 2882 0 0 0 GPC 25 Level mmc0 > 284: 95 0 0 0 GPC 37 Level 21a4000.i2c > 290: 36546 0 0 0 GPC 123 Level PCIe PME, b4xxp > 291: 2 0 0 0 GIC 137 Level 2101000.jr0 > 292: 0 0 0 0 GIC 138 Level 2102000.jr1 > IPI0: 0 0 0 0 CPU wakeup interrupts > IPI1: 0 0 0 0 Timer broadcast interrupts > IPI2: 1642 1038 1626 1781 Rescheduling interrupts > IPI3: 95 95 122 119 Function call interrupts > IPI4: 3 0 2 0 Single function call interrupts > IPI5: 0 0 0 0 CPU stop interrupts > IPI6: 0 0 0 0 IRQ work interrupts > IPI7: 0 0 0 0 completion interrupts > Err: 0 > > >> >> This sounds like what would happen if the downstream interrupts on the >> PCIe-to-PCI bridge are not mapped properly as was the case with a >> board I support (in which case I had to work out a bootloader fixup >> that placed a non-standard interrupt-map in the device-tree for the >> bridge). What bridge are you using? > > PCIe-to-PCI bridge is a Ti XIO2001 where we are using INTA/B only wired 1:1 > Roberto, That's right, we've talked about your bridge on IMX community. I don't see anything in your proc/interrupts other than GPC 123 - you probably only had one device populated when you did that. Put devices in all for slots then show me 'cat /proc/interrupts' as well as 'lspci -vv' (so that I can see what interrupt was given to pin1 and what interrupt that maps to on the IMX6). Check your XIO2001 routing and insure the following for proper IRQ mapping: Slot12: IDSEL A28: socket INTA = XIO2001 INTA Slot13: IDSEL A29: socket INTA = XIO2001 INTB Slot14: IDSEL A30: socket INTA = XIO2001 INTC Slot15: IDSEL A31: socket INTA = XIO2001 INTD The relationship between slot number of IDSEL is based on the PCI specification. The XIO2001 int mapping to socket mapping is based on Table 2 in the XIO2001 implementation guide. In my case what the hardware designer flipped the IDSEL mappings above such that slot12's idsel was hooked to A31 (so it was really slot15) etc, which created a non-standard mapping that required what ended up being a very time consuming and difficult to figure out software fixup (to say the least). Tim
[toc] | [prev] | [next] | [standalone]
| From | Roberto Fichera <kernel@tekno-soft.it> |
|---|---|
| Date | 2016-03-30 10:10 +0200 |
| Message-ID | <rijmr-2uB-17@gated-at.bofh.it> |
| In reply to | #1366507 |
On 03/29/2016 07:31 PM, Tim Harvey wrote:
> On Tue, Mar 29, 2016 at 9:44 AM, Roberto Fichera <kernel@tekno-soft.it> wrote:
>>>
>>> Roberto,
>>>
>>> What board/platform is this and what does /proc/interrupts look like?
>> It's a custom board
>>
>> root@voneus-janas-imx6q:~# cat /proc/interrupts
>> CPU0 CPU1 CPU2 CPU3
>> 16: 936 637 2057 938 GIC 29 Edge twd
>> 17: 0 0 0 0 GPC 55 Level i.MX Timer Tick
>> 22: 247 0 0 0 GPC 26 Level 2020000.serial
>> 34: 0 0 0 0 gpio-mxc 6 Edge Factory Reset Button
>> 267: 0 0 0 0 GPC 49 Level imx_thermal
>> 272: 0 0 0 0 GPC 19 Level rtc alarm
>> 278: 0 0 0 0 GPC 2 Level sdma
>> 281: 361 0 0 0 GIC 150 Level 2188000.ethernet
>> 282: 0 0 0 0 GIC 151 Level 2188000.ethernet
>> 283: 2882 0 0 0 GPC 25 Level mmc0
>> 284: 95 0 0 0 GPC 37 Level 21a4000.i2c
>> 290: 36546 0 0 0 GPC 123 Level PCIe PME, b4xxp
>> 291: 2 0 0 0 GIC 137 Level 2101000.jr0
>> 292: 0 0 0 0 GIC 138 Level 2102000.jr1
>> IPI0: 0 0 0 0 CPU wakeup interrupts
>> IPI1: 0 0 0 0 Timer broadcast interrupts
>> IPI2: 1642 1038 1626 1781 Rescheduling interrupts
>> IPI3: 95 95 122 119 Function call interrupts
>> IPI4: 3 0 2 0 Single function call interrupts
>> IPI5: 0 0 0 0 CPU stop interrupts
>> IPI6: 0 0 0 0 IRQ work interrupts
>> IPI7: 0 0 0 0 completion interrupts
>> Err: 0
>>
>>
>>> This sounds like what would happen if the downstream interrupts on the
>>> PCIe-to-PCI bridge are not mapped properly as was the case with a
>>> board I support (in which case I had to work out a bootloader fixup
>>> that placed a non-standard interrupt-map in the device-tree for the
>>> bridge). What bridge are you using?
>> PCIe-to-PCI bridge is a Ti XIO2001 where we are using INTA/B only wired 1:1
>>
> Roberto,
>
> That's right, we've talked about your bridge on IMX community.
>
> I don't see anything in your proc/interrupts other than GPC 123 - you
> probably only had one device populated when you did that.
Yep! That was the case
> Put devices
> in all for slots then show me 'cat /proc/interrupts' as well as 'lspci
> -vv' (so that I can see what interrupt was given to pin1 and what
> interrupt that maps to on the IMX6).
GPC 123 is serving J2 and J1, and looks ok
root@voneus-janas-imx6q:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 7409 25043 2869 71444 GIC 29 Edge twd
17: 0 0 0 0 GPC 55 Level i.MX Timer Tick
22: 526 0 0 0 GPC 26 Level 2020000.serial
34: 0 0 0 0 gpio-mxc 6 Edge Factory Reset Button
267: 0 0 0 0 GPC 49 Level imx_thermal
272: 0 0 0 0 GPC 19 Level rtc alarm
278: 0 0 0 0 GPC 2 Level sdma
281: 8808 0 0 0 GIC 150 Level 2188000.ethernet
282: 0 0 0 0 GIC 151 Level 2188000.ethernet
283: 2529 0 0 0 GPC 25 Level mmc0
284: 95 0 0 0 GPC 37 Level 21a4000.i2c
290: 2391578 0 0 0 GPC 123 Level PCIe PME, b4xxp, b4xxp
291: 2 0 0 0 GIC 137 Level 2101000.jr0
292: 0 0 0 0 GIC 138 Level 2102000.jr1
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 0 0 0 0 Timer broadcast interrupts
IPI2: 1122 3838 2051 9631 Rescheduling interrupts
IPI3: 60 56 48 54 Function call interrupts
IPI4: 2 1 2 3 Single function call interrupts
IPI5: 0 0 0 0 CPU stop interrupts
IPI6: 0 0 0 0 IRQ work interrupts
IPI7: 0 0 0 0 completion interrupts
Err: 0
root@voneus-janas-imx6q:~# lspci -vv -s 02:
02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev 01)
Subsystem: Cologne Chip Designs GmbH HFC-4S [OpenVox B200P / B400P]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 290
Region 0: I/O ports at 1000 [size=8]
Region 1: Memory at 01100000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: wcb4xxp
Kernel modules: wcb4xxp
02:04.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev 01)
Subsystem: Cologne Chip Designs GmbH HFC-4S [OpenVox B200P / B400P]
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 290
Region 0: I/O ports at 1008 [size=8]
Region 1: Memory at 01101000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: wcb4xxp
Kernel modules: wcb4xxp
>
> Check your XIO2001 routing and insure the following for proper IRQ mapping:
> Slot12: IDSEL A28: socket INTA = XIO2001 INTA
> Slot13: IDSEL A29: socket INTA = XIO2001 INTB
> Slot14: IDSEL A30: socket INTA = XIO2001 INTC
> Slot15: IDSEL A31: socket INTA = XIO2001 INTD
After crosschecking with our hardware designer the PCB IRQ mapping is the following:
J2 : IDSEL A16: => Device 0 : socket INTA = XIO2001 INTA
J3 : IDSEL A18: => Device 2 : socket INTA = XIO2001 INTA* **(This should be INTC)*
J11: IDSEL A20: => Device 4 : socket INTA = XIO2001 INTA
The interrupt routing for J3 is wrong. The XIO2001 driver may expect Device 2 to interrupt on INTC - but it will
interrupt on INTA.
>
> The relationship between slot number of IDSEL is based on the PCI
> specification. The XIO2001 int mapping to socket mapping is based on
> Table 2 in the XIO2001 implementation guide. In my case what the
> hardware designer flipped the IDSEL mappings above such that slot12's
> idsel was hooked to A31 (so it was really slot15) etc, which created a
> non-standard mapping that required what ended up being a very time
> consuming and difficult to figure out software fixup (to say the
> least).
Yep! I have read it
>
> Tim
>
[toc] | [prev] | [next] | [standalone]
| From | Arnd Bergmann <arnd@arndb.de> |
|---|---|
| Date | 2016-03-30 12:20 +0200 |
| Message-ID | <rilod-3NH-5@gated-at.bofh.it> |
| In reply to | #1366978 |
On Wednesday 30 March 2016 10:00:33 Roberto Fichera wrote: > > > > Check your XIO2001 routing and insure the following for proper IRQ mapping: > > Slot12: IDSEL A28: socket INTA = XIO2001 INTA > > Slot13: IDSEL A29: socket INTA = XIO2001 INTB > > Slot14: IDSEL A30: socket INTA = XIO2001 INTC > > Slot15: IDSEL A31: socket INTA = XIO2001 INTD > > After crosschecking with our hardware designer the PCB IRQ mapping is the following: > > J2 : IDSEL A16: => Device 0 : socket INTA = XIO2001 INTA > J3 : IDSEL A18: => Device 2 : socket INTA = XIO2001 INTA* **(This should be INTC)* > J11: IDSEL A20: => Device 4 : socket INTA = XIO2001 INTA > > The interrupt routing for J3 is wrong. The XIO2001 driver may expect Device 2 to interrupt on INTC - but it will > interrupt on INTA. What does your interrupt-map property look like then? Note that you have to override both map and map-mask in this case. Arnd
[toc] | [prev] | [next] | [standalone]
| From | Roberto Fichera <kernel@tekno-soft.it> |
|---|---|
| Date | 2016-03-30 15:00 +0200 |
| Message-ID | <rinT4-5vh-9@gated-at.bofh.it> |
| In reply to | #1367069 |
On 03/30/2016 12:10 PM, Arnd Bergmann wrote: > On Wednesday 30 March 2016 10:00:33 Roberto Fichera wrote: >>> Check your XIO2001 routing and insure the following for proper IRQ mapping: >>> Slot12: IDSEL A28: socket INTA = XIO2001 INTA >>> Slot13: IDSEL A29: socket INTA = XIO2001 INTB >>> Slot14: IDSEL A30: socket INTA = XIO2001 INTC >>> Slot15: IDSEL A31: socket INTA = XIO2001 INTD >> After crosschecking with our hardware designer the PCB IRQ mapping is the following: >> >> J2 : IDSEL A16: => Device 0 : socket INTA = XIO2001 INTA >> J3 : IDSEL A18: => Device 2 : socket INTA = XIO2001 INTA* **(This should be INTC)* >> J11: IDSEL A20: => Device 4 : socket INTA = XIO2001 INTA >> >> The interrupt routing for J3 is wrong. The XIO2001 driver may expect Device 2 to interrupt on INTC - but it will >> interrupt on INTA. > What does your interrupt-map property look like then? Unfortunately it seems that J3 slot doesn't work anymore. Inserting a card there, PCIe link will not come up anymore. Likely I broke it. Looking at some spare logs I have, a card inserted in J3 will get another interrupt, was 291 however unfortunately I don't have an usefull lspci -vv output, sorry! Will check in it against another PCB when I can. > Note that you have to override both map and map-mask in this case. Can you please give more details where should I have a look? > > Arnd >
[toc] | [prev] | [next] | [standalone]
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-30 15:40 +0200 |
| Message-ID | <riovM-64K-5@gated-at.bofh.it> |
| In reply to | #1367186 |
On Wed, Mar 30, 2016 at 5:50 AM, Roberto Fichera <kernel@tekno-soft.it> wrote:
> On 03/30/2016 12:10 PM, Arnd Bergmann wrote:
>> On Wednesday 30 March 2016 10:00:33 Roberto Fichera wrote:
>>>> Check your XIO2001 routing and insure the following for proper IRQ mapping:
>>>> Slot12: IDSEL A28: socket INTA = XIO2001 INTA
>>>> Slot13: IDSEL A29: socket INTA = XIO2001 INTB
>>>> Slot14: IDSEL A30: socket INTA = XIO2001 INTC
>>>> Slot15: IDSEL A31: socket INTA = XIO2001 INTD
>>> After crosschecking with our hardware designer the PCB IRQ mapping is the following:
>>>
>>> J2 : IDSEL A16: => Device 0 : socket INTA = XIO2001 INTA
>>> J3 : IDSEL A18: => Device 2 : socket INTA = XIO2001 INTA* **(This should be INTC)*
>>> J11: IDSEL A20: => Device 4 : socket INTA = XIO2001 INTA
>>>
>>> The interrupt routing for J3 is wrong. The XIO2001 driver may expect Device 2 to interrupt on INTC - but it will
>>> interrupt on INTA.
>> What does your interrupt-map property look like then?
>
> Unfortunately it seems that J3 slot doesn't work anymore. Inserting a card there, PCIe link will not come up anymore.
> Likely I broke it. Looking at some spare logs I have, a card inserted in J3 will get another interrupt, was 291 however
> unfortunately I don't have an usefull lspci -vv output, sorry! Will check in it against another PCB when I can.
>
>> Note that you have to override both map and map-mask in this case.
>
> Can you please give more details where should I have a look?
>
>>
>> Arnd
>>
>
Robert,
The interrupt-map / interrupt-map-mask properties are the ones that
dictate irq mapping. In most cases they are defined at the host
controller only and the kernel will assume standard interrupt
swizzling (aka barber pole'ing) through bridges and will rotate
interrupts (swizzle) for each bridge you go through. It is only if the
interrupts are 'not' mapped properly (as in your case, and as was mine
on a specific add-in card) that you need to define interrupt-map /
interrupt-map-mask for the actual bridge with the interrupt mapping
issue. So in other words, you won't have an interrupt-map/mask for
your TI XIO2001 'currently', but you need to add one to describe its
non-standard mapping.
If you look at imx6qdl.dtsi you'll see the interrupt-map/mask for
standard mapping is:
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>,
<0 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH>,
<0 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH>,
<0 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
This is because on the IMX6 INTA=GIC_123, INTB=GIC_122, INTC=GIC_121,
INTD=GIC_120. The interrupt-map-mask is important because it decribes
which bits of the 'unit interrupt specifier' each map pertains to. For
the standard mapping above only the pin is important because this is
the mapping for each slot so only the last three bits of the 'unit
interrupt specifier' which has four cells is specified in the mask
(0x7).
In your case you will need to describe a map that depends not only on
pin but on slot. The first 32bit cell in the 'unit interrupt
specifier' defines the bus/domain/function (BDF) as: bus << 16 | dev
<< 11 | func <<8. This is also the same format for the first cell in
the 'reg' property that describes each PCI device.
If you are saying that your hardware did not swizzle the interrupts
for the various slots then that means the above mapping needs to be
applied to each slot the same. I
You need to nest PCI devices as they appear on the bus, specifying reg
properly. So, in your case you have a XIO2001 bridge hanging off of
the IMX6 root complex. The root complex is at BDF 0:00.0, the XIO2001
is at BDF 1:00.0, and its sockets are at bus=2. So you will need to
add the following to your device-tree (fixing pinctrl and reset-gpio
per your board specifics):
&pcie {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pcie>;
reset-gpio = <&gpio1 29 GPIO_ACTIVE_LOW>;
status = "okay";
/* 0:00.0 root complex */
pcie@0,0,0 {
reg = <0 0 0 0 0>; /* bus=0, dev=0, func=0 */
/* 1:00.0 TI bridge with reversed IRQ mapping */
pcie@1,0,0 {
device_type = "pci";
#address-cells = <3>;
#size-cells = <2>;
reg = <0x010000 0 0 0 0>; /* bus=1, dev=0, func=0 */
#interrupt-cells = <1>;
interrupt-map-mask = <0xfff00 0 0 0x7>; /*
match bus and device as well as pin */
interrupt-map =
/* MAP for INTA/B/C/D in slot 2:00.00 -
standard mapping */
<0x26000 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
<0x26000 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
<0x26000 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
<0x26000 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
/* MAP for INTA/B/C/D in slot 2:02.00 -
wrong mapping */
<0x26800 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
<0x26800 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
<0x26800 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
<0x26800 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
/* MAP for INTA/B/C/D in slot 2:04.00 -
standard mapping */
<0x27000 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
<0x27000 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
<0x27000 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
<0x27000 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
...
};
};
};
You will need to provide a full mapping which means you'll need to
know how INTA/B/C/D are mapped to each slot. MiniPCIe only users
INTA/B but afaik you have to specify all four. I 'think' what you are
elluding to is that the hardware engineer didn't swizzle the slots at
all so all slots are mapped with INTA->INTA, INTB->INTB, INTC->INTC,
INTD->INTD. If this is the case then you just copy the above for all
slots taking care to specify the first cell properly with b<<16 |
d<<11.
You may find http://devicetree.org/Device_Tree_Usage#Advanced_Interrupt_Mapping
to be helpful as well.
Regards,
Tim
[toc] | [prev] | [next] | [standalone]
| From | Roberto Fichera <kernel@tekno-soft.it> |
|---|---|
| Date | 2016-03-30 17:30 +0200 |
| Message-ID | <riqed-7pH-3@gated-at.bofh.it> |
| In reply to | #1367212 |
On 03/30/2016 03:38 PM, Tim Harvey wrote:
> On Wed, Mar 30, 2016 at 5:50 AM, Roberto Fichera <kernel@tekno-soft.it> wrote:
>> On 03/30/2016 12:10 PM, Arnd Bergmann wrote:
>>> On Wednesday 30 March 2016 10:00:33 Roberto Fichera wrote:
>>>>> Check your XIO2001 routing and insure the following for proper IRQ mapping:
>>>>> Slot12: IDSEL A28: socket INTA = XIO2001 INTA
>>>>> Slot13: IDSEL A29: socket INTA = XIO2001 INTB
>>>>> Slot14: IDSEL A30: socket INTA = XIO2001 INTC
>>>>> Slot15: IDSEL A31: socket INTA = XIO2001 INTD
>>>> After crosschecking with our hardware designer the PCB IRQ mapping is the following:
>>>>
>>>> J2 : IDSEL A16: => Device 0 : socket INTA = XIO2001 INTA
>>>> J3 : IDSEL A18: => Device 2 : socket INTA = XIO2001 INTA* **(This should be INTC)*
>>>> J11: IDSEL A20: => Device 4 : socket INTA = XIO2001 INTA
>>>>
>>>> The interrupt routing for J3 is wrong. The XIO2001 driver may expect Device 2 to interrupt on INTC - but it will
>>>> interrupt on INTA.
>>> What does your interrupt-map property look like then?
>> Unfortunately it seems that J3 slot doesn't work anymore. Inserting a card there, PCIe link will not come up anymore.
>> Likely I broke it. Looking at some spare logs I have, a card inserted in J3 will get another interrupt, was 291 however
>> unfortunately I don't have an usefull lspci -vv output, sorry! Will check in it against another PCB when I can.
>>
>>> Note that you have to override both map and map-mask in this case.
>> Can you please give more details where should I have a look?
>>
>>> Arnd
>>>
> Robert,
>
> The interrupt-map / interrupt-map-mask properties are the ones that
> dictate irq mapping. In most cases they are defined at the host
> controller only and the kernel will assume standard interrupt
> swizzling (aka barber pole'ing) through bridges and will rotate
> interrupts (swizzle) for each bridge you go through. It is only if the
> interrupts are 'not' mapped properly (as in your case, and as was mine
> on a specific add-in card) that you need to define interrupt-map /
> interrupt-map-mask for the actual bridge with the interrupt mapping
> issue. So in other words, you won't have an interrupt-map/mask for
> your TI XIO2001 'currently', but you need to add one to describe its
> non-standard mapping.
>
> If you look at imx6qdl.dtsi you'll see the interrupt-map/mask for
> standard mapping is:
>
> interrupt-map-mask = <0 0 0 0x7>;
> interrupt-map = <0 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>,
> <0 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH>,
> <0 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH>,
> <0 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
>
> This is because on the IMX6 INTA=GIC_123, INTB=GIC_122, INTC=GIC_121,
> INTD=GIC_120. The interrupt-map-mask is important because it decribes
> which bits of the 'unit interrupt specifier' each map pertains to. For
> the standard mapping above only the pin is important because this is
> the mapping for each slot so only the last three bits of the 'unit
> interrupt specifier' which has four cells is specified in the mask
> (0x7).
>
> In your case you will need to describe a map that depends not only on
> pin but on slot. The first 32bit cell in the 'unit interrupt
> specifier' defines the bus/domain/function (BDF) as: bus << 16 | dev
> << 11 | func <<8. This is also the same format for the first cell in
> the 'reg' property that describes each PCI device.
>
> If you are saying that your hardware did not swizzle the interrupts
> for the various slots then that means the above mapping needs to be
> applied to each slot the same. I
>
> You need to nest PCI devices as they appear on the bus, specifying reg
> properly. So, in your case you have a XIO2001 bridge hanging off of
> the IMX6 root complex. The root complex is at BDF 0:00.0, the XIO2001
> is at BDF 1:00.0, and its sockets are at bus=2. So you will need to
> add the following to your device-tree (fixing pinctrl and reset-gpio
> per your board specifics):
>
> &pcie {
> pinctrl-names = "default";
> pinctrl-0 = <&pinctrl_pcie>;
> reset-gpio = <&gpio1 29 GPIO_ACTIVE_LOW>;
> status = "okay";
>
> /* 0:00.0 root complex */
> pcie@0,0,0 {
> reg = <0 0 0 0 0>; /* bus=0, dev=0, func=0 */
>
> /* 1:00.0 TI bridge with reversed IRQ mapping */
> pcie@1,0,0 {
> device_type = "pci";
> #address-cells = <3>;
> #size-cells = <2>;
> reg = <0x010000 0 0 0 0>; /* bus=1, dev=0, func=0 */
> #interrupt-cells = <1>;
> interrupt-map-mask = <0xfff00 0 0 0x7>; /*
> match bus and device as well as pin */
> interrupt-map =
> /* MAP for INTA/B/C/D in slot 2:00.00 -
> standard mapping */
> <0x26000 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
> <0x26000 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
> <0x26000 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
> <0x26000 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
> /* MAP for INTA/B/C/D in slot 2:02.00 -
> wrong mapping */
> <0x26800 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
> <0x26800 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
> <0x26800 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
> <0x26800 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
> /* MAP for INTA/B/C/D in slot 2:04.00 -
> standard mapping */
> <0x27000 0 0 1 &gpc GIC_SPI 123 IRQ_TYPE_LEVEL_H
> <0x27000 0 0 2 &gpc GIC_SPI 122 IRQ_TYPE_LEVEL_H
> <0x27000 0 0 3 &gpc GIC_SPI 121 IRQ_TYPE_LEVEL_H
> <0x27000 0 0 4 &gpc GIC_SPI 120 IRQ_TYPE_LEVEL_H
> ...
> };
> };
> };
>
> You will need to provide a full mapping which means you'll need to
> know how INTA/B/C/D are mapped to each slot. MiniPCIe only users
> INTA/B but afaik you have to specify all four. I 'think' what you are
> elluding to is that the hardware engineer didn't swizzle the slots at
> all so all slots are mapped with INTA->INTA, INTB->INTB, INTC->INTC,
> INTD->INTD. If this is the case then you just copy the above for all
> slots taking care to specify the first cell properly with b<<16 |
> d<<11.
>
> You may find http://devicetree.org/Device_Tree_Usage#Advanced_Interrupt_Mapping
> to be helpful as well.
Tim,
thanks for the detailed information. Will have a look soon.
>
> Regards,
>
> Tim
>
[toc] | [prev] | [next] | [standalone]
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-29 18:50 +0200 |
| Message-ID | <ri506-wp-11@gated-at.bofh.it> |
| In reply to | #1366446 |
On Tue, Mar 29, 2016 at 9:13 AM, Roberto Fichera <kernel@tekno-soft.it> wrote: > On 03/29/2016 05:10 PM, Tim Harvey wrote: >> Arnd, >> >> Right, on the IMX the MSI interrupt is GIC-120 which is also the >> legacy INTD and I do see that if I happen to put a radio in a slot >> where due to swizzling its pin1 becomes INTD (GIC-120) the interrupt >> does fire and the device works. Any other slot using GIC-123 (INTA), >> GIC-122 (INTB), or GIC-121 (INTC) never fires so its very possible >> that something in the designware core is masking out the legacy irqs. >> I would also think this was something IMX specific, but I really don't >> see any codepaths in pci-imx6.c that would cause that: a driver >> requesting a legacy PCI would get a GIC interrupt which is handled by >> the IMX6 gpc interrupt controller. >> >> Any dra7xxx, exynos, spear13xx, keystone, layerscape, hisi, qcom SoC >> users of designware PCIe core out there that can verify PCI MSI and >> legacy are both working at the same time? >> >> Lucas is the expert here and I believe he has the documentation for >> the designware core that Freescale doens't provide with the IMX6 >> documentation so hopefully he can provide some insight. He's the one >> that has authored all the MSI support and has been using it. >> >> I typically advise our users to 'not' enable MSI because >> architecturally you can spread 4 distinct legacy irq's across CPU's >> better than a single shared irq. > > Don't know if I'm facing similar problem, however devices connected in miniPCI slot behind > a PCIe-to-PCI bridge (MSI is disabled) using INTA all is working ok, including shared IRQ. > In case of INTB will not work, and the GIC irq quite often get stuck. > Roberto, What board/platform is this and what does /proc/interrupts look like? This sounds like what would happen if the downstream interrupts on the PCIe-to-PCI bridge are not mapped properly as was the case with a board I support (in which case I had to work out a bootloader fixup that placed a non-standard interrupt-map in the device-tree for the bridge). What bridge are you using? Tim
[toc] | [prev] | [next] | [standalone]
| From | khalasa@piap.pl (Krzysztof Hałasa) |
|---|---|
| Date | 2016-03-30 10:20 +0200 |
| Message-ID | <rijw6-2y5-29@gated-at.bofh.it> |
| In reply to | #1366156 |
Lucas Stach <l.stach@pengutronix.de> writes: >> TW6869: PCI 0000:04:00.0, IRQ 336, MMIO 0x1100000 >> TW686x 0000:04:00.0: enabling device (0140 -> 0142) >> > I don't see whee the device even tries to use MSI IRQs. Even if the > infrastructure is enabled it opts to use legacy INTA. It only tries to use normal IRQ. > There is no upstream driver for this chip, so I don't know where to look > to find out if the driver tries to enable MSI. It's been posted on linux-media list... I added pci_enable_msi() to this driver and it didn't help. > Is what you are saying that if you enable MSI support in the kernel, it > breaks legacy IRQs? Precisely. However, MSI doesn't seem to work either. Could be a problem specific to this TW6869 card. Ventana GW5410 has 5 mPCIe slots, and (with MSI enabled in the system) those IRQs (non-MSI) don't work in any slot: 304: 0 0 0 0 PCI-MSI 0 Edge PCIe PME, aerdrv 336: 0 0 0 0 GPC 123 Level TW8689 in J7 slot 337: 0 0 0 0 GPC 122 Level TW8689 in J8, J10, J11 338: 0 0 0 0 GPC 121 Level TW8689 in J6) If I enable MSI on this card (adding pci_enable_msi()): 313: 0 0 0 0 PCI-MSI 9 Edge TW6869 in J7 slot The only way I can get it to work is by disabling MSI (system wide). -- Krzysztof Halasa Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, Poland
[toc] | [prev] | [next] | [standalone]
| From | Tim Harvey <tharvey@gateworks.com> |
|---|---|
| Date | 2016-03-31 18:20 +0200 |
| Message-ID | <riNua-7Ia-3@gated-at.bofh.it> |
| In reply to | #1366992 |
On Wed, Mar 30, 2016 at 1:10 AM, Krzysztof Hałasa <khalasa@piap.pl> wrote: > Lucas Stach <l.stach@pengutronix.de> writes: > >>> TW6869: PCI 0000:04:00.0, IRQ 336, MMIO 0x1100000 >>> TW686x 0000:04:00.0: enabling device (0140 -> 0142) >>> >> I don't see whee the device even tries to use MSI IRQs. Even if the >> infrastructure is enabled it opts to use legacy INTA. > > It only tries to use normal IRQ. > >> There is no upstream driver for this chip, so I don't know where to look >> to find out if the driver tries to enable MSI. > > It's been posted on linux-media list... I added pci_enable_msi() to this > driver and it didn't help. > >> Is what you are saying that if you enable MSI support in the kernel, it >> breaks legacy IRQs? > > Precisely. > > However, MSI doesn't seem to work either. Could be a problem specific to > this TW6869 card. > > Ventana GW5410 has 5 mPCIe slots, and (with MSI enabled in the system) > those IRQs (non-MSI) don't work in any slot: > > 304: 0 0 0 0 PCI-MSI 0 Edge PCIe PME, aerdrv > 336: 0 0 0 0 GPC 123 Level TW8689 in J7 slot > 337: 0 0 0 0 GPC 122 Level TW8689 in J8, J10, J11 > 338: 0 0 0 0 GPC 121 Level TW8689 in J6) > > If I enable MSI on this card (adding pci_enable_msi()): > 313: 0 0 0 0 PCI-MSI 9 Edge TW6869 in J7 slot > > The only way I can get it to work is by disabling MSI (system wide). Krzysztof, I have found that MSI does work on IMX6 through a bridge (as on the GW5410) and not, for devices/drivers that support MSI, but it does break devices/drivers that use legacy irqs as we've discussed. The MSI capable devices/drivers I've been able to show working with MSI enabled are: - Marvell sky2 GigE (present on GW53xx and GW54xx both through a PEX switch) - Ath10k 802.11n radio miniPCIe socket (tested on GW51xx with no switch and GW53xx with switch). So perhaps there is something else going on with the tw8689 device/driver that is causing it to not work with MSI. We have an avc8000 miniPCIe with tw8689 here and can test if you send me your patches that enable tw8689 with msi. Regardless of MSI working in our tests we still disable it because of it breaking legacy irqs and for performance reasons. Regards, Tim
[toc] | [prev] | [next] | [standalone]
| From | Fabio Estevam <festevam@gmail.com> |
|---|---|
| Date | 2016-03-29 16:20 +0200 |
| Message-ID | <ri2EW-7se-9@gated-at.bofh.it> |
| In reply to | #1366026 |
On Tue, Mar 29, 2016 at 5:55 AM, Lucas Stach <l.stach@pengutronix.de> wrote: > I would suspect that most boards specify the reset polarity the wrong > way around. Fixing this without breaking DT stability is hard. OTOH we It is not hard if we just revert the buggy commit. > could just argue that the system description in DT is wrong and needs to > be fixed. Sure, this would be a nice thing to do as well.
[toc] | [prev] | [next] | [standalone]
| From | khalasa@piap.pl (Krzysztof Hałasa) |
|---|---|
| Date | 2016-03-29 07:30 +0200 |
| Message-ID | <rhUo1-1tB-3@gated-at.bofh.it> |
| In reply to | #1365664 |
Tim Harvey <tharvey@gateworks.com> writes: > ok - I'll respond there as I agree with the patch but not the wording > of the commit (It's Gateworks 'Ventana' using IMX6 not Laguna and we > do define the polarity properly as active-low in Ventana dt's). Right, it's Ventana of course (I had been working with Laguna boards recently). > However, there seems to be another regression in 4.5 that's keeping > PCI working for me and I'm still bisecting that (using 802.11n access > points to test). Can you confirm that PCI works in v4.5 on IMX6 boards > with only 5c5fb40de8f14391a1238db05cef88754faf9229 reverted? I will check with my frame buffer and wifi cards. -- Krzysztof Halasa Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, Poland
[toc] | [prev] | [next] | [standalone]
| From | Petr Štetiar <ynezz@true.cz> |
|---|---|
| Date | 2016-03-30 14:20 +0200 |
| Subject | Re: [PATCH] i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity |
| Message-ID | <ringm-5c9-5@gated-at.bofh.it> |
| In reply to | #1364668 |
Krzysztof Hałasa <khalasa@piap.pl> [2016-03-25 14:32:35]: Cześć, > I wonder if all boards (except maybe that Toradex set) use an active-low > PCIe reset and are now broken. Perhaps Toradex uses active-high and thus > works. I'm really puzzled by this :-) With your patch applied I get following on Toradex Apalis modules: DTS: reset-gpio = <&gpio1 28 GPIO_ACTIVE_LOW>; dmesg: imx6q-pcie 1ffc000.pcie: phy link never came up gpio: gpio-28 ( |reset ) out hi pin voltage: 0V DTS: reset-gpio = <&gpio1 28 GPIO_ACTIVE_HIGH>; dmesg: ath9k 0000:01:00.0: enabling device (0140 -> 0142) gpio: gpio-28 ( |reset ) out lo pin voltage: 3V3 So Toradex Apalis is actually active-high? Thanks. -- ynezz
[toc] | [prev] | [next] | [standalone]
| From | Fabio Estevam <festevam@gmail.com> |
|---|---|
| Date | 2016-03-30 14:50 +0200 |
| Message-ID | <rinJp-5qT-27@gated-at.bofh.it> |
| In reply to | #1367157 |
On Wed, Mar 30, 2016 at 9:06 AM, Petr Štetiar <ynezz@true.cz> wrote: > I'm really puzzled by this :-) With your patch applied I get following on > Toradex Apalis modules: > > DTS: reset-gpio = <&gpio1 28 GPIO_ACTIVE_LOW>; > dmesg: imx6q-pcie 1ffc000.pcie: phy link never came up > gpio: gpio-28 ( |reset ) out hi > pin voltage: 0V > > DTS: reset-gpio = <&gpio1 28 GPIO_ACTIVE_HIGH>; > dmesg: ath9k 0000:01:00.0: enabling device (0140 -> 0142) > gpio: gpio-28 ( |reset ) out lo > pin voltage: 3V3 > > So Toradex Apalis is actually active-high? Thanks. Yes, exactly. That's why you need to introduce a new property to handle the active-high case, so that old dtb's are not broken.
[toc] | [prev] | [next] | [standalone]
Page 2 of 3 — ← Prev page 1 [2] 3 Next page →
Back to top | Article view | linux.kernel
csiph-web