Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1346101 > unrolled thread

Re: [PATCH V5 00/15] MMCONFIG refactoring and support for ARM64 PCI hostbridge init based on ACPI

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH V5 00/15] MMCONFIG refactoring and support for ARM64 PCI  hostbridge init based on ACPI Sinan Kaya <okaya@codeaurora.org> - 2016-02-29 20:10 +0100
    Re: [PATCH V5 00/15] MMCONFIG refactoring and support for ARM64 PCI  hostbridge init based on ACPI Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> - 2016-03-03 12:30 +0100
      Re: [PATCH V5 00/15] MMCONFIG refactoring and support for ARM64 PCI  hostbridge init based on ACPI Sinan Kaya <okaya@codeaurora.org> - 2016-03-03 15:30 +0100

#1346101 — Re: [PATCH V5 00/15] MMCONFIG refactoring and support for ARM64 PCI hostbridge init based on ACPI

On 2/16/2016 8:53 AM, Tomasz Nowicki wrote:
> From the functionality point of view this series might be split into the
> following logic parts:
> 1. Make MMCONFIG code arch-agnostic which allows all architectures to collect
>    PCI config regions and used when necessary.
> 2. Move non-arch specific bits to the core code.
> 3. Use MMCONFIG code and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
> 
> Patches has been built on top of 4.5-rc3 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v5)
> 
> NOTE, this patch set depends on Lorenzo's fixes:
> https://patchwork.ozlabs.org/patch/576450/
> which can be found in pci-acpi-v5 branch.
> 
> This has been tested on Cavium ThunderX server, JunoR2, HP RX2660 IA64, x86,
> Hip05, X-Gene and QEMU-aarch64. Any help in reviewing and testing is very appreciated.
> 
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
> 
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
> 

Having tested v4 and v5, I'm seeing some resource assignment problems and address conflicts. 
And problems booting QEMU.

Anybody else seeing the same?



-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

[toc] | [next] | [standalone]

#1349002

[+ Yinghai]

On Mon, Feb 29, 2016 at 02:03:45PM -0500, Sinan Kaya wrote:
> On 2/16/2016 8:53 AM, Tomasz Nowicki wrote:
> > From the functionality point of view this series might be split into the
> > following logic parts:
> > 1. Make MMCONFIG code arch-agnostic which allows all architectures to collect
> >    PCI config regions and used when necessary.
> > 2. Move non-arch specific bits to the core code.
> > 3. Use MMCONFIG code and implement generic ACPI based PCI host controller driver.
> > 4. Enable above driver on ARM64
> > 
> > Patches has been built on top of 4.5-rc3 and can be found here:
> > git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v5)
> > 
> > NOTE, this patch set depends on Lorenzo's fixes:
> > https://patchwork.ozlabs.org/patch/576450/
> > which can be found in pci-acpi-v5 branch.
> > 
> > This has been tested on Cavium ThunderX server, JunoR2, HP RX2660 IA64, x86,
> > Hip05, X-Gene and QEMU-aarch64. Any help in reviewing and testing is very appreciated.
> > 
> > v4 -> v5
> > - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
> >   https://patchwork.ozlabs.org/patch/575525/
> > - rewrite PCI legacy IRQs allocation
> > - squashed two patches 11 and 12 from series v4, fixed bisection issue
> > - changelog improvements
> > - rebased to 4.5-rc3
> > 
> > v3 -> v4
> > - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> > - added Lorenzo's fix patch 19/24
> > - ACPI PCI bus domain number assigning cleanup
> > - changed resource management, we now claim and reassign resources
> > - improvements for applying quirks
> > - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> > - rebased to 4.5-rc1
> > 
> 
> Having tested v4 and v5, I'm seeing some resource assignment problems
> and address conflicts.  And problems booting QEMU.

I asked Tomasz to add resource claiming code in v4 to make sure that,
if FW has left resources in a reasonable set-up, we reuse it as-is.

Now, I was and I am aware this could trigger resource allocation
issues (in particular in relation to bridges apertures sizing),
that can be nonetheless solved by forcing the kernel to reallocate
resources (pci=realloc, that's exactly what's there for, release
the bridge apertures, resize the busses downstream and reassign
the respective hierarchy).

I am not entirely aware of how consistently pci=realloc was used on
x86, what I am aware of is the panoply of pci=* command line parameters
defined for x86 and I would certainly avoid that.

The decision on whether we claim resources before reassigning them
is either dictacted by the boot method (ie ACPI->claim resources by
default) or we should control it via a FW option or a command
line option, PCI standard (PCI FW revision 3.1, 3.5 "Device State
at Firmware/Operating System Handoff) IIUC does not stricly mandate
FW configuring the whole PCI hierarchy (and to be 100% compliant
we should check the device IO/MEM enable bits before claiming, as x86 does
- see pcibios_allocate_dev_resources() in arch/x86/pci/i386.c).

x86 and IA64 claim PCI resources on boot and live with that (well, minus
the gazillions x86 pci= parameters that change the PCI resources assignment
one way or another), comments very welcome in particular on the pci=realloc
option and its usage.

What's certain is, if we do not claim resources by default we will *never*
be able to do it, it will certainly trigger regressions.

Lorenzo

[toc] | [prev] | [next] | [standalone]

#1349211

On 3/3/2016 6:23 AM, Lorenzo Pieralisi wrote:
> x86 and IA64 claim PCI resources on boot and live with that (well, minus
> the gazillions x86 pci= parameters that change the PCI resources assignment
> one way or another), comments very welcome in particular on the pci=realloc
> option and its usage.

I have been working with Linux PCIe over 3 years. I never used pci=realloc argument.

The v5 series minus [PATCH V5 11/15] drivers: pci: add generic code to claim bus resources
is working just fine and is ready to go upstream in my opinion. It passed my internal 
testing with different types of endpoints.

The inclusion of this patch is now requiring everybody to add pci=realloc argument
otherwise the resources assigned by the UEFI BIOS are not working.

I think there is still some work to be done in this patch and is too early to be included
into the series. It is blocking progress of the series which is sitting on review over 1 
year already.

[    0.752916] pci 0000:01:00.0: VF(n) BAR2 space: [mem 0x80360800000-0x8037fffffff 64bit pref] (contains BAR2 for 63 VFs)
[    0.771799] pci 0000:00:00.0: PCI bridge to [bus 01-06]
[    0.777054] pci 0000:00:00.0: root [mem 0x80100100000-0x8013fffffff window] res [mem 0x8013ff00000-0x8013fffffff] nr 14
[    0.787846] pci 0000:00:00.0: pci_claim_bridge_resource:714 1: i:14
[    0.794135] pci 0000:00:00.0: root [mem 0x80300000000-0x8037fffffff window] res [mem 0x80360000000-0x8037fffffff 64bit pref] nr 15
[    0.805881] pci 0000:00:00.0: pci_claim_bridge_resource:714 1: i:15
[    0.812155] pci 0000:01:00.0: root [mem 0x8013ff00000-0x8013fffffff] res [mem 0x8013ff00000-0x8013fffffff 64bit] nr 0
[    0.822773] pci 0000:01:00.0: root [mem 0x80360000000-0x8037fffffff 64bit pref] res [mem 0x80360000000-0x803607fffff 64bit pref] nr 2
[    0.834778] pci 0000:01:00.0: root [mem 0x80360000000-0x8037fffffff 64bit pref] res [mem 0x8037ff00000-0x8037fffffff pref] nr 6
[    0.846265] pci 0000:01:00.0: can't claim BAR 9 [mem 0x80360800000-0x8037fffffff 64bit pref]: address conflict with 0000:01:00.0 [mem 0x8037ff00000-0x8037fffffff pref]
[    0.861237] pci 0000:01:00.0: BAR 9: no space for [mem size 0x1f800000 64bit pref]
[    0.868811] pci 0000:01:00.0: BAR 9: failed to assign [mem size 0x1f800000 64bit pref]


I keep saying this but the type of CPU is not important when it comes to PCIe. Both PCIe and
ACPI are governed by specs. If it is working for x86 and i64; it needs to work for ARM64 as
well. 

Even ARM64 has the luxury to omit the old BIOS behaviors. Most ARM64 systems use tianocore based
UEFI BIOS.

This is pointing to an implementation problem in arm64 adaptation. Need to figure out
what is different.


-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web