Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #80739

Bug#1015871: Enabling PCI_P2PDMA for distro kernels?

Cross-posted to 2 groups.

Show all headers | View raw

On 2023-10-25 11:35, Bjorn Helgaas wrote:
> On Wed, Oct 25, 2023 at 07:11:26PM +0200, Lukas Wunner wrote:
>> On Wed, Oct 25, 2023 at 10:30:07AM -0600, Logan Gunthorpe wrote:
>>> In addition to the above, P2PDMA transfers are only allowed by the
>>> kernel for traffic that flows through certain host bridges that are
>>> known to work. For AMD, all modern CPUs are on this list, but for Intel,
>>> the list is very patchy.
>>
>> This has recently been brought up internally at Intel and nobody could
>> understand why there's a whitelist in the first place.  A long-time PCI
>> architect told me that Intel silicon validation has been testing P2PDMA
>> at least since the Lindenhurst days, i.e. since 2005.
>>
>> What's the reason for the whitelist?  Was there Intel hardware which
>> didn't support it or turned out to be broken?
>>
>> I imagine (but am not certain) that the feature might only be enabled
>> for server SKUs, is that the reason?
> 
> No, the reason is that the PCIe spec doesn't require routing of
> peer-to-peer transactions between Root Ports:
> https://git.kernel.org/linus/0f97da831026
> 
> I think there was a little discussion about adding a firmware
> interface to advertise this capability, but I guess nobody cared
> enough to advance it.

Yes, I remember someone advancing that in the PCI spec, but I don't know
that it got anywhere.

I definitely remember also testing Intel hardware several years ago
where P2PDMA "worked" but the performance was so awful there was no point.

I vaguely remember this not working on non-server machines in the past
(circa 2015). That's why we had to buy a Xeon. Though this was a long
time ago and my memory is fuzzy.

I'd love it if someone from Intel can give us a reasonable check on the
CPU that guarantees P2PDMA will work for everything that passes the
check (like AMD has done.) But in the absence of Intel telling us this
we can't easily make these assumptions.

Logan

Back to linux.debian.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Uwe Kleine-König <u.kleine-koenig@pengutronix.de> - 2023-10-25 08:30 +0200
  Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Logan Gunthorpe <logang@deltatee.com> - 2023-10-25 19:00 +0200
    Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Lukas Wunner <lukas@wunner.de> - 2023-10-25 19:30 +0200
      Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Bjorn Helgaas <helgaas@kernel.org> - 2023-10-25 19:40 +0200
        Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Logan Gunthorpe <logang@deltatee.com> - 2023-10-25 20:10 +0200
      Bug#1015871: Enabling PCI_P2PDMA for distro kernels? Simon Richter <sjr@debian.org> - 2023-10-26 01:50 +0200

csiph-web