Bug#1015871: Please enable CONFIG_PCI_P2PDMA

From	Simon Richter <sjr@debian.org>
Newsgroups	linux.debian.bugs.dist, linux.debian.kernel
Subject	Bug#1015871: Please enable CONFIG_PCI_P2PDMA
Date	2023-10-25 14:00 +0200
Message-ID	<HsKIh-Kfa-1@gated-at.bofh.it> (permalink)
References	(1 earlier) <EM78d-d8EX-5@gated-at.bofh.it> <Hst4J-yZ0-3@gated-at.bofh.it> <HsutP-zK3-1@gated-at.bofh.it> <EM78d-d8EX-5@gated-at.bofh.it> <HsutP-zK3-1@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Cross-posted to 2 groups.

Show all headers | View raw

Hi,

On 10/25/23 03:29, Emanuele Rocca wrote:

>> I hesitate to actually enable it because I don't understand PCI good
>> enough to judge it's a safe choice for the Debian kernel.

There is a kernel API for "return the physical address of a BAR mapping 
on another PCI device, as seen from the point of view of this device."

With the P2PDMA option disabled, that API always returns "no 
peer-to-peer capable path exists", which requires drivers to fall back 
to allocating a buffer.

If the option is enabled, a driver exporting a DMA buffer (such as a 
video capture device or a GPU) can provide an address that is part of 
one of its BAR mappings. The driver importing the DMA buffer will then 
use the physical address it was given in DMA requests, which routes the 
requests directly and avoids passing through the root complex.

I have successfully used this to make an nVidia GPU generate PCIe 
requests for textures that were answered by an FPGA card. :> The driver 
importing the buffer needs no special support.

There is no generic mechanism to use peer-to-peer transfers if not at 
least one side is aware of that mechanism, as it essentially requires 
the exporting device to implement full support for prefetchable BAR 
mappings.

Im principle, all platforms that support PCIe can benefit from this, as 
it allows peer-to-peer transfers even when the root complex does not 
support this, as long as all bridges on the path between the involved 
devices do. In practice, most transfers will happen between a GPU (in a 
slot directly connected to a root bridge) and some data capture device, 
so the benefit on platforms other than amd64 and ppc64le will be limited 
-- although these will just fall back to the current behaviour, and the 
only code paths affected are slow paths that usually end in TLB flushes 
for CPU and IO MMUs.

    Simon

Back to linux.debian.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Bug#1015871: Please enable CONFIG_PCI_P2PDMA Uwe Kleine-König <u.kleine-koenig@pengutronix.de> - 2023-10-24 19:10 +0200
  Bug#1015871: Please enable CONFIG_PCI_P2PDMA Emanuele Rocca <ema@debian.org> - 2023-10-24 20:40 +0200
    Bug#1015871: Please enable CONFIG_PCI_P2PDMA Simon Richter <sjr@debian.org> - 2023-10-25 14:00 +0200

csiph-web