Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1201008 > unrolled thread

RE: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings Varun Sethi <Varun.Sethi@freescale.com> - 2015-08-05 19:20 +0200
    Re: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings Mark Rutland <mark.rutland@arm.com> - 2015-08-06 19:40 +0200
      RE: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings Varun Sethi <Varun.Sethi@freescale.com> - 2015-08-08 17:10 +0200

#1201008 — RE: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings

Hi Mark
Thanks for the patch. Please find my comment inline.

Regards
Varun

> -----Original Message-----
> From: iommu-bounces@lists.linux-foundation.org [mailto:iommu-
> bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland
> Sent: Thursday, July 23, 2015 10:23 PM
> To: devicetree@vger.kernel.org
> Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de;
> marc.zyngier@arm.com; will.deacon@arm.com; linux-
> kernel@vger.kernel.org; ddaney@caviumnetworks.com; iommu@lists.linux-
> foundation.org; tirumalesh.chalamarla@caviumnetworks.com;
> laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com;
> treding@nvidia.com; linux-arm-kernel@lists.infradead.org;
> majun258@huawei.com
> Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings
> 
> Currently msi-parent is used by a few bindings to describe the relationship
> between a PCI root complex and a single MSI controller, but this property
> does not have a generic binding document.
> 
> Additionally, msi-parent is insufficient to describe more complex
> relationships between MSI controllers and devices under a root complex,
> where devices may be able to target multiple MSI controllers, or where MSI
> controllers use (non-probeable) sideband information to distinguish devices.
> 
> This patch adds a generic binding for mapping PCI devices to MSI controllers.
> This document covers msi-parent, and a new msi-map property (specific to
> PCI*) which may be used to map devices (identified by their Requester ID) to
> sideband data for each MSI controller that they may target.
> 
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> ---
>  Documentation/devicetree/bindings/pci/pci-msi.txt | 220
> ++++++++++++++++++++++
>  1 file changed, 220 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt
> 
> diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt
> b/Documentation/devicetree/bindings/pci/pci-msi.txt
> new file mode 100644
> index 0000000..9b3cc81
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt
> @@ -0,0 +1,220 @@
> +This document describes the generic device tree binding for describing
> +the relationship between PCI devices and MSI controllers.
> +
> +Each PCI device under a root complex is uniquely identified by its
> +Requester ID (AKA RID). A Requester ID is a triplet of a Bus number,
> +Device number, and Function number.
> +
> +For the purpose of this document, when treated as a numeric value, a
> +RID is formatted such that:
> +
> +* Bits [15:8] are the Bus number.
> +* Bits [7:3] are the Device number.
> +* Bits [2:0] are the Function number.
> +* Any other bits required for padding must be zero.
> +
> +MSIs may be distinguished in part through the use of sideband data
> +accompanying writes. In the case of PCI devices, this sideband data may
> +be derived from the Requester ID. A mechanism is required to associate
> +a device with both the MSI controllers it can address, and the sideband
> +data that will be associated with its writes to those controllers.
> +
> +For generic MSI bindings, see
> +Documentation/devicetree/bindings/interrupt-controller/msi.txt.
> +
> +
> +PCI root complex
> +================
> +
> +Optional properties
> +-------------------
> +
> +- msi-map: Maps a Requester ID to an MSI controller and associated
> +  msi-specifier data. The property is an arbitrary number of tuples of
> +  (rid-base,msi-controller,msi-base,length), where:
[varun] How would we account for hot plug PCI devices and SR-IOV use cases, with the rid base and length?  How do we take in to account for a PCIe bridge, while setting up the requestor ID base and length?

> +
> +  * rid-base is a single cell describing the first RID matched by the entry.
> +
> +  * msi-controller is a single phandle to an MSI controller
> +
> +  * msi-base is an msi-specifier describing the msi-specifier produced for the
> +    first RID matched by the entry.
> +
> +  * length is a single cell describing how many consecutive RIDs are matched
> +    following the rid-base.
> +
> +  Any RID r in the interval [rid-base, rid-base + length) is associated
> + with  the listed msi-controller, with the msi-specifier (r - rid-base + msi-
> base).
> +
> +- msi-map-mask: A mask to be applied to each Requester ID prior to
> +being mapped
> +  to an msi-specifier per the msi-map property.
> +
[varun] Can you please elaborate on a use case, where this would help. 



> +- msi-parent: Describes the MSI parent of the root complex itself.
> +Where
> +  the root complex and MSI controller do not pass sideband data with
> +MSI
> +  writes, this property may be used to describe the MSI controller(s)
> +  used by PCI devices under the root complex, if defined as such in the
> +  binding for the root complex.
> +
> +
> +Example (1)
> +===========
> +
> +/ {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	msi: msi-controller@a {
> +		reg = <0xa 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	pci: pci@f {
> +		reg = <0xf 0x1>;
> +		compatible = "vendor,pcie-root-complex";
> +		device_type = "pci";
> +
> +		/*
> +		 * The sideband data provided to the MSI controller is
> +		 * the RID, identity-mapped.
> +		 */
> +		msi-map = <0x0 &msi_a 0x0 0x10000>,
> +	};
> +};
> +
> +
> +Example (2)
> +===========
> +
> +/ {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	msi: msi-controller@a {
> +		reg = <0xa 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	pci: pci@f {
> +		reg = <0xf 0x1>;
> +		compatible = "vendor,pcie-root-complex";
> +		device_type = "pci";
> +
> +		/*
> +		 * The sideband data provided to the MSI controller is
> +		 * the RID, masked to only the device and function bits.
> +		 */
> +		msi-map = <0x0 &msi_a 0x0 0x100>,
> +		msi-map-mask = <0xff>
> +	};
> +};
> +
> +
> +Example (3)
> +===========
> +
> +/ {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	msi: msi-controller@a {
> +		reg = <0xa 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	pci: pci@f {
> +		reg = <0xf 0x1>;
> +		compatible = "vendor,pcie-root-complex";
> +		device_type = "pci";
> +
> +		/*
> +		 * The sideband data provided to the MSI controller is
> +		 * the RID, but the high bit of the bus number is
> +		 * ignored.
> +		 */
> +		msi-map = <0x0000 &msi 0x0000 0x8000>,
> +			  <0x8000 &msi 0x0000 0x8000>;
> +	};
> +};
> +
> +
> +Example (4)
> +===========
> +
> +/ {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	msi: msi-controller@a {
> +		reg = <0xa 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	pci: pci@f {
> +		reg = <0xf 0x1>;
> +		compatible = "vendor,pcie-root-complex";
> +		device_type = "pci";
> +
> +		/*
> +		 * The sideband data provided to the MSI controller is
> +		 * the RID, but the high bit of the bus number is
> +		 * negated.
> +		 */
> +		msi-map = <0x0000 &msi 0x8000 0x8000>,
> +			  <0x8000 &msi 0x0000 0x8000>;
> +	};
> +};
> +
> +
> +Example (5)
> +===========
> +
> +/ {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +
> +	msi_a: msi-controller@a {
> +		reg = <0xa 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	msi_b: msi-controller@b {
> +		reg = <0xb 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	msi_c: msi-controller@c {
> +		reg = <0xc 0x1>;
> +		compatible = "vendor,some-controller";
> +		msi-controller;
> +		#msi-cells = <1>;
> +	};
> +
> +	pci: pci@c {
> +		reg = <0xf 0x1>;
> +		compatible = "vendor,pcie-root-complex";
> +		device_type = "pci";
> +
> +		/*
> +		 * The sideband data provided to MSI controller a is the
> +		 * RID, but the high bit of the bus number is negated.
> +		 * The sideband data provided to MSI controller b is the
> +		 * RID, identity-mapped.
> +		 * MSI controller c is not addressable.
> +		 */
> +		msi-map = <0x0000 &msi_a 0x8000 0x08000>,
> +			  <0x8000 &msi_a 0x0000 0x08000>,
> +			  <0x0000 &msi_b 0x0000 0x10000>;
> +	};
> +};
> --
> 1.9.1
> 
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1201932

On Wed, Aug 05, 2015 at 05:39:33PM +0100, Varun Sethi wrote:
> Hi Mark
> Thanks for the patch. Please find my comment inline.
> 
> Regards
> Varun
> 
> > -----Original Message-----
> > From: iommu-bounces@lists.linux-foundation.org [mailto:iommu-
> > bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland
> > Sent: Thursday, July 23, 2015 10:23 PM
> > To: devicetree@vger.kernel.org
> > Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de;
> > marc.zyngier@arm.com; will.deacon@arm.com; linux-
> > kernel@vger.kernel.org; ddaney@caviumnetworks.com; iommu@lists.linux-
> > foundation.org; tirumalesh.chalamarla@caviumnetworks.com;
> > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com;
> > treding@nvidia.com; linux-arm-kernel@lists.infradead.org;
> > majun258@huawei.com
> > Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings
> >
> > Currently msi-parent is used by a few bindings to describe the relationship
> > between a PCI root complex and a single MSI controller, but this property
> > does not have a generic binding document.
> >
> > Additionally, msi-parent is insufficient to describe more complex
> > relationships between MSI controllers and devices under a root complex,
> > where devices may be able to target multiple MSI controllers, or where MSI
> > controllers use (non-probeable) sideband information to distinguish devices.
> >
> > This patch adds a generic binding for mapping PCI devices to MSI controllers.
> > This document covers msi-parent, and a new msi-map property (specific to
> > PCI*) which may be used to map devices (identified by their Requester ID) to
> > sideband data for each MSI controller that they may target.
> >
> > Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> > ---
> >  Documentation/devicetree/bindings/pci/pci-msi.txt | 220
> > ++++++++++++++++++++++
> >  1 file changed, 220 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt
> >
> > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt
> > b/Documentation/devicetree/bindings/pci/pci-msi.txt
> > new file mode 100644
> > index 0000000..9b3cc81
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt
> > @@ -0,0 +1,220 @@
> > +This document describes the generic device tree binding for describing
> > +the relationship between PCI devices and MSI controllers.
> > +
> > +Each PCI device under a root complex is uniquely identified by its
> > +Requester ID (AKA RID). A Requester ID is a triplet of a Bus number,
> > +Device number, and Function number.
> > +
> > +For the purpose of this document, when treated as a numeric value, a
> > +RID is formatted such that:
> > +
> > +* Bits [15:8] are the Bus number.
> > +* Bits [7:3] are the Device number.
> > +* Bits [2:0] are the Function number.
> > +* Any other bits required for padding must be zero.
> > +
> > +MSIs may be distinguished in part through the use of sideband data
> > +accompanying writes. In the case of PCI devices, this sideband data may
> > +be derived from the Requester ID. A mechanism is required to associate
> > +a device with both the MSI controllers it can address, and the sideband
> > +data that will be associated with its writes to those controllers.
> > +
> > +For generic MSI bindings, see
> > +Documentation/devicetree/bindings/interrupt-controller/msi.txt.
> > +
> > +
> > +PCI root complex
> > +================
> > +
> > +Optional properties
> > +-------------------
> > +
> > +- msi-map: Maps a Requester ID to an MSI controller and associated
> > +  msi-specifier data. The property is an arbitrary number of tuples of
> > +  (rid-base,msi-controller,msi-base,length), where:
> [varun] How would we account for hot plug PCI devices and SR-IOV use cases, with the rid base and length?

For hotplug, you simply need the mapping from RID to msi-specifier to be
defined in advance in the DT, for the set of RIDs that could possibly
occur.

For SR-IOV, are you asking about ARI? I should update the description of
the RID to describe that for ARI it has the format:

* Bits [15:8] are the Bus number
* Bits [7:0] are the Identifier

Other than that, the handling would be identical to the non-ARI case.

What else am I missing?

> How do we take in to account for a PCIe bridge, while setting up the requestor ID base and length?

I'm not sure I follow the question. I don't see why this is any
different to any other requester ID.

What do you see as being the problem for this case?

> > +
> > +  * rid-base is a single cell describing the first RID matched by the entry.
> > +
> > +  * msi-controller is a single phandle to an MSI controller
> > +
> > +  * msi-base is an msi-specifier describing the msi-specifier produced for the
> > +    first RID matched by the entry.
> > +
> > +  * length is a single cell describing how many consecutive RIDs are matched
> > +    following the rid-base.
> > +
> > +  Any RID r in the interval [rid-base, rid-base + length) is associated
> > + with  the listed msi-controller, with the msi-specifier (r - rid-base + msi-
> > base).
> > +
> > +- msi-map-mask: A mask to be applied to each Requester ID prior to
> > +being mapped
> > +  to an msi-specifier per the msi-map property.
> > +
> [varun] Can you please elaborate on a use case, where this would help.

It may be the case that at the MSI controller's ID space is smaller
than the RID space, and so only a subset of RID bits matter. For
example, it might be the case that only the Bus ID matters.

Using the msi-map-mask allows for a much smaller msi-map for this case,
e.g.

	msi-map-mask = <0xff00>;
	msi-map = <0x0000 &msi 0x00 1>,
		  <0x0100 &msi 0x01 1>,
		  <0x0200 &msi 0x01 1>,
		  <0x0300 &msi 0x01 1>,
		  ...
		  <0xff00 &msi 0xff 1>;

Rather than:

	msi-map = <0x0000 &msi 0x00 1>,
		  <0x0001 &msi 0x00 1>,
		  <0x0002 &msi 0x00 1>,
		  <0x0003 &msi 0x00 1>,
		  ...
		  <0x00ff &msi 0x00 1>,

		  <0x0100 &msi 0x00 1>,
		  <0x0101 &msi 0x00 1>,
		  <0x0102 &msi 0x00 1>,
		  <0x0103 &msi 0x00 1>,
		  ...
		  <0x01ff &msi 0x00 1>,
		  ...
		  <0xff00 &msi 0xff 1>,
		  <0xff01 &msi 0xff 1>,
		  <0xff02 &msi 0xff 1>,
		  <0xff03 &msi 0xff 1>,
		  ....
		  <0xffff &msi 0xff 1>;

Or for the case that everything maps to a single ID:

		msi-map-mask = <0x0000>;
		msi-map = <0x0000 &msi 0x0000 1>;

Thanks,
Mark.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1203280

Hi Mark,
Thanks for the response. Please find my comments inline.

Regards
Varun

> -----Original Message-----
> From: Mark Rutland [mailto:mark.rutland@arm.com]
> Sent: Thursday, August 06, 2015 11:09 PM
> To: Sethi Varun-B16395
> Cc: devicetree@vger.kernel.org; Lorenzo Pieralisi; arnd@arndb.de; Marc
> Zyngier; Will Deacon; linux-kernel@vger.kernel.org;
> ddaney@caviumnetworks.com; iommu@lists.linux-foundation.org;
> tirumalesh.chalamarla@caviumnetworks.com;
> laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com;
> treding@nvidia.com; linux-arm-kernel@lists.infradead.org;
> majun258@huawei.com; Yoder Stuart-B08248
> Subject: Re: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings
> 
> On Wed, Aug 05, 2015 at 05:39:33PM +0100, Varun Sethi wrote:
> > Hi Mark
> > Thanks for the patch. Please find my comment inline.
> >
> > Regards
> > Varun
> >
> > > -----Original Message-----
> > > From: iommu-bounces@lists.linux-foundation.org [mailto:iommu-
> > > bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland
> > > Sent: Thursday, July 23, 2015 10:23 PM
> > > To: devicetree@vger.kernel.org
> > > Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de;
> > > marc.zyngier@arm.com; will.deacon@arm.com; linux-
> > > kernel@vger.kernel.org; ddaney@caviumnetworks.com;
> > > iommu@lists.linux- foundation.org;
> > > tirumalesh.chalamarla@caviumnetworks.com;
> > > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com;
> > > treding@nvidia.com; linux-arm-kernel@lists.infradead.org;
> > > majun258@huawei.com
> > > Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings
> > >
> > > Currently msi-parent is used by a few bindings to describe the
> > > relationship between a PCI root complex and a single MSI controller,
> > > but this property does not have a generic binding document.
> > >
> > > Additionally, msi-parent is insufficient to describe more complex
> > > relationships between MSI controllers and devices under a root
> > > complex, where devices may be able to target multiple MSI
> > > controllers, or where MSI controllers use (non-probeable) sideband
> information to distinguish devices.
> > >
> > > This patch adds a generic binding for mapping PCI devices to MSI
> controllers.
> > > This document covers msi-parent, and a new msi-map property
> > > (specific to
> > > PCI*) which may be used to map devices (identified by their
> > > Requester ID) to sideband data for each MSI controller that they may
> target.
> > >
> > > Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> > > ---
> > >  Documentation/devicetree/bindings/pci/pci-msi.txt | 220
> > > ++++++++++++++++++++++
> > >  1 file changed, 220 insertions(+)
> > >  create mode 100644
> > > Documentation/devicetree/bindings/pci/pci-msi.txt
> > >
> > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt
> > > b/Documentation/devicetree/bindings/pci/pci-msi.txt
> > > new file mode 100644
> > > index 0000000..9b3cc81
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt
> > > @@ -0,0 +1,220 @@
> > > +This document describes the generic device tree binding for
> > > +describing the relationship between PCI devices and MSI controllers.
> > > +
> > > +Each PCI device under a root complex is uniquely identified by its
> > > +Requester ID (AKA RID). A Requester ID is a triplet of a Bus
> > > +number, Device number, and Function number.
> > > +
> > > +For the purpose of this document, when treated as a numeric value,
> > > +a RID is formatted such that:
> > > +
> > > +* Bits [15:8] are the Bus number.
> > > +* Bits [7:3] are the Device number.
> > > +* Bits [2:0] are the Function number.
> > > +* Any other bits required for padding must be zero.
> > > +
> > > +MSIs may be distinguished in part through the use of sideband data
> > > +accompanying writes. In the case of PCI devices, this sideband data
> > > +may be derived from the Requester ID. A mechanism is required to
> > > +associate a device with both the MSI controllers it can address,
> > > +and the sideband data that will be associated with its writes to those
> controllers.
> > > +
> > > +For generic MSI bindings, see
> > > +Documentation/devicetree/bindings/interrupt-controller/msi.txt.
> > > +
> > > +
> > > +PCI root complex
> > > +================
> > > +
> > > +Optional properties
> > > +-------------------
> > > +
> > > +- msi-map: Maps a Requester ID to an MSI controller and associated
> > > +  msi-specifier data. The property is an arbitrary number of tuples
> > > +of
> > > +  (rid-base,msi-controller,msi-base,length), where:
> > [varun] How would we account for hot plug PCI devices and SR-IOV use
> cases, with the rid base and length?
> 
> For hotplug, you simply need the mapping from RID to msi-specifier to be
> defined in advance in the DT, for the set of RIDs that could possibly occur.
> 
> For SR-IOV, are you asking about ARI? I should update the description of the
> RID to describe that for ARI it has the format:
> 
> * Bits [15:8] are the Bus number
> * Bits [7:0] are the Identifier
> 
> Other than that, the handling would be identical to the non-ARI case.
> 
> What else am I missing?
> 
> > How do we take in to account for a PCIe bridge, while setting up the
> requestor ID base and length?
> 
> I'm not sure I follow the question. I don't see why this is any different to any
> other requester ID.
> 
> What do you see as being the problem for this case?


[varun]  Would the boot loader be responsible for the  PCI bus probe and setting up of the requestor id information in the device tree. Also, during bus probe the boot loader would understand the topology, and handle cases like non transparent host bridge and non standard devices (e.g. require special handling based on quirks). 

Also, the msi identifier would typically be the stream ID provided by the device.  So, this would be limited by the number of SMRs, right? Considering that the each device can have its specific IOMMU domain, which would also require mapping for MSI access, we may also be restricted by the number of context banks. So, we may be able to support a restricted number of msi identifiers.

We already have the PCI bus probe in Linux. If my assumption about the boot loader doing the probe is correct, why do we want to duplicate the work. Also, can we just provide a list of possible MSI identifiers to Linux for each PCIe controller. The IOMMU driver in the Linux kernel can interface with PCIe controller for setting up "requestor id"->"msi identifier" translation.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web