Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1578706 > unrolled thread

Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host hotplug ports

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host  hotplug ports Bjorn Helgaas <helgaas@kernel.org> - 2017-02-10 20:00 +0100
    Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host  hotplug ports Lukas Wunner <lukas@wunner.de> - 2017-02-12 18:20 +0100
      Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host hotplug ports "Rafael J. Wysocki" <rjw@rjwysocki.net> - 2017-02-13 13:30 +0100

#1578706 — Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host hotplug ports

On Sun, Jan 15, 2017 at 09:03:45PM +0100, Lukas Wunner wrote:
> Hotplug ports generally block their parents from suspending to D3hot as
> otherwise their interrupts couldn't be delivered.

This sounds related to PCIe r3.1, sec 5.3.1.4, which says functions
supporting PME generation from D3 must support it for both D3cold and
D3hot.  I think in PCIe, PMEs mean PME Messages, and the 5.3.1
implementation note says Messages are not affected by the PM state of
virtual bridges.

So that seems to say that hotplug ports *should* be able to deliver
PMEs even while in D3hot.

Maybe you're referring to the hotplug interrupts themselves, not the
PME?  I assume a hotplug event (presence detect, attention button,
etc) would first cause a PME, then the OS would return the path to D0,
then the hotplug interrupt would be delivered.

> An exception are Thunderbolt host controllers:  They have a separate
> GPIO pin to side-band signal plug events even if the controller is
> powered down or its parent ports are suspended to D3.  They can be told
> apart from Thunderbolt controllers in attached devices by checking if
> they're situated below a non-Thunderbolt device (typically a root port,
> or the downstream port of a PCIe switch in the case of the MacPro6,1).

In PCIe terms, does a "Thunderbolt host controller" look like a
downstream port that supports hotplug?

It seems like the PCIe PME mechanism *should* work pretty much like
this sideband GPIO.  But I might be reading the spec wrong.

Bjorn

[toc] | [next] | [standalone]

#1579254

On Fri, Feb 10, 2017 at 12:39:12PM -0600, Bjorn Helgaas wrote:
> On Sun, Jan 15, 2017 at 09:03:45PM +0100, Lukas Wunner wrote:
> > Hotplug ports generally block their parents from suspending to D3hot as
> > otherwise their interrupts couldn't be delivered.
> 
> This sounds related to PCIe r3.1, sec 5.3.1.4, which says functions
> supporting PME generation from D3 must support it for both D3cold and
> D3hot.  I think in PCIe, PMEs mean PME Messages, and the 5.3.1
> implementation note says Messages are not affected by the PM state of
> virtual bridges.
> 
> So that seems to say that hotplug ports *should* be able to deliver
> PMEs even while in D3hot.
> 
> Maybe you're referring to the hotplug interrupts themselves, not the
> PME?  I assume a hotplug event (presence detect, attention button,
> etc) would first cause a PME, then the OS would return the path to D0,
> then the hotplug interrupt would be delivered.
> 
> > An exception are Thunderbolt host controllers:  They have a separate
> > GPIO pin to side-band signal plug events even if the controller is
> > powered down or its parent ports are suspended to D3.  They can be told
> > apart from Thunderbolt controllers in attached devices by checking if
> > they're situated below a non-Thunderbolt device (typically a root port,
> > or the downstream port of a PCIe switch in the case of the MacPro6,1).
> 
> In PCIe terms, does a "Thunderbolt host controller" look like a
> downstream port that supports hotplug?
> 
> It seems like the PCIe PME mechanism *should* work pretty much like
> this sideband GPIO.  But I might be reading the spec wrong.

I am dropping this patch in v6 of my Thunderbolt runpm series.

The "Light Ridge" Thunderbolt controller in my machine claims to support
PME, but its WAKE# pin is not connected.  (It's pulled up to 3.3V.)
I also have an external Thunderbolt chassis with the same controller,
and the controller likewise claims to support PME, but its WAKE# pin is
not connected to the PCIe root im my machine in any way.

This led me to the conclusion that PME is not reliable and therefore
in a Thunderbolt daisy chain, which is a cascade of PCIe switches,
only the hotplug port at the farthest end of the chain may suspend to
D3hot, whereas all the ports between it and the root bridge must stay
awake for interrupts to come through.

What I failed to see, due to poor understanding of the (undocumented)
Thunderbolt technology, is that even though ports in the middle of the
chain may be in D3hot and prevent PCIe interrupts from coming through,
the network of Converged I/O switches underlying the PCI tunnels
continues to function and deliver plug events.  It is thus the NHI
driver's job to runtime resume the PCIe switch where a CIO plug event
occurred, as well as its parents, to ensure delivery of PCIe interrupts.
The CIO plug event essentially serves as a sideband signal.  So far the
thunderbolt NHI driver does not support daisy chaining, so this is not
yet an issue.  (Only for tunnels established by the EFI driver.)

Thanks,

Lukas

[toc] | [prev] | [next] | [standalone]

#1579677 — Re: [PATCH v5 3/8] PCI: Don't block runtime PM for Thunderbolt host hotplug ports

On Sunday, February 12, 2017 06:13:01 PM Lukas Wunner wrote:
> On Fri, Feb 10, 2017 at 12:39:12PM -0600, Bjorn Helgaas wrote:
> > On Sun, Jan 15, 2017 at 09:03:45PM +0100, Lukas Wunner wrote:
> > > Hotplug ports generally block their parents from suspending to D3hot as
> > > otherwise their interrupts couldn't be delivered.
> > 
> > This sounds related to PCIe r3.1, sec 5.3.1.4, which says functions
> > supporting PME generation from D3 must support it for both D3cold and
> > D3hot.  I think in PCIe, PMEs mean PME Messages, and the 5.3.1
> > implementation note says Messages are not affected by the PM state of
> > virtual bridges.
> > 
> > So that seems to say that hotplug ports *should* be able to deliver
> > PMEs even while in D3hot.
> > 
> > Maybe you're referring to the hotplug interrupts themselves, not the
> > PME?  I assume a hotplug event (presence detect, attention button,
> > etc) would first cause a PME, then the OS would return the path to D0,
> > then the hotplug interrupt would be delivered.
> > 
> > > An exception are Thunderbolt host controllers:  They have a separate
> > > GPIO pin to side-band signal plug events even if the controller is
> > > powered down or its parent ports are suspended to D3.  They can be told
> > > apart from Thunderbolt controllers in attached devices by checking if
> > > they're situated below a non-Thunderbolt device (typically a root port,
> > > or the downstream port of a PCIe switch in the case of the MacPro6,1).
> > 
> > In PCIe terms, does a "Thunderbolt host controller" look like a
> > downstream port that supports hotplug?
> > 
> > It seems like the PCIe PME mechanism *should* work pretty much like
> > this sideband GPIO.  But I might be reading the spec wrong.
> 
> I am dropping this patch in v6 of my Thunderbolt runpm series.
> 
> The "Light Ridge" Thunderbolt controller in my machine claims to support
> PME, but its WAKE# pin is not connected.  (It's pulled up to 3.3V.)
> I also have an external Thunderbolt chassis with the same controller,
> and the controller likewise claims to support PME, but its WAKE# pin is
> not connected to the PCIe root im my machine in any way.

WAKE# should not be necessary if PME messages can be delivered in-band.

Of course, root ports still need to be able to signal PME wakeup via port
interrupts for this to work, and some kind of side-band wakeup signaling
may be necessary for waking up the system from sleep states via PME.

Thanks,
Rafael

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web