Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1589362 > unrolled thread

RE: [PATCH V2] PCI/ASPM: reconfigure ASPM following hotplug for POLICY_DEFAULT

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  RE: [PATCH V2] PCI/ASPM: reconfigure ASPM following hotplug for  POLICY_DEFAULT "Patel, Mayurkumar" <mayurkumar.patel@intel.com> - 2017-02-28 12:10 +0100
    Re: [PATCH V2] PCI/ASPM: reconfigure ASPM following hotplug for  POLICY_DEFAULT Sinan Kaya <okaya@codeaurora.org> - 2017-02-28 16:20 +0100
      RE: [PATCH V2] PCI/ASPM: reconfigure ASPM following hotplug for  POLICY_DEFAULT "Patel, Mayurkumar" <mayurkumar.patel@intel.com> - 2017-02-28 22:50 +0100

#1589362 — RE: [PATCH V2] PCI/ASPM: reconfigure ASPM following hotplug for POLICY_DEFAULT

>
>When the operating system is booted with the default ASPM policy
>(POLICY_DEFAULT), the current code is determining the ASPM policy
>set up by the BIOS by querying the enable/disable states from ASPM
>registers.
>
>In the case of hotplug removal, the ASPM registers get cleared by
>calling the pcie_aspm_exit_link_state() function.
>

I see the same problem. So cherry-picked your change to resolve the problem
on top of Linus mainline.

>An insertion following hotplug removal reads incorrect policy as
>ASPM disabled even though ASPM was enabled during boot.
>
>Adding a flag to the struct pci_dev and saving the power up policy
>in the bridge to be reused during hotplug insertion. Bridge's enable
>counter is used as a switch to determine when to use saved value.
>
>Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
>---
> drivers/pci/pcie/aspm.c | 21 +++++++++++++++++----
> include/linux/pci.h     |  1 +
> 2 files changed, 18 insertions(+), 4 deletions(-)
>
>diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
>index 17ac1dc..5cfcc6d 100644
>--- a/drivers/pci/pcie/aspm.c
>+++ b/drivers/pci/pcie/aspm.c
>@@ -338,8 +338,9 @@ static void pcie_aspm_check_latency(struct pci_dev *endpoint)
> 	}
> }
>
>-static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
>+static void pcie_aspm_cap_init(struct pci_dev *pdev, int blacklist)
> {
>+	struct pcie_link_state *link = pdev->link_state;
> 	struct pci_dev *child, *parent = link->pdev;
> 	struct pci_bus *linkbus = parent->subordinate;
> 	struct aspm_register_info upreg, dwreg;
>@@ -397,8 +398,20 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
> 	link->latency_up.l1 = calc_l1_latency(upreg.latency_encoding_l1);
> 	link->latency_dw.l1 = calc_l1_latency(dwreg.latency_encoding_l1);
>
>-	/* Save default state */
>-	link->aspm_default = link->aspm_enabled;

But, I am finding a problem with this change, if Policy is set to default,
BIOS enables ASPM L1, but pcie_config_aspm_link() disables ASPM L1
due to link->aspm_enabled is different than link->aspm_default while
called from pcie_aspm_init_link_state() during hotplug insertion.


>+	/*
>+	 * Save default state from FW when enabling ASPM for the first time
>+	 * during boot by looking at the calculated link->aspm_enabled bits
>+	 * above and enable_cnt will be zero.
>+	 *
>+	 * If this path is getting called for the second/third time
>+	 * (enable_cnt will be non-zero). Assume that the current state of the
>+	 * ASPM registers may not necessarily match what FW asked us to do as
>+	 * in the case of hotplug insertion/removal.
>+	 */
>+	if (!atomic_read(&pdev->enable_cnt))
>+		pdev->aspm_default = link->aspm_default = link->aspm_enabled;
>+	else
>+		link->aspm_default = pdev->aspm_default;
>
> 	/* Setup initial capable state. Will be updated later */
> 	link->aspm_capable = link->aspm_support;
>@@ -599,7 +612,7 @@ void pcie_aspm_init_link_state(struct pci_dev *pdev)
> 	 * upstream links also because capable state of them can be
> 	 * update through pcie_aspm_cap_init().
> 	 */
>-	pcie_aspm_cap_init(link, blacklist);
>+	pcie_aspm_cap_init(pdev, blacklist);
>
> 	/* Setup initial Clock PM state */
> 	pcie_clkpm_cap_init(link, blacklist);
>diff --git a/include/linux/pci.h b/include/linux/pci.h
>index e2d1a12..521f88c 100644
>--- a/include/linux/pci.h
>+++ b/include/linux/pci.h
>@@ -321,6 +321,7 @@ struct pci_dev {
>
> #ifdef CONFIG_PCIEASPM
> 	struct pcie_link_state	*link_state;	/* ASPM link state */
>+	unsigned int	aspm_default;		/* ASPM policy set by BIOS */
> #endif
>
> 	pci_channel_state_t error_state;	/* current connectivity state */
>--
>1.9.1
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

[toc] | [next] | [standalone]

#1589547

On 2/28/2017 6:03 AM, Patel, Mayurkumar wrote:
>> -	/* Save default state */
>> -	link->aspm_default = link->aspm_enabled;
> But, I am finding a problem with this change, if Policy is set to default,
> BIOS enables ASPM L1, but pcie_config_aspm_link() disables ASPM L1
> due to link->aspm_enabled is different than link->aspm_default while
> called from pcie_aspm_init_link_state() during hotplug insertion.
> 

Thanks for testing. I think it means the enable count logic I put below
is not working.

I was trying to figure out when to use saved values vs. the values in
registers by looking at the enable_cnt.

enable_cnt is 0 during boot on my system.

The enable_cnt should have been non-zero following an insertion following
remove and default and enabled values should have been identical. 

Can you print out the values of pdev->aspm_default, link->aspm_default, 
and link->aspm_enabled in this function along with atomic_read(&pdev->enable_cnt)?

> 
>> +	/*
>> +	 * Save default state from FW when enabling ASPM for the first time
>> +	 * during boot by looking at the calculated link->aspm_enabled bits
>> +	 * above and enable_cnt will be zero.
>> +	 *
>> +	 * If this path is getting called for the second/third time
>> +	 * (enable_cnt will be non-zero). Assume that the current state of the
>> +	 * ASPM registers may not necessarily match what FW asked us to do as
>> +	 * in the case of hotplug insertion/removal.
>> +	 */
>> +	if (!atomic_read(&pdev->enable_cnt))
>> +		pdev->aspm_default = link->aspm_default = link->aspm_enabled;
>> +	else
>> +		link->aspm_default = pdev->aspm_default;


-- 
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.

[toc] | [prev] | [next] | [standalone]

#1589827

>
>On 2/28/2017 6:03 AM, Patel, Mayurkumar wrote:
>>> -	/* Save default state */
>>> -	link->aspm_default = link->aspm_enabled;
>> But, I am finding a problem with this change, if Policy is set to default,
>> BIOS enables ASPM L1, but pcie_config_aspm_link() disables ASPM L1
>> due to link->aspm_enabled is different than link->aspm_default while
>> called from pcie_aspm_init_link_state() during hotplug insertion.
>>
>
>Thanks for testing. I think it means the enable count logic I put below
>is not working.

Yes I believe so.

>
>I was trying to figure out when to use saved values vs. the values in
>registers by looking at the enable_cnt.

>enable_cnt is 0 during boot on my system.

enable_cnt for the root port on my system is set to 1 for "root port" already without saving
any ASPM related settings. 


>
>The enable_cnt should have been non-zero following an insertion following
>remove and default and enabled values should have been identical.
>

pcie_aspm_init_link_state() is called on the "root port" from hotplug routine when I plug in PCIE Endpoint.
And in my case root port "enable_cnt" is already set to 1 because of root port driver probed already. So
link->aspm_default by  default always sets to pdev->aspm_default which is 0 as it was never set to
BIOS values before. I will continue dig in further to see how the "root port" probe gets triggered
and why it does not call pcie_aspm_init_link_state() to store aspm policy before "enable_cnt" becomes 1 for it.

>Can you print out the values of pdev->aspm_default, link->aspm_default,
>and link->aspm_enabled in this function along with atomic_read(&pdev->enable_cnt)?
>
>>

Yes. Also printed link control register(lnkctl) set by BIOS for upstream port and downstream port
Which gets disabled afterwards due to the link->aspm_default is 0. The prints are in pcie_aspm_cap_init()
Function.

[  199.287732] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[  199.287748] pci 0000:03:00.0: reg 0x18: [mem 0x00000000-0x000000ff 64bit]
[  199.287864] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[  199.288000] pci 0000:03:00.0: System wakeup disabled by ACPI
[  199.299868] upreg lnkcap 00000003 lnkctl=00000042
[  199.299996] dwreg lnkcap  00000002 lnkctl=00000142
[  199.300076] pdev_aspm_default: 0x00000000 link_aspm_default: 0x00000000 aspm_enabled: 0x00000004 enable_cnt: 0x00000001 
[  199.300402] pci 0000:03:00.0: BAR 0: assigned [mem 0xaa000000-0xaa000fff 64bit]
[  199.300413] pci 0000:03:00.0: BAR 2: assigned [mem 0xaa001000-0xaa0010ff 64bit]


>>> +	/*
>>> +	 * Save default state from FW when enabling ASPM for the first time
>>> +	 * during boot by looking at the calculated link->aspm_enabled bits
>>> +	 * above and enable_cnt will be zero.
>>> +	 *
>>> +	 * If this path is getting called for the second/third time
>>> +	 * (enable_cnt will be non-zero). Assume that the current state of the
>>> +	 * ASPM registers may not necessarily match what FW asked us to do as
>>> +	 * in the case of hotplug insertion/removal.
>>> +	 */
>>> +	if (!atomic_read(&pdev->enable_cnt))
>>> +		pdev->aspm_default = link->aspm_default = link->aspm_enabled;
>>> +	else
>>> +		link->aspm_default = pdev->aspm_default;
>
>
>--
>Sinan Kaya
>Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
>Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web