Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1360532 > unrolled thread

Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Grygorii Strashko <grygorii.strashko@ti.com> - 2016-03-18 12:20 +0100
    Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Jon Hunter <jonathanh@nvidia.com> - 2016-03-18 13:30 +0100
      Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Grygorii Strashko <grygorii.strashko@ti.com> - 2016-03-18 15:30 +0100
        Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Jon Hunter <jonathanh@nvidia.com> - 2016-03-18 15:50 +0100
          Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Jon Hunter <jonathanh@nvidia.com> - 2016-03-18 16:00 +0100
            Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Grygorii Strashko <grygorii.strashko@ti.com> - 2016-03-18 19:00 +0100
              Re: [PATCH 08/15] genirq: Add runtime power management support for  IRQ chips Jon Hunter <jonathanh@nvidia.com> - 2016-03-21 11:10 +0100

#1360532 — Re: [PATCH 08/15] genirq: Add runtime power management support for IRQ chips

Hi Jon,

On 03/17/2016 04:19 PM, Jon Hunter wrote:
> Some IRQ chips may be located in a power domain outside of the CPU
> subsystem and hence will require device specific runtime power
> management. In order to support such IRQ chips, add a pointer for a
> device structure to the irq_chip structure, and if this pointer is
> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
> configuration, then the pm_runtime_get/put APIs for this chip will be
> called when an IRQ is requested/freed, respectively.
> 
> When entering system suspend and each interrupt is disabled if there is
> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
> power management, print a warning message for each active interrupt that
> has no wake-up set because these interrupts may be unnecessarily keeping
> the IRQ chip enabled during system suspend.
> 
> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> ---
>   include/linux/irq.h    |  5 +++++
>   kernel/irq/chip.c      | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   kernel/irq/internals.h |  1 +
>   kernel/irq/manage.c    | 14 +++++++++++---
>   kernel/irq/pm.c        |  3 +++
>   5 files changed, 72 insertions(+), 3 deletions(-)
> 
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index c4de62348ff2..82f36390048d 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>   /**
>    * struct irq_chip - hardware interrupt chip descriptor
>    *
> + * @parent:		pointer to associated device
>    * @name:		name for /proc/interrupts
>    * @irq_startup:	start up the interrupt (defaults to ->enable if NULL)
>    * @irq_shutdown:	shut down the interrupt (defaults to ->disable if NULL)
> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>    * @flags:		chip specific flags
>    */

[..]

>   
> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
> index cea1de0161f1..ab436119084f 100644
> --- a/kernel/irq/pm.c
> +++ b/kernel/irq/pm.c
> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>   		 * suspend_device_irqs().
>   		 */
>   		return true;
> +	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
> +		pr_warn("irq %d has no wakeup set and has not been freed!\n",
> +			desc->irq_data.irq);

Sry. But I did not get this part of the patch :(

static bool suspend_device_irq(struct irq_desc *desc)
{
	if (!desc->action || irq_desc_is_chained(desc) ||
	    desc->no_suspend_depth) {
		pr_err("skip irq %d\n", irq_desc_get_irq(desc));
		return false;
	}

	if (irqd_is_wakeup_set(&desc->irq_data)) {
		irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
		/*
		 * We return true here to force the caller to issue
		 * synchronize_irq(). We need to make sure that the
		 * IRQD_WAKEUP_ARMED is visible before we return from
		 * suspend_device_irqs().
		 */
		pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
		return true;
	}

^^^^ Here you've added a warning 

	desc->istate |= IRQS_SUSPENDED;
	__disable_irq(desc);

^^^^ Here non wakeup IRQs will be disabled

	pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));

	/*
	 * Hardware which has no wakeup source configuration facility
	 * requires that the non wakeup interrupts are masked at the
	 * chip level. The chip implementation indicates that with
	 * IRQCHIP_MASK_ON_SUSPEND.
	 */
	if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
		mask_irq(desc);
		pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
	}

	return true;
}

As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
is wakeup source, but all other are not. 

Also I'd like to note that:
- it is not expected that any IRQs have to be freed on enter Suspend
- Primary interrupt controller is expected to be suspended from syscore_suspend()
- not Primary interrupt controllers may be Suspended from:
  -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
  GPIO expanders (I2C, SPI ..)
  -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
	dpm_suspend_noirq
	|- suspend_device_irqs()
	|- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
  -- as always, some arches/maches may require hacks in platform code.
  
So, In my opinion, suspend has to be handled by each irqchip driver separately,
most probably at suspend_noirq level [1], because only  irqchip driver  
now sees a full picture and knows if it can suspend or not, and when, and how.
(may require to use pm_runtime_force_suspend/resume()).

I propose do not touch common/generic suspend code now. Any common code can be always
refactored later once there will be real drivers updated to use irqchip RPM
and which will support Suspend.


-- 
regards,
-grygorii

[toc] | [next] | [standalone]

#1360565

On 18/03/16 11:11, Grygorii Strashko wrote:
> Hi Jon,
> 
> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>> Some IRQ chips may be located in a power domain outside of the CPU
>> subsystem and hence will require device specific runtime power
>> management. In order to support such IRQ chips, add a pointer for a
>> device structure to the irq_chip structure, and if this pointer is
>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>> configuration, then the pm_runtime_get/put APIs for this chip will be
>> called when an IRQ is requested/freed, respectively.
>>
>> When entering system suspend and each interrupt is disabled if there is
>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>> power management, print a warning message for each active interrupt that
>> has no wake-up set because these interrupts may be unnecessarily keeping
>> the IRQ chip enabled during system suspend.
>>
>> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
>> ---
>>   include/linux/irq.h    |  5 +++++
>>   kernel/irq/chip.c      | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>   kernel/irq/internals.h |  1 +
>>   kernel/irq/manage.c    | 14 +++++++++++---
>>   kernel/irq/pm.c        |  3 +++
>>   5 files changed, 72 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index c4de62348ff2..82f36390048d 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>   /**
>>    * struct irq_chip - hardware interrupt chip descriptor
>>    *
>> + * @parent:		pointer to associated device
>>    * @name:		name for /proc/interrupts
>>    * @irq_startup:	start up the interrupt (defaults to ->enable if NULL)
>>    * @irq_shutdown:	shut down the interrupt (defaults to ->disable if NULL)
>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>    * @flags:		chip specific flags
>>    */
> 
> [..]
> 
>>   
>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>> index cea1de0161f1..ab436119084f 100644
>> --- a/kernel/irq/pm.c
>> +++ b/kernel/irq/pm.c
>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>>   		 * suspend_device_irqs().
>>   		 */
>>   		return true;
>> +	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>> +		pr_warn("irq %d has no wakeup set and has not been freed!\n",
>> +			desc->irq_data.irq);
> 
> Sry. But I did not get this part of the patch :(
> 
> static bool suspend_device_irq(struct irq_desc *desc)
> {
> 	if (!desc->action || irq_desc_is_chained(desc) ||
> 	    desc->no_suspend_depth) {
> 		pr_err("skip irq %d\n", irq_desc_get_irq(desc));
> 		return false;
> 	}
> 
> 	if (irqd_is_wakeup_set(&desc->irq_data)) {
> 		irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
> 		/*
> 		 * We return true here to force the caller to issue
> 		 * synchronize_irq(). We need to make sure that the
> 		 * IRQD_WAKEUP_ARMED is visible before we return from
> 		 * suspend_device_irqs().
> 		 */
> 		pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
> 		return true;
> 	}
> 
> ^^^^ Here you've added a warning 

Yes, to warn if the IRQ is enabled but not a wake-up source ...

	if (irqd_is_wakeup_set(&desc->irq_data)) {
		...
	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
		...
	}

> 	desc->istate |= IRQS_SUSPENDED;
> 	__disable_irq(desc);
> 
> ^^^^ Here non wakeup IRQs will be disabled

Yes, but this will not turn off the irqchip. It is legitimate for the
chip to be enabled during suspend if an IRQ is enabled as a wakeup.

The purpose of the warning is to report any IRQs that are enabled at
this point, but NOT wake-up sources. These could be unintentionally be
keeping the chip active when it does not need to be.

> 	pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
> 
> 	/*
> 	 * Hardware which has no wakeup source configuration facility
> 	 * requires that the non wakeup interrupts are masked at the
> 	 * chip level. The chip implementation indicates that with
> 	 * IRQCHIP_MASK_ON_SUSPEND.
> 	 */
> 	if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
> 		mask_irq(desc);
> 		pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
> 	}
> 
> 	return true;
> }
> 
> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
> is wakeup source, but all other are not. 

No there should not be. Remember this is an else-if and ONLY if an IRQ
is not a wake-up source AND enabled will you get a warning.

> Also I'd like to note that:
> - it is not expected that any IRQs have to be freed on enter Suspend

True, but surely they should have a wake-up enabled then? If not you are
wasting power unnecessarily.

I realise that this is different to how interrupts for irqchips work
today, but when we discussed this before, the only way to ensure that we
can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
a slightly different mindset for irqchips with PM, that may be we
shouldn't hold references to IRQs forever if we are not using them.

> - Primary interrupt controller is expected to be suspended from syscore_suspend()
> - not Primary interrupt controllers may be Suspended from:
>   -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
>   GPIO expanders (I2C, SPI ..)
>   -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
> 	dpm_suspend_noirq
> 	|- suspend_device_irqs()
> 	|- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
>   -- as always, some arches/maches may require hacks in platform code.
>   
> So, In my opinion, suspend has to be handled by each irqchip driver separately,
> most probably at suspend_noirq level [1], because only  irqchip driver  
> now sees a full picture and knows if it can suspend or not, and when, and how.
> (may require to use pm_runtime_force_suspend/resume()).

I understand what you are saying, but at least in my mind if would be
better if the clients of the IRQ chips using PM freed their interrupts
when entering suspend. Quite possibly I am overlooking a use-case here
or overhead of doing this, but it would avoid every irqchip having to
handle this themselves and having a custom handler.

> I propose do not touch common/generic suspend code now. Any common code can be always
> refactored later once there will be real drivers updated to use irqchip RPM
> and which will support Suspend.

If this is strongly opposed, I would concede to making this a pr_debug()
as I think it could be useful.

Jon

[0] http://marc.info/?l=linux-pm&m=145340595315514&w=2

[toc] | [prev] | [next] | [standalone]

#1360689

On 03/18/2016 02:27 PM, Jon Hunter wrote:
> 
> On 18/03/16 11:11, Grygorii Strashko wrote:
>> Hi Jon,
>>
>> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>>> Some IRQ chips may be located in a power domain outside of the CPU
>>> subsystem and hence will require device specific runtime power
>>> management. In order to support such IRQ chips, add a pointer for a
>>> device structure to the irq_chip structure, and if this pointer is
>>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>>> configuration, then the pm_runtime_get/put APIs for this chip will be
>>> called when an IRQ is requested/freed, respectively.
>>>
>>> When entering system suspend and each interrupt is disabled if there is
>>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>>> power management, print a warning message for each active interrupt that
>>> has no wake-up set because these interrupts may be unnecessarily keeping
>>> the IRQ chip enabled during system suspend.
>>>
>>> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
>>> ---
>>>    include/linux/irq.h    |  5 +++++
>>>    kernel/irq/chip.c      | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>>    kernel/irq/internals.h |  1 +
>>>    kernel/irq/manage.c    | 14 +++++++++++---
>>>    kernel/irq/pm.c        |  3 +++
>>>    5 files changed, 72 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>>> index c4de62348ff2..82f36390048d 100644
>>> --- a/include/linux/irq.h
>>> +++ b/include/linux/irq.h
>>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>    /**
>>>     * struct irq_chip - hardware interrupt chip descriptor
>>>     *
>>> + * @parent:		pointer to associated device
>>>     * @name:		name for /proc/interrupts
>>>     * @irq_startup:	start up the interrupt (defaults to ->enable if NULL)
>>>     * @irq_shutdown:	shut down the interrupt (defaults to ->disable if NULL)
>>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>     * @flags:		chip specific flags
>>>     */
>>
>> [..]
>>
>>>    
>>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>>> index cea1de0161f1..ab436119084f 100644
>>> --- a/kernel/irq/pm.c
>>> +++ b/kernel/irq/pm.c
>>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>>>    		 * suspend_device_irqs().
>>>    		 */
>>>    		return true;
>>> +	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>>> +		pr_warn("irq %d has no wakeup set and has not been freed!\n",
>>> +			desc->irq_data.irq);
>>
>> Sry. But I did not get this part of the patch :(
>>
>> static bool suspend_device_irq(struct irq_desc *desc)
>> {
>> 	if (!desc->action || irq_desc_is_chained(desc) ||
>> 	    desc->no_suspend_depth) {
>> 		pr_err("skip irq %d\n", irq_desc_get_irq(desc));
>> 		return false;
>> 	}
>>
>> 	if (irqd_is_wakeup_set(&desc->irq_data)) {
>> 		irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
>> 		/*
>> 		 * We return true here to force the caller to issue
>> 		 * synchronize_irq(). We need to make sure that the
>> 		 * IRQD_WAKEUP_ARMED is visible before we return from
>> 		 * suspend_device_irqs().
>> 		 */
>> 		pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
>> 		return true;
>> 	}
>>
>> ^^^^ Here you've added a warning
> 
> Yes, to warn if the IRQ is enabled but not a wake-up source ...
> 
> 	if (irqd_is_wakeup_set(&desc->irq_data)) {
> 		...
> 	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
> 		...
> 	}
> 
>> 	desc->istate |= IRQS_SUSPENDED;
>> 	__disable_irq(desc);
>>
>> ^^^^ Here non wakeup IRQs will be disabled
> 
> Yes, but this will not turn off the irqchip. It is legitimate for the
> chip to be enabled during suspend if an IRQ is enabled as a wakeup.
> 
> The purpose of the warning is to report any IRQs that are enabled at
> this point, but NOT wake-up sources. These could be unintentionally be
> keeping the chip active when it does not need to be.
> 
>> 	pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>
>> 	/*
>> 	 * Hardware which has no wakeup source configuration facility
>> 	 * requires that the non wakeup interrupts are masked at the
>> 	 * chip level. The chip implementation indicates that with
>> 	 * IRQCHIP_MASK_ON_SUSPEND.
>> 	 */
>> 	if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
>> 		mask_irq(desc);
>> 		pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>> 	}
>>
>> 	return true;
>> }
>>
>> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
>> is wakeup source, but all other are not.
> 
> No there should not be. Remember this is an else-if and ONLY if an IRQ
> is not a wake-up source AND enabled will you get a warning.

Sorry, but I don't see the "AND enabled" check any where in this file and
disabled irqs can be wakeup source - they shouldn't be masked.
But I'll stop commenting until i reproduce it.

Or do you mean free? 

> 
>> Also I'd like to note that:
>> - it is not expected that any IRQs have to be freed on enter Suspend
> 
> True, but surely they should have a wake-up enabled then? If not you are
> wasting power unnecessarily.
> 
> I realise that this is different to how interrupts for irqchips work
> today, but when we discussed this before, the only way to ensure that we
> can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
> a slightly different mindset for irqchips with PM, that may be we
> shouldn't hold references to IRQs forever if we are not using them.
> 
>> - Primary interrupt controller is expected to be suspended from syscore_suspend()
>> - not Primary interrupt controllers may be Suspended from:
>>    -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
>>    GPIO expanders (I2C, SPI ..)
>>    -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
>> 	dpm_suspend_noirq
>> 	|- suspend_device_irqs()
>> 	|- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
>>    -- as always, some arches/maches may require hacks in platform code.
>>    
>> So, In my opinion, suspend has to be handled by each irqchip driver separately,
>> most probably at suspend_noirq level [1], because only  irqchip driver
>> now sees a full picture and knows if it can suspend or not, and when, and how.
>> (may require to use pm_runtime_force_suspend/resume()).
> 
> I understand what you are saying, but at least in my mind if would be
> better if the clients of the IRQ chips using PM freed their interrupts
> when entering suspend. Quite possibly I am overlooking a use-case here
> or overhead of doing this, 

ok. seems you do mean "free".

oh :( That will require updating of all drivers (and if it will be taken into account that
wakeup can be configured from sysfs + devm_ - it will be painful).

> but it would avoid every irqchip having to
> handle this themselves and having a custom handler.

irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
to be Powered off during Suspend or deep CPUIdle states, simply because its state
in suspend is unknown - PM state managed automatically (and depends on many factors)
and wakeup can be handled by special HW in case if GPIO bank was really switched off.

>> I propose do not touch common/generic suspend code now. Any common code can be always
>> refactored later once there will be real drivers updated to use irqchip RPM
>> and which will support Suspend.
> 
> If this is strongly opposed, I would concede to making this a pr_debug()
> as I think it could be useful.

Probably yes, because most of the drivers now and IRQ PM core are not ready
for this approach. 

-- 
regards,
-grygorii

[toc] | [prev] | [next] | [standalone]

#1360708

On 18/03/16 14:23, Grygorii Strashko wrote:
> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>
>> On 18/03/16 11:11, Grygorii Strashko wrote:
>>> Hi Jon,
>>>
>>> On 03/17/2016 04:19 PM, Jon Hunter wrote:
>>>> Some IRQ chips may be located in a power domain outside of the CPU
>>>> subsystem and hence will require device specific runtime power
>>>> management. In order to support such IRQ chips, add a pointer for a
>>>> device structure to the irq_chip structure, and if this pointer is
>>>> populated by the IRQ chip driver and CONFIG_PM is selected in the kernel
>>>> configuration, then the pm_runtime_get/put APIs for this chip will be
>>>> called when an IRQ is requested/freed, respectively.
>>>>
>>>> When entering system suspend and each interrupt is disabled if there is
>>>> no wake-up set for that interrupt. For an IRQ chip that utilises runtime
>>>> power management, print a warning message for each active interrupt that
>>>> has no wake-up set because these interrupts may be unnecessarily keeping
>>>> the IRQ chip enabled during system suspend.
>>>>
>>>> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
>>>> ---
>>>>    include/linux/irq.h    |  5 +++++
>>>>    kernel/irq/chip.c      | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>    kernel/irq/internals.h |  1 +
>>>>    kernel/irq/manage.c    | 14 +++++++++++---
>>>>    kernel/irq/pm.c        |  3 +++
>>>>    5 files changed, 72 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>>>> index c4de62348ff2..82f36390048d 100644
>>>> --- a/include/linux/irq.h
>>>> +++ b/include/linux/irq.h
>>>> @@ -315,6 +315,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>>    /**
>>>>     * struct irq_chip - hardware interrupt chip descriptor
>>>>     *
>>>> + * @parent:		pointer to associated device
>>>>     * @name:		name for /proc/interrupts
>>>>     * @irq_startup:	start up the interrupt (defaults to ->enable if NULL)
>>>>     * @irq_shutdown:	shut down the interrupt (defaults to ->disable if NULL)
>>>> @@ -354,6 +355,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>>>>     * @flags:		chip specific flags
>>>>     */
>>>
>>> [..]
>>>
>>>>    
>>>> diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
>>>> index cea1de0161f1..ab436119084f 100644
>>>> --- a/kernel/irq/pm.c
>>>> +++ b/kernel/irq/pm.c
>>>> @@ -83,6 +83,9 @@ static bool suspend_device_irq(struct irq_desc *desc)
>>>>    		 * suspend_device_irqs().
>>>>    		 */
>>>>    		return true;
>>>> +	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>>>> +		pr_warn("irq %d has no wakeup set and has not been freed!\n",
>>>> +			desc->irq_data.irq);
>>>
>>> Sry. But I did not get this part of the patch :(
>>>
>>> static bool suspend_device_irq(struct irq_desc *desc)
>>> {
>>> 	if (!desc->action || irq_desc_is_chained(desc) ||
>>> 	    desc->no_suspend_depth) {
>>> 		pr_err("skip irq %d\n", irq_desc_get_irq(desc));
>>> 		return false;
>>> 	}
>>>
>>> 	if (irqd_is_wakeup_set(&desc->irq_data)) {
>>> 		irqd_set(&desc->irq_data, IRQD_WAKEUP_ARMED);
>>> 		/*
>>> 		 * We return true here to force the caller to issue
>>> 		 * synchronize_irq(). We need to make sure that the
>>> 		 * IRQD_WAKEUP_ARMED is visible before we return from
>>> 		 * suspend_device_irqs().
>>> 		 */
>>> 		pr_err("wakeup irq %d\n", irq_desc_get_irq(desc));
>>> 		return true;
>>> 	}
>>>
>>> ^^^^ Here you've added a warning
>>
>> Yes, to warn if the IRQ is enabled but not a wake-up source ...
>>
>> 	if (irqd_is_wakeup_set(&desc->irq_data)) {
>> 		...
>> 	} else if (!irq_chip_pm_suspended(&desc->irq_data)) {
>> 		...
>> 	}
>>
>>> 	desc->istate |= IRQS_SUSPENDED;
>>> 	__disable_irq(desc);
>>>
>>> ^^^^ Here non wakeup IRQs will be disabled
>>
>> Yes, but this will not turn off the irqchip. It is legitimate for the
>> chip to be enabled during suspend if an IRQ is enabled as a wakeup.
>>
>> The purpose of the warning is to report any IRQs that are enabled at
>> this point, but NOT wake-up sources. These could be unintentionally be
>> keeping the chip active when it does not need to be.
>>
>>> 	pr_err("%s __disable_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>>
>>> 	/*
>>> 	 * Hardware which has no wakeup source configuration facility
>>> 	 * requires that the non wakeup interrupts are masked at the
>>> 	 * chip level. The chip implementation indicates that with
>>> 	 * IRQCHIP_MASK_ON_SUSPEND.
>>> 	 */
>>> 	if (irq_desc_get_chip(desc)->flags & IRQCHIP_MASK_ON_SUSPEND) {
>>> 		mask_irq(desc);
>>> 		pr_err("%s mask_irq irq %d\n", irq_desc_get_chip(desc)->name, irq_desc_get_irq(desc));
>>> 	}
>>>
>>> 	return true;
>>> }
>>>
>>> As result, there should be a ton of warnings if one IRQ (for example in GPIO irqchip)
>>> is wakeup source, but all other are not.
>>
>> No there should not be. Remember this is an else-if and ONLY if an IRQ
>> is not a wake-up source AND enabled will you get a warning.
> 
> Sorry, but I don't see the "AND enabled" check any where in this file and
> disabled irqs can be wakeup source - they shouldn't be masked.
> But I'll stop commenting until i reproduce it.
> 
> Or do you mean free? 

Yes, to be correct I mean not a wake-up source AND not freed (requested).

>>
>>> Also I'd like to note that:
>>> - it is not expected that any IRQs have to be freed on enter Suspend
>>
>> True, but surely they should have a wake-up enabled then? If not you are
>> wasting power unnecessarily.
>>
>> I realise that this is different to how interrupts for irqchips work
>> today, but when we discussed this before, the only way to ensure that we
>> can power-down an irqchip with PM is if all IRQs are freed [0]. So it is
>> a slightly different mindset for irqchips with PM, that may be we
>> shouldn't hold references to IRQs forever if we are not using them.
>>
>>> - Primary interrupt controller is expected to be suspended from syscore_suspend()
>>> - not Primary interrupt controllers may be Suspended from:
>>>    -- dpm_suspend() or dpm_suspend_late() - usual case for external interrupt controllers
>>>    GPIO expanders (I2C, SPI ..)
>>>    -- dpm_suspend_noirq() - usual case for SoC peripherals like OMAP GPIO
>>> 	dpm_suspend_noirq
>>> 	|- suspend_device_irqs()
>>> 	|- device_suspend_noirq() [1] <-- OMAP GPIO do suspend here.
>>>    -- as always, some arches/maches may require hacks in platform code.
>>>    
>>> So, In my opinion, suspend has to be handled by each irqchip driver separately,
>>> most probably at suspend_noirq level [1], because only  irqchip driver
>>> now sees a full picture and knows if it can suspend or not, and when, and how.
>>> (may require to use pm_runtime_force_suspend/resume()).
>>
>> I understand what you are saying, but at least in my mind if would be
>> better if the clients of the IRQ chips using PM freed their interrupts
>> when entering suspend. Quite possibly I am overlooking a use-case here
>> or overhead of doing this, 
> 
> ok. seems you do mean "free".
> 
> oh :( That will require updating of all drivers (and if it will be taken into account that
> wakeup can be configured from sysfs + devm_ - it will be painful).

Will it? I know that there are a few gpio chips that have some hacked
ways to get around the PM issue, but I wonder how many drivers this
really impacts. What sysfs entries are you referring too?

>> but it would avoid every irqchip having to
>> handle this themselves and having a custom handler.
> 
> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
> to be Powered off during Suspend or deep CPUIdle states, simply because its state
> in suspend is unknown - PM state managed automatically (and depends on many factors)
> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
> 
>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>> refactored later once there will be real drivers updated to use irqchip RPM
>>> and which will support Suspend.
>>
>> If this is strongly opposed, I would concede to making this a pr_debug()
>> as I think it could be useful.
> 
> Probably yes, because most of the drivers now and IRQ PM core are not ready
> for this approach. 

May be this calls for a new flag to not WARN if non-wakeup IRQs are not
freed when entering suspend.

Cheers
Jon

[toc] | [prev] | [next] | [standalone]

#1360714

On 18/03/16 14:40, Jon Hunter wrote:
> On 18/03/16 14:23, Grygorii Strashko wrote:
>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>
>>> On 18/03/16 11:11, Grygorii Strashko wrote:

[snip]

>> oh :( That will require updating of all drivers (and if it will be taken into account that
>> wakeup can be configured from sysfs + devm_ - it will be painful).
> 
> Will it? I know that there are a few gpio chips that have some hacked
> ways to get around the PM issue, but I wonder how many drivers this
> really impacts. What sysfs entries are you referring too?

Thinking about this some more, yes I guess it would impact all drivers
that use a gpio but don't use it for a wake-up. I could see that could
be a few drivers indeed.

>>> but it would avoid every irqchip having to
>>> handle this themselves and having a custom handler.
>>
>> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
>> to be Powered off during Suspend or deep CPUIdle states, simply because its state
>> in suspend is unknown - PM state managed automatically (and depends on many factors)
>> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
>>
>>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>>> refactored later once there will be real drivers updated to use irqchip RPM
>>>> and which will support Suspend.
>>>
>>> If this is strongly opposed, I would concede to making this a pr_debug()
>>> as I think it could be useful.
>>
>> Probably yes, because most of the drivers now and IRQ PM core are not ready
>> for this approach. 
> 
> May be this calls for a new flag to not WARN if non-wakeup IRQs are not
> freed when entering suspend.

Flag or pr_debug()?

Jon

[toc] | [prev] | [next] | [standalone]

#1360846

On 03/18/2016 04:56 PM, Jon Hunter wrote:
> 
> On 18/03/16 14:40, Jon Hunter wrote:
>> On 18/03/16 14:23, Grygorii Strashko wrote:
>>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>>
>>>> On 18/03/16 11:11, Grygorii Strashko wrote:
> 
> [snip]
> 
>>> oh :( That will require updating of all drivers (and if it will be taken into account that
>>> wakeup can be configured from sysfs + devm_ - it will be painful).
>>
>> Will it? I know that there are a few gpio chips that have some hacked
>> ways to get around the PM issue, but I wonder how many drivers this
>> really impacts. What sysfs entries are you referring too?

echo enabled > /sys/devices/platform/44000000.ocp/48020000.serial/tty/ttyS2/power/wakeup

> 
> Thinking about this some more, yes I guess it would impact all drivers
> that use a gpio but don't use it for a wake-up. I could see that could
> be a few drivers indeed.

yep. I've just tested it
- gpio was requested through sysfs and configured as IRQ
- do suspend

the same is if GPIO is requested as IRQ only and not configured as wakeup source 

[  319.669760] PM: late suspend of devices complete after 0.213 msecs
[  319.671195] irq 191 has no wakeup set and has not been freed!
[  319.673453] PM: noirq suspend of devices complete after 2.258 msecs

this is very minimal configuration - the regular one is at ~30-50 devices
most of them will use IRQ and only ~10% are used as wakeup sources.


> 
>>>> but it would avoid every irqchip having to
>>>> handle this themselves and having a custom handler.
>>>
>>> irqchip like TI OMAP GPIO will need custom handling any way even if it's not expected
>>> to be Powered off during Suspend or deep CPUIdle states, simply because its state
>>> in suspend is unknown - PM state managed automatically (and depends on many factors)
>>> and wakeup can be handled by special HW in case if GPIO bank was really switched off.
>>>
>>>>> I propose do not touch common/generic suspend code now. Any common code can be always
>>>>> refactored later once there will be real drivers updated to use irqchip RPM
>>>>> and which will support Suspend.
>>>>
>>>> If this is strongly opposed, I would concede to making this a pr_debug()
>>>> as I think it could be useful.
>>>
>>> Probably yes, because most of the drivers now and IRQ PM core are not ready
>>> for this approach.
>>
>> May be this calls for a new flag to not WARN if non-wakeup IRQs are not
>> freed when entering suspend.
> 
> Flag or pr_debug()?
> 

Honestly, I don't know how to proceed - minimum is pr_debug.
My personal opinion is still the same - don't touch suspend core code now, within this series.


-- 
regards,
-grygorii

[toc] | [prev] | [next] | [standalone]

#1361670

On 18/03/16 17:52, Grygorii Strashko wrote:
> On 03/18/2016 04:56 PM, Jon Hunter wrote:
>>
>> On 18/03/16 14:40, Jon Hunter wrote:
>>> On 18/03/16 14:23, Grygorii Strashko wrote:
>>>> On 03/18/2016 02:27 PM, Jon Hunter wrote:
>>>>>
>>>>> On 18/03/16 11:11, Grygorii Strashko wrote:
>>
>> [snip]
>>
>>>> oh :( That will require updating of all drivers (and if it will be taken into account that
>>>> wakeup can be configured from sysfs + devm_ - it will be painful).
>>>
>>> Will it? I know that there are a few gpio chips that have some hacked
>>> ways to get around the PM issue, but I wonder how many drivers this
>>> really impacts. What sysfs entries are you referring too?
> 
> echo enabled > /sys/devices/platform/44000000.ocp/48020000.serial/tty/ttyS2/power/wakeup
> 
>>
>> Thinking about this some more, yes I guess it would impact all drivers
>> that use a gpio but don't use it for a wake-up. I could see that could
>> be a few drivers indeed.
> 
> yep. I've just tested it
> - gpio was requested through sysfs and configured as IRQ
> - do suspend
> 
> the same is if GPIO is requested as IRQ only and not configured as wakeup source 
> 
> [  319.669760] PM: late suspend of devices complete after 0.213 msecs
> [  319.671195] irq 191 has no wakeup set and has not been freed!
> [  319.673453] PM: noirq suspend of devices complete after 2.258 msecs
> 
> this is very minimal configuration - the regular one is at ~30-50 devices
> most of them will use IRQ and only ~10% are used as wakeup sources.

Then it is working as intended :-)

However, if this is too verbose for some irqchips, then as I mentioned
we can have a flag to avoid these messages.

Cheers
Jon

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web