Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1481554 > unrolled thread

Re: [PATCH v2 00/17] Make rpmsg a framework

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v2 00/17] Make rpmsg a framework Lina Iyer <lina.iyer@linaro.org> - 2016-09-12 19:00 +0200
    Re: [PATCH v2 00/17] Make rpmsg a framework Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-09-12 20:10 +0200
      Re: [PATCH v2 00/17] Make rpmsg a framework Jeffrey Hugo <jhugo@codeaurora.org> - 2016-09-12 20:20 +0200
        Re: [PATCH v2 00/17] Make rpmsg a framework Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-09-12 21:00 +0200
          Re: [PATCH v2 00/17] Make rpmsg a framework Jeffrey Hugo <jhugo@codeaurora.org> - 2016-09-12 21:30 +0200
            Re: [PATCH v2 00/17] Make rpmsg a framework Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-09-12 22:00 +0200
              Re: [PATCH v2 00/17] Make rpmsg a framework Karthikeyan Ramasubramanian <kramasub@codeaurora.org> - 2016-09-12 23:40 +0200
                Re: [PATCH v2 00/17] Make rpmsg a framework Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-09-13 00:00 +0200
    Re: [PATCH v2 00/17] Make rpmsg a framework Lina Iyer <lina.iyer@linaro.org> - 2016-09-13 00:30 +0200
      Re: [PATCH v2 00/17] Make rpmsg a framework Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-09-13 01:00 +0200

#1481554 — Re: [PATCH v2 00/17] Make rpmsg a framework

Hi Bjorn,

On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
>This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
>backend/wireformat.
>
>
>As we discussed the Qualcomm SMD implementation a couple of years back people
>suggested that I should make it "a rpmsg thingie". With the introduction of the
>Qualcomm 8996 platform, we must support a variant of the communication
>mechanism that share many of the characteristics of SMD, but are different
>enough that it can't be done in a single implementation. As such there is
>enough benefit to do the necessary work and being able to make SMD a "rpmsg
>thingie".
>
>On-top of this series I have patches to switch the current smd clients over to
>rpmsg (and by that drop the existing SMD implementation).
>
>All this allows me to implement the new backend and reuse all existing SMD
>drivers with the new mechanism.
>

RPM Communication has to supported even when IRQs are disabled. The most
important use of this communication is to set the wake up time for the
CPU subsystem when all the CPUs are powered off. In addition to that,
"sleep" votes that are sent by the application processor subsystem to
allow system to go into deep sleep modes can only be triggered when the
CPU PM domains are power collapsed, drivers do not have a knowledge of
when that happens. This has to be done by a platform code that registers
for CPU PM domain power_off/on callbacks.

Using rpmsg may be nice for RPM SMD communication, but mutexes need to
go away for this driver to be any useful than bare bones active mode
resource requests for QCOM SoCs. By not doing that now, we lock
ourselves out of using this SMD driver in the near future when CPU PM
domains are available in the kernel with an ability to do system low
power modes.

I hope you would make rpmsg work in IRQ disabled contexts first before
porting the SMD driver.

Thanks,
Lina

>
>Changes from v1:
>- Split up the patch moving core code to rpmsg_core into several commits
>- Dropped the wrapping struct in rpmsg_core and just added the ops to the
>  public API (but hid the implementation details)
>- Reordered things to reduce the size of the later patches
>
>Bjorn Andersson (17):
>  rpmsg: Enable matching devices with drivers based on DT
>  rpmsg: Name rpmsg devices based on channel id
>  rpmsg: rpmsg_send() operations takes rpmsg_endpoint
>  rpmsg: Make rpmsg_create_ept() take channel_info struct
>  rpmsg: Clean up rpmsg device vs channel naming
>  rpmsg: Introduce indirection table for rpmsg_device operations
>  rpmsg: Move rpmsg_device API to new file
>  rpmsg: Indirection table for rpmsg_endpoint operations
>  rpmsg: Move endpoint related interface to rpmsg core
>  rpmsg: Move helper for finding rpmsg devices to core
>  rpmsg: Split off generic tail of create_channel()
>  rpmsg: Split rpmsg core and virtio backend
>  rpmsg: Hide rpmsg indirection tables
>  rpmsg: virtio: Hide vrp pointer from the public API
>  rpmsg: Move virtio specifics from public header
>  rpmsg: Allow callback to return errors
>  rpmsg: Introduce Qualcomm SMD backend
>
> drivers/remoteproc/Kconfig          |    4 +-
> drivers/rpmsg/Kconfig               |   14 +
> drivers/rpmsg/Makefile              |    4 +-
> drivers/rpmsg/qcom_smd.c            | 1434 +++++++++++++++++++++++++++++++++++
> drivers/rpmsg/rpmsg_core.c          |  498 ++++++++++++
> drivers/rpmsg/rpmsg_internal.h      |   82 ++
> drivers/rpmsg/virtio_rpmsg_bus.c    |  487 +++++-------
> include/linux/rpmsg.h               |  246 +-----
> samples/rpmsg/rpmsg_client_sample.c |   14 +-
> 9 files changed, 2266 insertions(+), 517 deletions(-)
> create mode 100644 drivers/rpmsg/qcom_smd.c
> create mode 100644 drivers/rpmsg/rpmsg_core.c
> create mode 100644 drivers/rpmsg/rpmsg_internal.h
>
>-- 
>2.5.0
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [next] | [standalone]

#1481780

On Mon 12 Sep 09:52 PDT 2016, Lina Iyer wrote:

> Hi Bjorn,
> 
> On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
> >This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
> >backend/wireformat.
> >
> >
> >As we discussed the Qualcomm SMD implementation a couple of years back people
> >suggested that I should make it "a rpmsg thingie". With the introduction of the
> >Qualcomm 8996 platform, we must support a variant of the communication
> >mechanism that share many of the characteristics of SMD, but are different
> >enough that it can't be done in a single implementation. As such there is
> >enough benefit to do the necessary work and being able to make SMD a "rpmsg
> >thingie".
> >
> >On-top of this series I have patches to switch the current smd clients over to
> >rpmsg (and by that drop the existing SMD implementation).
> >
> >All this allows me to implement the new backend and reuse all existing SMD
> >drivers with the new mechanism.
> >
> 
> RPM Communication has to supported even when IRQs are disabled. The most
> important use of this communication is to set the wake up time for the
> CPU subsystem when all the CPUs are powered off.

Can you point me to the downstream code where this is implemented so I
can have a look? Do you expect to get the response on that request?

> In addition to that,
> "sleep" votes that are sent by the application processor subsystem to
> allow system to go into deep sleep modes can only be triggered when the
> CPU PM domains are power collapsed, drivers do not have a knowledge of
> when that happens.

Do you mean the actual sleep votes can only be with the CPU PM domains
collapsed?

It's been a while since I dug through that code, but there was several
cases where sleep votes would be sent out during normal execution as
well, and then there's the optimization of flushing out all cached sleep
votes when we're on the way down.

> This has to be done by a platform code that registers
> for CPU PM domain power_off/on callbacks.
> 

Ok, sounds like we have a legit use case for improving this.

> Using rpmsg may be nice for RPM SMD communication, but mutexes need to
> go away for this driver to be any useful than bare bones active mode
> resource requests for QCOM SoCs. By not doing that now, we lock
> ourselves out of using this SMD driver in the near future when CPU PM
> domains are available in the kernel with an ability to do system low
> power modes.
> 

The last time I looked at this there where no cases when it was
_required_ to support transmitting requests to the rpm from IRQ context.

iirc we could set up the sleep votes in normal context and the
transition was triggered through SAW(?)

> I hope you would make rpmsg work in IRQ disabled contexts first before
> porting the SMD driver.
> 

There are two parts of this request;

The first is to be able to send data from irq context. The proposed
patches doesn't affect the implementation of send, so it's just a matter
of changing qcom_smd_send() and make the necessary adjustments in the
rpm driver.

In the event of the tx fifo being full we normally do want to sleep on
there being space, but if we switch to spinlocks you would be able to
issue an rpmsg_trysend() which would bypass this - and you can roll a
busy wait in the caller.



The other part is how to receive responses in this mode. Messages are
pulled off the fifo in IRQ context and delivered to the consumer in IRQ
context. But if you have irqs disabled then this wouldn't be triggered.

So if you need your responses we need to figure something out here. And
part of the ugliness of downstream is the need to drain the fifo just
enough before going to sleep, so that the RPM won't stall on a full
fifo.

Regards,
Bjorn

[toc] | [prev] | [next] | [standalone]

#1481791

On 9/12/2016 12:00 PM, Bjorn Andersson wrote:
> On Mon 12 Sep 09:52 PDT 2016, Lina Iyer wrote:
>
>> Hi Bjorn,
>>
>> On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
>>> This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
>>> backend/wireformat.
>>>
>>>
>>> As we discussed the Qualcomm SMD implementation a couple of years back people
>>> suggested that I should make it "a rpmsg thingie". With the introduction of the
>>> Qualcomm 8996 platform, we must support a variant of the communication
>>> mechanism that share many of the characteristics of SMD, but are different
>>> enough that it can't be done in a single implementation. As such there is
>>> enough benefit to do the necessary work and being able to make SMD a "rpmsg
>>> thingie".
>>>
>>> On-top of this series I have patches to switch the current smd clients over to
>>> rpmsg (and by that drop the existing SMD implementation).
>>>
>>> All this allows me to implement the new backend and reuse all existing SMD
>>> drivers with the new mechanism.
>>>
>>
>> RPM Communication has to supported even when IRQs are disabled. The most
>> important use of this communication is to set the wake up time for the
>> CPU subsystem when all the CPUs are powered off.
>
> Can you point me to the downstream code where this is implemented so I
> can have a look? Do you expect to get the response on that request?

Have a look at -
smd_mask_receive_interrupt()
smd_is_pkt_avail()

Every request to the RPM generates a response.  The Linux RPM driver may 
decide to let the response sit in the fifo, or it may need to read and 
process it.

>
>> In addition to that,
>> "sleep" votes that are sent by the application processor subsystem to
>> allow system to go into deep sleep modes can only be triggered when the
>> CPU PM domains are power collapsed, drivers do not have a knowledge of
>> when that happens.
>
> Do you mean the actual sleep votes can only be with the CPU PM domains
> collapsed?
>
> It's been a while since I dug through that code, but there was several
> cases where sleep votes would be sent out during normal execution as
> well, and then there's the optimization of flushing out all cached sleep
> votes when we're on the way down.
>
>> This has to be done by a platform code that registers
>> for CPU PM domain power_off/on callbacks.
>>
>
> Ok, sounds like we have a legit use case for improving this.
>
>> Using rpmsg may be nice for RPM SMD communication, but mutexes need to
>> go away for this driver to be any useful than bare bones active mode
>> resource requests for QCOM SoCs. By not doing that now, we lock
>> ourselves out of using this SMD driver in the near future when CPU PM
>> domains are available in the kernel with an ability to do system low
>> power modes.
>>
>
> The last time I looked at this there where no cases when it was
> _required_ to support transmitting requests to the rpm from IRQ context.

I no longer work on SMD, but when I did this was in fact a strict 
requirement.  If I recall correctly, there was a parameter in the RPM 
driver for the transmit function that indicated if the request was being 
made in atomic context or not, which would change the behavior of how 
the transmit was handled.

>
> iirc we could set up the sleep votes in normal context and the
> transition was triggered through SAW(?)
>
>> I hope you would make rpmsg work in IRQ disabled contexts first before
>> porting the SMD driver.
>>
>
> There are two parts of this request;
>
> The first is to be able to send data from irq context. The proposed
> patches doesn't affect the implementation of send, so it's just a matter
> of changing qcom_smd_send() and make the necessary adjustments in the
> rpm driver.
>
> In the event of the tx fifo being full we normally do want to sleep on
> there being space, but if we switch to spinlocks you would be able to
> issue an rpmsg_trysend() which would bypass this - and you can roll a
> busy wait in the caller.
>
>
>
> The other part is how to receive responses in this mode. Messages are
> pulled off the fifo in IRQ context and delivered to the consumer in IRQ
> context. But if you have irqs disabled then this wouldn't be triggered.
>
> So if you need your responses we need to figure something out here. And
> part of the ugliness of downstream is the need to drain the fifo just
> enough before going to sleep, so that the RPM won't stall on a full
> fifo.
>
> Regards,
> Bjorn
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Jeffrey Hugo
Qualcomm Datacenter Technologies as an affiliate of Qualcomm 
Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

[toc] | [prev] | [next] | [standalone]

#1481851

On Mon 12 Sep 11:13 PDT 2016, Jeffrey Hugo wrote:

> On 9/12/2016 12:00 PM, Bjorn Andersson wrote:
> >On Mon 12 Sep 09:52 PDT 2016, Lina Iyer wrote:
> >
> >>Hi Bjorn,
> >>
> >>On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
> >>>This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
> >>>backend/wireformat.
> >>>
> >>>
> >>>As we discussed the Qualcomm SMD implementation a couple of years back people
> >>>suggested that I should make it "a rpmsg thingie". With the introduction of the
> >>>Qualcomm 8996 platform, we must support a variant of the communication
> >>>mechanism that share many of the characteristics of SMD, but are different
> >>>enough that it can't be done in a single implementation. As such there is
> >>>enough benefit to do the necessary work and being able to make SMD a "rpmsg
> >>>thingie".
> >>>
> >>>On-top of this series I have patches to switch the current smd clients over to
> >>>rpmsg (and by that drop the existing SMD implementation).
> >>>
> >>>All this allows me to implement the new backend and reuse all existing SMD
> >>>drivers with the new mechanism.
> >>>
> >>
> >>RPM Communication has to supported even when IRQs are disabled. The most
> >>important use of this communication is to set the wake up time for the
> >>CPU subsystem when all the CPUs are powered off.
> >
> >Can you point me to the downstream code where this is implemented so I
> >can have a look? Do you expect to get the response on that request?
> 
> Have a look at -
> smd_mask_receive_interrupt()
> smd_is_pkt_avail()
> 

In msm-3.18 these still seems to only come from either
msm_rpm_enter_sleep() and the rpm-clock driver, related to flushing
cached sleep state requests.

> Every request to the RPM generates a response.  The Linux RPM driver may
> decide to let the response sit in the fifo, or it may need to read and
> process it.
> 

Right, I presume we save some time by not waiting for these responses as
we want to reach sleep as soon as possible. The answer I got last time
this was discussed was that it was an optimization, not a functional
requirement.


I'm not at all against having the rpm driver cache the state
information and the smd driver process read/writes from the rpm driver
in IRQ context. I do however not know how to trigger the flush in a sane
way.

> >
> >>In addition to that,
> >>"sleep" votes that are sent by the application processor subsystem to
> >>allow system to go into deep sleep modes can only be triggered when the
> >>CPU PM domains are power collapsed, drivers do not have a knowledge of
> >>when that happens.
> >
> >Do you mean the actual sleep votes can only be with the CPU PM domains
> >collapsed?
> >
> >It's been a while since I dug through that code, but there was several
> >cases where sleep votes would be sent out during normal execution as
> >well, and then there's the optimization of flushing out all cached sleep
> >votes when we're on the way down.
> >
> >>This has to be done by a platform code that registers
> >>for CPU PM domain power_off/on callbacks.
> >>
> >
> >Ok, sounds like we have a legit use case for improving this.
> >
> >>Using rpmsg may be nice for RPM SMD communication, but mutexes need to
> >>go away for this driver to be any useful than bare bones active mode
> >>resource requests for QCOM SoCs. By not doing that now, we lock
> >>ourselves out of using this SMD driver in the near future when CPU PM
> >>domains are available in the kernel with an ability to do system low
> >>power modes.
> >>
> >
> >The last time I looked at this there where no cases when it was
> >_required_ to support transmitting requests to the rpm from IRQ context.
> 
> I no longer work on SMD, but when I did this was in fact a strict
> requirement.

When I dissected all the users of the API I came to the conclusion that
this requirement (on the SMD driver) came from above mentioned
optimization.

> If I recall correctly, there was a parameter in the RPM driver
> for the transmit function that indicated if the request was being made in
> atomic context or not, which would change the behavior of how the transmit
> was handled.
> 

You're correct, the question is still which of these code paths are
actually needed and to motivate the endless maintenance of the extra
code.


Nice to see you on the mailing list again Jeff.

Regards,
Bjorn

[toc] | [prev] | [next] | [standalone]

#1481896

On 9/12/2016 12:49 PM, Bjorn Andersson wrote:
> On Mon 12 Sep 11:13 PDT 2016, Jeffrey Hugo wrote:
>
>> On 9/12/2016 12:00 PM, Bjorn Andersson wrote:
>>> On Mon 12 Sep 09:52 PDT 2016, Lina Iyer wrote:
>>>
>>>> Hi Bjorn,
>>>>
>>>> On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
>>>>> This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
>>>>> backend/wireformat.
>>>>>
>>>>>
>>>>> As we discussed the Qualcomm SMD implementation a couple of years back people
>>>>> suggested that I should make it "a rpmsg thingie". With the introduction of the
>>>>> Qualcomm 8996 platform, we must support a variant of the communication
>>>>> mechanism that share many of the characteristics of SMD, but are different
>>>>> enough that it can't be done in a single implementation. As such there is
>>>>> enough benefit to do the necessary work and being able to make SMD a "rpmsg
>>>>> thingie".
>>>>>
>>>>> On-top of this series I have patches to switch the current smd clients over to
>>>>> rpmsg (and by that drop the existing SMD implementation).
>>>>>
>>>>> All this allows me to implement the new backend and reuse all existing SMD
>>>>> drivers with the new mechanism.
>>>>>
>>>>
>>>> RPM Communication has to supported even when IRQs are disabled. The most
>>>> important use of this communication is to set the wake up time for the
>>>> CPU subsystem when all the CPUs are powered off.
>>>
>>> Can you point me to the downstream code where this is implemented so I
>>> can have a look? Do you expect to get the response on that request?
>>
>> Have a look at -
>> smd_mask_receive_interrupt()
>> smd_is_pkt_avail()
>>
>
> In msm-3.18 these still seems to only come from either
> msm_rpm_enter_sleep() and the rpm-clock driver, related to flushing
> cached sleep state requests.
>
>> Every request to the RPM generates a response.  The Linux RPM driver may
>> decide to let the response sit in the fifo, or it may need to read and
>> process it.
>>
>
> Right, I presume we save some time by not waiting for these responses as
> we want to reach sleep as soon as possible. The answer I got last time
> this was discussed was that it was an optimization, not a functional
> requirement.

Two optimizations in play here.

First, disabling interrupts prevents an immediate wakeup.  When the 
system is entering sleep, IRQs are disabled.  The sleep request to RPM 
will trigger a response, and the IRQ for that response will be queued. 
Once the sleep processing is done, IRQs get enabled, so the pending IRQ 
from RPM will cause an immediate wakeup.  The system will process the 
wakeup, and then go back to sleep (sans request because nothing has 
changed).  This down-up-down processing burns a lot of power.

Second is not waiting for the response.  Linux doesn't really do 
anything with the sleep request response, so we can enter sleep faster 
by not waiting for the response and processing (discarding) it when the 
system wakes up as scheduled.  However, Linux needs to ensure there is 
enough fifo space to hold that response while asleep, otherwise the RPM 
will panic and crash the system.  Therefore, if there are a number of 
outstanding requests that would fill the fifo, then the RPM driver on 
Linux needs to spin and drain requests from the fifo until a minimum 
free space buffer to hold additional expected pending responses is 
established.  This has to occur with IRQs disabled.

>
>
> I'm not at all against having the rpm driver cache the state
> information and the smd driver process read/writes from the rpm driver
> in IRQ context. I do however not know how to trigger the flush in a sane
> way.
>
>>>
>>>> In addition to that,
>>>> "sleep" votes that are sent by the application processor subsystem to
>>>> allow system to go into deep sleep modes can only be triggered when the
>>>> CPU PM domains are power collapsed, drivers do not have a knowledge of
>>>> when that happens.
>>>
>>> Do you mean the actual sleep votes can only be with the CPU PM domains
>>> collapsed?
>>>
>>> It's been a while since I dug through that code, but there was several
>>> cases where sleep votes would be sent out during normal execution as
>>> well, and then there's the optimization of flushing out all cached sleep
>>> votes when we're on the way down.
>>>
>>>> This has to be done by a platform code that registers
>>>> for CPU PM domain power_off/on callbacks.
>>>>
>>>
>>> Ok, sounds like we have a legit use case for improving this.
>>>
>>>> Using rpmsg may be nice for RPM SMD communication, but mutexes need to
>>>> go away for this driver to be any useful than bare bones active mode
>>>> resource requests for QCOM SoCs. By not doing that now, we lock
>>>> ourselves out of using this SMD driver in the near future when CPU PM
>>>> domains are available in the kernel with an ability to do system low
>>>> power modes.
>>>>
>>>
>>> The last time I looked at this there where no cases when it was
>>> _required_ to support transmitting requests to the rpm from IRQ context.
>>
>> I no longer work on SMD, but when I did this was in fact a strict
>> requirement.
>
> When I dissected all the users of the API I came to the conclusion that
> this requirement (on the SMD driver) came from above mentioned
> optimization.
>
>> If I recall correctly, there was a parameter in the RPM driver
>> for the transmit function that indicated if the request was being made in
>> atomic context or not, which would change the behavior of how the transmit
>> was handled.
>>
>
> You're correct, the question is still which of these code paths are
> actually needed and to motivate the endless maintenance of the extra
> code.

If we are just talking about transmitting in atomic context (not 
necessarily related to sleep), if I recall correctly, some bus requests 
are sent to RPM in atomic context, some APR requests to the Audio DSP 
are done in atomic context, and I think IPC Router uses atomic context 
in some cases.  As a generic framework that should support usecases to 
all processors/subsystems, I don't think transmitting in atomic context 
is a special case for RPM/sleep.

Lina et al would probably know better about the usecase details than I 
at this point however.

>
>
> Nice to see you on the mailing list again Jeff.
>
> Regards,
> Bjorn
>


-- 
Jeffrey Hugo
Qualcomm Datacenter Technologies as an affiliate of Qualcomm 
Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

[toc] | [prev] | [next] | [standalone]

#1481930

On Mon 12 Sep 12:21 PDT 2016, Jeffrey Hugo wrote:

> On 9/12/2016 12:49 PM, Bjorn Andersson wrote:
> >On Mon 12 Sep 11:13 PDT 2016, Jeffrey Hugo wrote:
> >
> >>On 9/12/2016 12:00 PM, Bjorn Andersson wrote:
[..]
> >>>Can you point me to the downstream code where this is implemented so I
> >>>can have a look? Do you expect to get the response on that request?
> >>
> >>Have a look at -
> >>smd_mask_receive_interrupt()
> >>smd_is_pkt_avail()
> >>
> >
> >In msm-3.18 these still seems to only come from either
> >msm_rpm_enter_sleep() and the rpm-clock driver, related to flushing
> >cached sleep state requests.
> >
> >>Every request to the RPM generates a response.  The Linux RPM driver may
> >>decide to let the response sit in the fifo, or it may need to read and
> >>process it.
> >>
> >
> >Right, I presume we save some time by not waiting for these responses as
> >we want to reach sleep as soon as possible. The answer I got last time
> >this was discussed was that it was an optimization, not a functional
> >requirement.
> 
> Two optimizations in play here.
> 
> First, disabling interrupts prevents an immediate wakeup.  When the system
> is entering sleep, IRQs are disabled.  The sleep request to RPM will trigger
> a response, and the IRQ for that response will be queued. Once the sleep
> processing is done, IRQs get enabled, so the pending IRQ from RPM will cause
> an immediate wakeup.  The system will process the wakeup, and then go back
> to sleep (sans request because nothing has changed).  This down-up-down
> processing burns a lot of power.
> 

But which "sleep request" is this? The only one I can find is the
flushing of sleep state values from the rpm resource tables.

> Second is not waiting for the response.  Linux doesn't really do anything
> with the sleep request response, so we can enter sleep faster by not waiting
> for the response and processing (discarding) it when the system wakes up as
> scheduled.

Right, as long as the RPM code doesn't consider it a timeout don't have
a problem if those ack's are handled after the resume.

> However, Linux needs to ensure there is enough fifo space to
> hold that response while asleep, otherwise the RPM will panic and crash the
> system.  Therefore, if there are a number of outstanding requests that would
> fill the fifo, then the RPM driver on Linux needs to spin and drain requests
> from the fifo until a minimum free space buffer to hold additional expected
> pending responses is established.  This has to occur with IRQs disabled.
> 

Right. Which means that the RPM driver needs to know how large the rx
fifo is, what overhead the underlaying transport mechanism has and then
calculate how many responses it should leave room for.

[..]
> >>If I recall correctly, there was a parameter in the RPM driver
> >>for the transmit function that indicated if the request was being made in
> >>atomic context or not, which would change the behavior of how the transmit
> >>was handled.
> >>
> >
> >You're correct, the question is still which of these code paths are
> >actually needed and to motivate the endless maintenance of the extra
> >code.
> 
> If we are just talking about transmitting in atomic context (not necessarily
> related to sleep), if I recall correctly, some bus requests are sent to RPM
> in atomic context, some APR requests to the Audio DSP are done in atomic
> context, and I think IPC Router uses atomic context in some cases.  As a
> generic framework that should support usecases to all processors/subsystems,
> I don't think transmitting in atomic context is a special case for
> RPM/sleep.
> 

I have not looked through all of APR yet and don't know where msm_bus is
heading, but for IPC-router your correct that the downstream driver does
indeed require this; but that's a side effect of the downstream
ipcrouter implementation, not the problem itself.

Regards,
Bjorn

[toc] | [prev] | [next] | [standalone]

#1481997

On 9/12/2016 1:58 PM, Bjorn Andersson wrote:
> On Mon 12 Sep 12:21 PDT 2016, Jeffrey Hugo wrote:
>
>> On 9/12/2016 12:49 PM, Bjorn Andersson wrote:
>>> On Mon 12 Sep 11:13 PDT 2016, Jeffrey Hugo wrote:
>>>
>>>> On 9/12/2016 12:00 PM, Bjorn Andersson wrote:
> [..]
>>>>> Can you point me to the downstream code where this is implemented so I
>>>>> can have a look? Do you expect to get the response on that request?
>>>>
>>>> Have a look at -
>>>> smd_mask_receive_interrupt()
>>>> smd_is_pkt_avail()
>>>>
>>>
>>> In msm-3.18 these still seems to only come from either
>>> msm_rpm_enter_sleep() and the rpm-clock driver, related to flushing
>>> cached sleep state requests.
>>>
>>>> Every request to the RPM generates a response.  The Linux RPM driver may
>>>> decide to let the response sit in the fifo, or it may need to read and
>>>> process it.
>>>>
>>>
>>> Right, I presume we save some time by not waiting for these responses as
>>> we want to reach sleep as soon as possible. The answer I got last time
>>> this was discussed was that it was an optimization, not a functional
>>> requirement.
>>
>> Two optimizations in play here.
>>
>> First, disabling interrupts prevents an immediate wakeup.  When the system
>> is entering sleep, IRQs are disabled.  The sleep request to RPM will trigger
>> a response, and the IRQ for that response will be queued. Once the sleep
>> processing is done, IRQs get enabled, so the pending IRQ from RPM will cause
>> an immediate wakeup.  The system will process the wakeup, and then go back
>> to sleep (sans request because nothing has changed).  This down-up-down
>> processing burns a lot of power.
>>
>
> But which "sleep request" is this? The only one I can find is the
> flushing of sleep state values from the rpm resource tables.
>
>> Second is not waiting for the response.  Linux doesn't really do anything
>> with the sleep request response, so we can enter sleep faster by not waiting
>> for the response and processing (discarding) it when the system wakes up as
>> scheduled.
>
> Right, as long as the RPM code doesn't consider it a timeout don't have
> a problem if those ack's are handled after the resume.
>
>> However, Linux needs to ensure there is enough fifo space to
>> hold that response while asleep, otherwise the RPM will panic and crash the
>> system.  Therefore, if there are a number of outstanding requests that would
>> fill the fifo, then the RPM driver on Linux needs to spin and drain requests
>> from the fifo until a minimum free space buffer to hold additional expected
>> pending responses is established.  This has to occur with IRQs disabled.
>>
>
> Right. Which means that the RPM driver needs to know how large the rx
> fifo is, what overhead the underlaying transport mechanism has and then
> calculate how many responses it should leave room for.
>
> [..]
>>>> If I recall correctly, there was a parameter in the RPM driver
>>>> for the transmit function that indicated if the request was being made in
>>>> atomic context or not, which would change the behavior of how the transmit
>>>> was handled.
>>>>
>>>
>>> You're correct, the question is still which of these code paths are
>>> actually needed and to motivate the endless maintenance of the extra
>>> code.
>>
>> If we are just talking about transmitting in atomic context (not necessarily
>> related to sleep), if I recall correctly, some bus requests are sent to RPM
>> in atomic context, some APR requests to the Audio DSP are done in atomic
>> context, and I think IPC Router uses atomic context in some cases.  As a
>> generic framework that should support usecases to all processors/subsystems,
>> I don't think transmitting in atomic context is a special case for
>> RPM/sleep.
>>
>
> I have not looked through all of APR yet and don't know where msm_bus is
> heading, but for IPC-router your correct that the downstream driver does
> indeed require this; but that's a side effect of the downstream
> ipcrouter implementation, not the problem itself.
>
APR does send messages in atomic context in addition to the RPM Driver, 
but IPC Router does not to the extent of my knowledge.
> Regards,
> Bjorn
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Regards,
Karthik.
-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
  hosted by The Linux Foundation

[toc] | [prev] | [next] | [standalone]

#1482005

On Mon 12 Sep 14:36 PDT 2016, Karthikeyan Ramasubramanian wrote:

> On 9/12/2016 1:58 PM, Bjorn Andersson wrote:
[..]
> >I have not looked through all of APR yet and don't know where msm_bus is
> >heading, but for IPC-router your correct that the downstream driver does
> >indeed require this; but that's a side effect of the downstream
> >ipcrouter implementation, not the problem itself.
> >
> APR does send messages in atomic context in addition to the RPM Driver, but
> IPC Router does not to the extent of my knowledge.

Thanks for the information, Karthik.

There seem to be 196 calls to apr_send_pkt() in the msm-3.18 kernel, I
will have to go through them all later, but could you help me by
pointing me to one of those that actually need to be executed in atomic
context?

Thanks,
Bjorn

[toc] | [prev] | [next] | [standalone]

#1482040

On Mon, Sep 12 2016 at 10:52 -0600, Lina Iyer wrote:
>Hi Bjorn,
>
>On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
>>This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
>>backend/wireformat.
>>
>>
>>As we discussed the Qualcomm SMD implementation a couple of years back people
>>suggested that I should make it "a rpmsg thingie". With the introduction of the
>>Qualcomm 8996 platform, we must support a variant of the communication
>>mechanism that share many of the characteristics of SMD, but are different
>>enough that it can't be done in a single implementation. As such there is
>>enough benefit to do the necessary work and being able to make SMD a "rpmsg
>>thingie".
>>
>>On-top of this series I have patches to switch the current smd clients over to
>>rpmsg (and by that drop the existing SMD implementation).
>>
>>All this allows me to implement the new backend and reuse all existing SMD
>>drivers with the new mechanism.
>>
>
>RPM Communication has to supported even when IRQs are disabled. The most
>important use of this communication is to set the wake up time for the
>CPU subsystem when all the CPUs are powered off. In addition to that,
>"sleep" votes that are sent by the application processor subsystem to
>allow system to go into deep sleep modes can only be triggered when the
>CPU PM domains are power collapsed, drivers do not have a knowledge of
>when that happens. This has to be done by a platform code that registers
>for CPU PM domain power_off/on callbacks.
>
Ok, my bad. These two cases are not critical for the SoC supported by
this driver. So you are good to go from cpuidle perspective

>Using rpmsg may be nice for RPM SMD communication, but mutexes need to
>go away for this driver to be any useful than bare bones active mode
>resource requests for QCOM SoCs. By not doing that now, we lock
>ourselves out of using this SMD driver in the near future when CPU PM
>domains are available in the kernel with an ability to do system low
>power modes.
>
>I hope you would make rpmsg work in IRQ disabled contexts first before
>porting the SMD driver.
>
>Thanks,
>Lina
>
>>
>>Changes from v1:
>>- Split up the patch moving core code to rpmsg_core into several commits
>>- Dropped the wrapping struct in rpmsg_core and just added the ops to the
>> public API (but hid the implementation details)
>>- Reordered things to reduce the size of the later patches
>>
>>Bjorn Andersson (17):
>> rpmsg: Enable matching devices with drivers based on DT
>> rpmsg: Name rpmsg devices based on channel id
>> rpmsg: rpmsg_send() operations takes rpmsg_endpoint
>> rpmsg: Make rpmsg_create_ept() take channel_info struct
>> rpmsg: Clean up rpmsg device vs channel naming
>> rpmsg: Introduce indirection table for rpmsg_device operations
>> rpmsg: Move rpmsg_device API to new file
>> rpmsg: Indirection table for rpmsg_endpoint operations
>> rpmsg: Move endpoint related interface to rpmsg core
>> rpmsg: Move helper for finding rpmsg devices to core
>> rpmsg: Split off generic tail of create_channel()
>> rpmsg: Split rpmsg core and virtio backend
>> rpmsg: Hide rpmsg indirection tables
>> rpmsg: virtio: Hide vrp pointer from the public API
>> rpmsg: Move virtio specifics from public header
>> rpmsg: Allow callback to return errors
>> rpmsg: Introduce Qualcomm SMD backend
>>
>>drivers/remoteproc/Kconfig          |    4 +-
>>drivers/rpmsg/Kconfig               |   14 +
>>drivers/rpmsg/Makefile              |    4 +-
>>drivers/rpmsg/qcom_smd.c            | 1434 +++++++++++++++++++++++++++++++++++
>>drivers/rpmsg/rpmsg_core.c          |  498 ++++++++++++
>>drivers/rpmsg/rpmsg_internal.h      |   82 ++
>>drivers/rpmsg/virtio_rpmsg_bus.c    |  487 +++++-------
>>include/linux/rpmsg.h               |  246 +-----
>>samples/rpmsg/rpmsg_client_sample.c |   14 +-
>>9 files changed, 2266 insertions(+), 517 deletions(-)
>>create mode 100644 drivers/rpmsg/qcom_smd.c
>>create mode 100644 drivers/rpmsg/rpmsg_core.c
>>create mode 100644 drivers/rpmsg/rpmsg_internal.h
>>
>>-- 
>>2.5.0
>>
>>--
>>To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [next] | [standalone]

#1482050

On Mon 12 Sep 15:22 PDT 2016, Lina Iyer wrote:

> On Mon, Sep 12 2016 at 10:52 -0600, Lina Iyer wrote:
> >Hi Bjorn,
> >
> >On Thu, Sep 01 2016 at 16:28 -0600, Bjorn Andersson wrote:
> >>This series splits the virtio rpmsg bus driver into a rpmsg bus and a virtio
> >>backend/wireformat.
> >>
> >>
> >>As we discussed the Qualcomm SMD implementation a couple of years back people
> >>suggested that I should make it "a rpmsg thingie". With the introduction of the
> >>Qualcomm 8996 platform, we must support a variant of the communication
> >>mechanism that share many of the characteristics of SMD, but are different
> >>enough that it can't be done in a single implementation. As such there is
> >>enough benefit to do the necessary work and being able to make SMD a "rpmsg
> >>thingie".
> >>
> >>On-top of this series I have patches to switch the current smd clients over to
> >>rpmsg (and by that drop the existing SMD implementation).
> >>
> >>All this allows me to implement the new backend and reuse all existing SMD
> >>drivers with the new mechanism.
> >>
> >
> >RPM Communication has to supported even when IRQs are disabled. The most
> >important use of this communication is to set the wake up time for the
> >CPU subsystem when all the CPUs are powered off. In addition to that,
> >"sleep" votes that are sent by the application processor subsystem to
> >allow system to go into deep sleep modes can only be triggered when the
> >CPU PM domains are power collapsed, drivers do not have a knowledge of
> >when that happens. This has to be done by a platform code that registers
> >for CPU PM domain power_off/on callbacks.
> >
> Ok, my bad. These two cases are not critical for the SoC supported by
> this driver. So you are good to go from cpuidle perspective
> 

Thanks for letting me know. Please keep me updated if you find any
changes to this.

Just to be clear, I'm not against supporting sending and receiving
messages in atomic context as long as it doesn't just add accidental
complexity.

Regards,
Bjorn

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web