Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1451770 > unrolled thread

[RFC v0 0/8] Reuse firmware loader helpers

Back to article view | Back to linux.kernel

Contents

  [RFC v0 0/8] Reuse firmware loader helpers Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
    [RFC v0 8/8] iwl4965: use firmware_stat instead of completion Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
    [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
      Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-07-28 20:40 +0200
        Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-07-28 21:10 +0200
          Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-07-29 08:20 +0200
            Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Arend van Spriel <arend.vanspriel@broadcom.com> - 2016-07-30 14:50 +0200
              Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-07-30 19:00 +0200
                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-07-31 09:30 +0200
                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-08-01 14:40 +0200
                    Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-02 00:40 +0200
                      Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-08-02 08:00 +0200
                        Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-02 08:40 +0200
                          Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-08-02 09:00 +0200
                            Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-02 10:00 +0200
                              Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-08-03 09:10 +0200
                                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-08-03 18:20 +0200
                                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 20:20 +0200
                                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 18:50 +0200
                                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 20:50 +0200
                                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-08-04 00:40 +0200
                              Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-08-03 09:50 +0200
                                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Arend van Spriel <arend.vanspriel@broadcom.com> - 2016-08-03 13:50 +0200
                                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 17:20 +0200
                                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-08-03 17:40 +0200
                                    Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Arend van Spriel <arend.vanspriel@broadcom.com> - 2016-08-03 23:00 +0200
                                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 18:10 +0200
                              Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-08-03 19:50 +0200
                                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 22:40 +0200
                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-01 22:20 +0200
                Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Bjorn Andersson <bjorn.andersson@linaro.org> - 2016-08-01 19:30 +0200
                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-01 22:20 +0200
                  Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of  completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-08-01 23:50 +0200
              Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-07-31 09:20 +0200
    [RFC v0 2/8] selftests: firmware: do not clutter output Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
    [RFC v0 6/8] remoteproc: use firmware_stat instead of completion Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
    [RFC v0 3/8] firmware: Factor out firmware load helpers Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
      Re: [RFC v0 3/8] firmware: Factor out firmware load helpers Dan Williams <dcbw@redhat.com> - 2016-07-28 17:10 +0200
        Re: [RFC v0 3/8] firmware: Factor out firmware load helpers Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-07-29 08:10 +0200
      Re: [RFC v0 3/8] firmware: Factor out firmware load helpers Dmitry Torokhov <dmitry.torokhov@gmail.com> - 2016-07-28 20:00 +0200
        Re: [RFC v0 3/8] firmware: Factor out firmware load helpers Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-07-29 08:10 +0200
    [RFC v0 4/8] Input: goodix: use firmware_stat instead of completion Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200
      Re: [RFC v0 4/8] Input: goodix: use firmware_stat instead of  completion Bastien Nocera <hadess@hadess.net> - 2016-07-28 13:30 +0200
        Re: [RFC v0 4/8] Input: goodix: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-07-28 14:10 +0200
          Re: [RFC v0 4/8] Input: goodix: use firmware_stat instead of  completion Bastien Nocera <hadess@hadess.net> - 2016-07-28 14:30 +0200
            Re: [RFC v0 4/8] Input: goodix: use firmware_stat instead of  completion Daniel Wagner <daniel.wagner@bmw-carit.de> - 2016-07-28 15:20 +0200
    [RFC v0 1/8] selftests: firmware: do not abort test too early Daniel Wagner <wagi@monom.org> - 2016-07-28 10:00 +0200

Page 1 of 3  [1] 2 3  Next page →

#1451770 — [RFC v0 0/8] Reuse firmware loader helpers

From: Daniel Wagner <daniel.wagner@bmw-carit.de>

Hi,

While reviewing all the complete_all() users, I realized there is
recouring pattern how the completion API is used to synchronize the
stages of the firmware loading. Since firmware_class.c contains a
fairly complete implemetation for synching the loading, it worthwhile
to export it and reuse it in drivers, At the same time one
complete_all() user is gone which is a good thing for -rt (*).

The first 2 patches are bug fixes for the test script.

Patch 3 adds the new API, and the following patches update a few
drivers.

I haven't updated all the drivers because I wanted to see first if
this is going into the right direction. Since naming is a difficult, I
am more than please to pick a better name if you have one.

cheers,
daniel

(*) Under -rt waking all waiters via complete_all() is not good. If
the complete_all() call happens in IRQ context we have an unbound
latency there. Therefore the aim is to reduce the complete_all() users
and get rid of them where possible.

Daniel Wagner (8):
  selftests: firmware: do not abort test too early
  selftests: firmware: do not clutter output
  firmware: Factor out firmware load helpers
  Input: goodix: use firmware_stat instead of completion
  ath9k_htc: use firmware_stat instead of completion
  remoteproc: use firmware_stat instead of completion
  Input: ims-pcu: use firmware_stat instead of completion
  iwl4965: use firmware_stat instead of completion

 drivers/base/firmware_class.c                     | 112 ++++++++++------------
 drivers/input/misc/ims-pcu.c                      |  10 +-
 drivers/input/touchscreen/goodix.c                |  10 +-
 drivers/net/wireless/ath/ath9k/hif_usb.c          |  10 +-
 drivers/net/wireless/ath/ath9k/hif_usb.h          |   2 +-
 drivers/net/wireless/intel/iwlegacy/4965-mac.c    |   8 +-
 drivers/net/wireless/intel/iwlegacy/common.h      |   3 +-
 drivers/remoteproc/remoteproc_core.c              |  10 +-
 drivers/soc/ti/wkup_m3_ipc.c                      |   2 +-
 include/linux/firmware.h                          |  71 ++++++++++++++
 include/linux/remoteproc.h                        |   6 +-
 tools/testing/selftests/firmware/fw_filesystem.sh |   6 +-
 tools/testing/selftests/firmware/fw_userhelper.sh |   2 +-
 13 files changed, 158 insertions(+), 94 deletions(-)

-- 
2.7.4

[toc] | [next] | [standalone]

#1451771 — [RFC v0 8/8] iwl4965: use firmware_stat instead of completion

From: Daniel Wagner <daniel.wagner@bmw-carit.de>

Loading firmware is an operation many drivers implement in various ways
around the completion API. And most of them do it almost in the same
way. Let's reuse the firmware_stat API which is used also by the
firmware_class loader. Apart of streamlining the firmware loading states
we also document it slightly better.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 drivers/net/wireless/intel/iwlegacy/4965-mac.c | 8 ++++----
 drivers/net/wireless/intel/iwlegacy/common.h   | 3 ++-
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/wireless/intel/iwlegacy/4965-mac.c b/drivers/net/wireless/intel/iwlegacy/4965-mac.c
index a91d170..d5e5808 100644
--- a/drivers/net/wireless/intel/iwlegacy/4965-mac.c
+++ b/drivers/net/wireless/intel/iwlegacy/4965-mac.c
@@ -5005,7 +5005,7 @@ il4965_ucode_callback(const struct firmware *ucode_raw, void *context)
 
 	/* We have our copies now, allow OS release its copies */
 	release_firmware(ucode_raw);
-	complete(&il->_4965.firmware_loading_complete);
+	fw_loading_done(il->_4965.fw_st);
 	return;
 
 try_again:
@@ -5019,7 +5019,7 @@ err_pci_alloc:
 	IL_ERR("failed to allocate pci memory\n");
 	il4965_dealloc_ucode_pci(il);
 out_unbind:
-	complete(&il->_4965.firmware_loading_complete);
+	fw_loading_done(il->_4965.fw_st);
 	device_release_driver(&il->pci_dev->dev);
 	release_firmware(ucode_raw);
 }
@@ -6678,7 +6678,7 @@ il4965_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	il_power_initialize(il);
 
-	init_completion(&il->_4965.firmware_loading_complete);
+	firmware_stat_init(&il->_4965.fw_st);
 
 	err = il4965_request_firmware(il, true);
 	if (err)
@@ -6716,7 +6716,7 @@ il4965_pci_remove(struct pci_dev *pdev)
 	if (!il)
 		return;
 
-	wait_for_completion(&il->_4965.firmware_loading_complete);
+	fw_loading_wait(il->_4965.fw_st);
 
 	D_INFO("*** UNLOAD DRIVER ***\n");
 
diff --git a/drivers/net/wireless/intel/iwlegacy/common.h b/drivers/net/wireless/intel/iwlegacy/common.h
index 726ede3..94af7b7 100644
--- a/drivers/net/wireless/intel/iwlegacy/common.h
+++ b/drivers/net/wireless/intel/iwlegacy/common.h
@@ -32,6 +32,7 @@
 #include <linux/leds.h>
 #include <linux/wait.h>
 #include <linux/io.h>
+#include <linux/firmware.h>
 #include <net/mac80211.h>
 #include <net/ieee80211_radiotap.h>
 
@@ -1357,7 +1358,7 @@ struct il_priv {
 			bool last_phy_res_valid;
 			u32 ampdu_ref;
 
-			struct completion firmware_loading_complete;
+			struct firmware_stat fw_st;
 
 			/*
 			 * chain noise reset and gain commands are the
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1451772 — [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

From: Daniel Wagner <daniel.wagner@bmw-carit.de>

Loading firmware is an operation many drivers implement in various ways
around the completion API. And most of them do it almost in the same
way. Let's reuse the firmware_stat API which is used also by the
firmware_class loader. Apart of streamlining the firmware loading states
we also document it slightly better.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 drivers/input/misc/ims-pcu.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
index 9c0ea36..cda1fbf 100644
--- a/drivers/input/misc/ims-pcu.c
+++ b/drivers/input/misc/ims-pcu.c
@@ -109,7 +109,7 @@ struct ims_pcu {
 
 	u32 fw_start_addr;
 	u32 fw_end_addr;
-	struct completion async_firmware_done;
+	struct firmware_stat fw_st;
 
 	struct ims_pcu_buttons buttons;
 	struct ims_pcu_gamepad *gamepad;
@@ -940,7 +940,7 @@ static void ims_pcu_process_async_firmware(const struct firmware *fw,
 	release_firmware(fw);
 
 out:
-	complete(&pcu->async_firmware_done);
+	fw_loading_done(pcu->fw_st);
 }
 
 /*********************************************************************
@@ -1967,7 +1967,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
 					ims_pcu_process_async_firmware);
 	if (error) {
 		/* This error is not fatal, let userspace have another chance */
-		complete(&pcu->async_firmware_done);
+		fw_loading_abort(pcu->fw_st);
 	}
 
 	return 0;
@@ -1976,7 +1976,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
 static void ims_pcu_destroy_bootloader_mode(struct ims_pcu *pcu)
 {
 	/* Make sure our initial firmware request has completed */
-	wait_for_completion(&pcu->async_firmware_done);
+	fw_loading_wait(pcu->fw_st);
 }
 
 #define IMS_PCU_APPLICATION_MODE	0
@@ -2000,7 +2000,7 @@ static int ims_pcu_probe(struct usb_interface *intf,
 	pcu->bootloader_mode = id->driver_info == IMS_PCU_BOOTLOADER_MODE;
 	mutex_init(&pcu->cmd_mutex);
 	init_completion(&pcu->cmd_done);
-	init_completion(&pcu->async_firmware_done);
+	firmware_stat_init(&pcu->fw_st);
 
 	error = ims_pcu_parse_cdc_data(intf, pcu);
 	if (error)
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1452044 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> From: Daniel Wagner <daniel.wagner@bmw-carit.de>
> 
> Loading firmware is an operation many drivers implement in various ways
> around the completion API. And most of them do it almost in the same
> way. Let's reuse the firmware_stat API which is used also by the
> firmware_class loader. Apart of streamlining the firmware loading states
> we also document it slightly better.
> 
> Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
> ---
>  drivers/input/misc/ims-pcu.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
> index 9c0ea36..cda1fbf 100644
> --- a/drivers/input/misc/ims-pcu.c
> +++ b/drivers/input/misc/ims-pcu.c
> @@ -109,7 +109,7 @@ struct ims_pcu {
>  
>  	u32 fw_start_addr;
>  	u32 fw_end_addr;
> -	struct completion async_firmware_done;
> +	struct firmware_stat fw_st;
>  
>  	struct ims_pcu_buttons buttons;
>  	struct ims_pcu_gamepad *gamepad;
> @@ -940,7 +940,7 @@ static void ims_pcu_process_async_firmware(const struct firmware *fw,
>  	release_firmware(fw);
>  
>  out:
> -	complete(&pcu->async_firmware_done);
> +	fw_loading_done(pcu->fw_st);

Why does the driver have to do it? If firmware loader manages this, then
it should let waiters know when callback finishes. 

>  }
>  
>  /*********************************************************************
> @@ -1967,7 +1967,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
>  					ims_pcu_process_async_firmware);
>  	if (error) {
>  		/* This error is not fatal, let userspace have another chance */
> -		complete(&pcu->async_firmware_done);
> +		fw_loading_abort(pcu->fw_st);

Why should the driver signal abort if it does not manage completion in
this case?

>  	}
>  
>  	return 0;
> @@ -1976,7 +1976,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
>  static void ims_pcu_destroy_bootloader_mode(struct ims_pcu *pcu)
>  {
>  	/* Make sure our initial firmware request has completed */
> -	wait_for_completion(&pcu->async_firmware_done);
> +	fw_loading_wait(pcu->fw_st);
>  }
>  
>  #define IMS_PCU_APPLICATION_MODE	0
> @@ -2000,7 +2000,7 @@ static int ims_pcu_probe(struct usb_interface *intf,
>  	pcu->bootloader_mode = id->driver_info == IMS_PCU_BOOTLOADER_MODE;
>  	mutex_init(&pcu->cmd_mutex);
>  	init_completion(&pcu->cmd_done);
> -	init_completion(&pcu->async_firmware_done);
> +	firmware_stat_init(&pcu->fw_st);

Do not quite like it... I'd rather asynchronous request give out a
firmware status pointer that could be used later on.

	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
					    pcu,
					    ims_pcu_process_async_firmware);
	if (IS_ERR(pcu->fw_st))
		return PTR_ERR(pcu->fw_st);

	....

	fw_loading_wait(pcu->fw_st);

Thanks.

-- 
Dmitry

[toc] | [prev] | [next] | [standalone]

#1452061 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:

> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> > From: Daniel Wagner <daniel.wagner@bmw-carit.de>
> > 
[..]
> 
> Do not quite like it... I'd rather asynchronous request give out a
> firmware status pointer that could be used later on.
> 
> 	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
> 					    pcu,
> 					    ims_pcu_process_async_firmware);
> 	if (IS_ERR(pcu->fw_st))
> 		return PTR_ERR(pcu->fw_st);
> 
> 	....
> 
> 	fw_loading_wait(pcu->fw_st);
> 

In the remoteproc case (patch 6) this would clean up the code, rather
than replacing the completion API 1 to 1. I like it!

Regards,
Bjorn

[toc] | [prev] | [next] | [standalone]

#1452269 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>
>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>>> From: Daniel Wagner <daniel.wagner@bmw-carit.de>
>>>
> [..]
>>
>> Do not quite like it... I'd rather asynchronous request give out a
>> firmware status pointer that could be used later on.
>>
>> 	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
>> 					    pcu,
>> 					    ims_pcu_process_async_firmware);
>> 	if (IS_ERR(pcu->fw_st))
>> 		return PTR_ERR(pcu->fw_st);
>>
>> 	....
>>
>> 	fw_loading_wait(pcu->fw_st);
>>
>
> In the remoteproc case (patch 6) this would clean up the code, rather
> than replacing the completion API 1 to 1. I like it!

IIRC most drivers do it the same way. So request_firmware_async() indeed 
would be good thing to have. Let me try that.

Thanks for the excellent feedback.

cheers,
daniel

[toc] | [prev] | [next] | [standalone]

#1452680 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

+ Luis (again) ;-)

On 29-07-16 08:13, Daniel Wagner wrote:
> On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>>
>>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>>>> From: Daniel Wagner <daniel.wagner@bmw-carit.de>
>>>>
>> [..]
>>>
>>> Do not quite like it... I'd rather asynchronous request give out a
>>> firmware status pointer that could be used later on.

Excellent. Why not get rid of the callback function as well and have
fw_loading_wait() return result (0 = firmware available, < 0 = fail).
Just to confirm, you are proposing a new API function next to
request_firmware_nowait(), right?

>>>     pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
>>> -                       pcu,
>>> -                       ims_pcu_process_async_firmware);
    +			    pcu);
>>>     if (IS_ERR(pcu->fw_st))
>>>         return PTR_ERR(pcu->fw_st);
>>>
>>>     ....
>>>
>>>     err = fw_loading_wait(pcu->fw_st);
	if (err)
		return err;

	fw = fwstat_get_firmware(pcu->fw_st);

Or whatever consistent prefix it is going to be.

>>>
>>
>> In the remoteproc case (patch 6) this would clean up the code, rather
>> than replacing the completion API 1 to 1. I like it!
> 
> IIRC most drivers do it the same way. So request_firmware_async() indeed
> would be good thing to have. Let me try that.

While the idea behind this series is a good one I am wondering about the
need for these drivers to use the asynchronous API. The historic reason
might be to avoid timeout caused by user-mode helper, but that may no
longer apply and these drivers could be better off using
request_firmware_direct().

There have been numerous discussions about the firmware API. Here most
recent one:

http://www.spinics.net/lists/linux-wireless/index.html#152755

Regards,
Arend

> Thanks for the excellent feedback.
> 
> cheers,
> daniel
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [next] | [standalone]

#1452728 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> + Luis (again) ;-)
> 
> On 29-07-16 08:13, Daniel Wagner wrote:
> > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> >>
> >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> >>>> From: Daniel Wagner <daniel.wagner@bmw-carit.de>
> >>>>
> >> [..]
> >>>
> >>> Do not quite like it... I'd rather asynchronous request give out a
> >>> firmware status pointer that could be used later on.
> 
> Excellent. Why not get rid of the callback function as well and have
> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> Just to confirm, you are proposing a new API function next to
> request_firmware_nowait(), right?

If proposing new firmware_class patches please bounce / Cc me, I've
recently asked for me to be added to MAINTAINERS so I get these
e-mails as I'm working on a new flexible API which would allow us
to extend the firmware API without having to care about the old
stupid usermode helper at all.

> >>>     pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
> >>> -                       pcu,
> >>> -                       ims_pcu_process_async_firmware);
>     +			    pcu);
> >>>     if (IS_ERR(pcu->fw_st))
> >>>         return PTR_ERR(pcu->fw_st);
> >>>
> >>>     ....
> >>>
> >>>     err = fw_loading_wait(pcu->fw_st);
> 	if (err)
> 		return err;
> 
> 	fw = fwstat_get_firmware(pcu->fw_st);
> 
> Or whatever consistent prefix it is going to be.
> 
> >>>
> >>
> >> In the remoteproc case (patch 6) this would clean up the code, rather
> >> than replacing the completion API 1 to 1. I like it!
> > 
> > IIRC most drivers do it the same way. So request_firmware_async() indeed
> > would be good thing to have. Let me try that.
> 
> While the idea behind this series is a good one I am wondering about the
> need for these drivers to use the asynchronous API. The historic reason
> might be to avoid timeout caused by user-mode helper, but that may no
> longer apply and these drivers could be better off using
> request_firmware_direct().

BTW I have in my queue for the sysdata API something like firmware_request_direct()
but with async support. The only thing left to do I think is just add the devm
helpers so drivers no longer need to worry about the release of the firmware.

> There have been numerous discussions about the firmware API. Here most
> recent one:
> 
> http://www.spinics.net/lists/linux-wireless/index.html#152755

And more importantly, the sysdata API queue:

https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20160616-sysdata-v2

  Luis

[toc] | [prev] | [next] | [standalone]

#1452781 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <mcgrof@kernel.org> wrote:
>On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
>> + Luis (again) ;-)
>> 
>> On 29-07-16 08:13, Daniel Wagner wrote:
>> > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>> >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>> >>
>> >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>> >>>> From: Daniel Wagner <daniel.wagner@bmw-carit.de>
>> >>>>
>> >> [..]
>> >>>
>> >>> Do not quite like it... I'd rather asynchronous request give out
>a
>> >>> firmware status pointer that could be used later on.
>> 
>> Excellent. Why not get rid of the callback function as well and have
>> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
>> Just to confirm, you are proposing a new API function next to
>> request_firmware_nowait(), right?
>
>If proposing new firmware_class patches please bounce / Cc me, I've
>recently asked for me to be added to MAINTAINERS so I get these
>e-mails as I'm working on a new flexible API which would allow us
>to extend the firmware API without having to care about the old
>stupid usermode helper at all.

I am not sure why we started calling usermode helper "stupid". We only had to implement direct kernel firmware loading because udev/stsremd folks had "interesting" ideas how events should be handled; but having userspace to feed us data is not stupid.

If we want to overhaul firmware loading support we need to figure out how to support case when a driver want to [asynchronously] request firmware/config/blob and the rest of the system is not ready. Even if we want kernel to do read/load the data we need userspace to tell kernel when firmware partition is available, until then the kernel should not fail the request.


Thanks.

-- 
Dmitry

[toc] | [prev] | [next] | [standalone]

#1453185 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On 07/31/2016 09:23 AM, Dmitry Torokhov wrote:
> On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <mcgrof@kernel.org> wrote:
>> On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
>>> On 29-07-16 08:13, Daniel Wagner wrote:
>>>> On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>>>>> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:

>>> + Luis (again) ;-)

That was not on purpose :) My attempt to keep the Cc list a bit shorter 
was a failure.

>>>>>> Do not quite like it... I'd rather asynchronous request give out
>>>>>> firmware status pointer that could be used later on.
>>>
>>> Excellent. Why not get rid of the callback function as well and have
>>> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
>>> Just to confirm, you are proposing a new API function next to
>>> request_firmware_nowait(), right?
>>
>> If proposing new firmware_class patches please bounce / Cc me, I've
>> recently asked for me to be added to MAINTAINERS so I get these
>> e-mails as I'm working on a new flexible API which would allow us
>> to extend the firmware API without having to care about the old
>> stupid usermode helper at all.

These patches here are a first attempt to clean up a bit of the code 
around the completion API. As Dmitry correctly pointed out, it makes 
more sense to go bit further and make the async loading a bit more 
convenient for the drivers.

> I am not sure why we started calling usermode helper "stupid". We
> only  had to implement direct kernel firmware loading because udev/stsremd
> folks had "interesting" ideas how events should be handled; but having
> userspace to feed us data is not stupid.

I was ignorant on all the nasty details around the firmware loading. If 
I parse Luis' patches correctly they introduce an API which calls 
kernel_read_file_from_path() asynchronously:

sysdata_file_request_async(..., &cookie)
   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)

   request_sysdata_file_work_fun()
     _sysdata_file_request()
       fw_get_filesystem_firmware()
	kernel_read_file_from_path()

sysdata_synchronize_request(&cookie);

Doesn't look like what your asking for.

> If we want to overhaul firmware loading support we need to figure
> out  how to support case when a driver want to [asynchronously] request
> firmware/config/blob and the rest of the system is not ready. Even if we
> want kernel to do read/load the data we need userspace to tell kernel
> when firmware partition is available, until then the kernel should not
> fail the request.

I gather from Luis' blog post and comments that he is on the quest on 
removing userspace support completely.

Maybe this attempt here could be a step before. Step 1 would be changing 
request_firmware_nowait() to request_firmware_async so drivers don't 
have to come up with their own sync primitives, e.g.

   cookie = request_firmware_async()
   fw_load_wait(cookie)

Step 2 could be something more towards sysdata approach.

cheers,
daniel

[toc] | [prev] | [next] | [standalone]

#1453496 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Mon, Aug 01, 2016 at 02:26:04PM +0200, Daniel Wagner wrote:
> On 07/31/2016 09:23 AM, Dmitry Torokhov wrote:
> >On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <mcgrof@kernel.org> wrote:
> >>On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> >>>On 29-07-16 08:13, Daniel Wagner wrote:
> >>>>On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> >>>>>On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> 
> >>>+ Luis (again) ;-)
> 
> That was not on purpose :) My attempt to keep the Cc list a bit
> shorter was a failure.
> 
> >>>>>>Do not quite like it... I'd rather asynchronous request give out
> >>>>>>firmware status pointer that could be used later on.
> >>>
> >>>Excellent. Why not get rid of the callback function as well and have
> >>>fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> >>>Just to confirm, you are proposing a new API function next to
> >>>request_firmware_nowait(), right?
> >>
> >>If proposing new firmware_class patches please bounce / Cc me, I've
> >>recently asked for me to be added to MAINTAINERS so I get these
> >>e-mails as I'm working on a new flexible API which would allow us
> >>to extend the firmware API without having to care about the old
> >>stupid usermode helper at all.
> 
> These patches here are a first attempt to clean up a bit of the code
> around the completion API. As Dmitry correctly pointed out, it makes
> more sense to go bit further and make the async loading a bit more
> convenient for the drivers.
> 
> >I am not sure why we started calling usermode helper "stupid". We
> >only  had to implement direct kernel firmware loading because udev/stsremd
> >folks had "interesting" ideas how events should be handled; but having
> >userspace to feed us data is not stupid.
> 
> I was ignorant on all the nasty details around the firmware loading.
> If I parse Luis' patches correctly they introduce an API which calls
> kernel_read_file_from_path() asynchronously:
> 
> sysdata_file_request_async(..., &cookie)
>   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)
> 
>   request_sysdata_file_work_fun()
>     _sysdata_file_request()
>       fw_get_filesystem_firmware()
> 	kernel_read_file_from_path()
> 
> sysdata_synchronize_request(&cookie);
> 
> Doesn't look like what your asking for.

No, but its also a generic kernel read issue as I noted in my last
reply.

> >If we want to overhaul firmware loading support we need to figure
> >out  how to support case when a driver want to [asynchronously] request
> >firmware/config/blob and the rest of the system is not ready. Even if we
> >want kernel to do read/load the data we need userspace to tell kernel
> >when firmware partition is available, until then the kernel should not
> >fail the request.
> 
> I gather from Luis' blog post and comments that he is on the quest
> on removing userspace support completely.

No, I explained in my last proposed documentation patch series that we cannot
get rid of the usermode helper. Its not well understood why so I explained and
documented why. Best we can do is compartamentalize its uses.

The sysdata API's main goal rather is to provide a flexible API first,
compartamentalizing the usermode helper was secondary. But now it seems
I may just also add devm support too to help simplify code further.

What Dmitry notes is an existential issue with kernel_read_file_from_path()
and we need a common solution for it.

> Maybe this attempt here could be a step before. Step 1 would be
> changing request_firmware_nowait() to request_firmware_async so
> drivers don't have to come up with their own sync primitives, e.g.
> 
>   cookie = request_firmware_async()
>   fw_load_wait(cookie)

That's one of the features already part of async mechanism of the sysdata API :)

  Luis

[toc] | [prev] | [next] | [standalone]

#1453612 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

Hi Luis,

>> I was ignorant on all the nasty details around the firmware loading.
>> If I parse Luis' patches correctly they introduce an API which calls
>> kernel_read_file_from_path() asynchronously:
>>
>> sysdata_file_request_async(..., &cookie)
>>   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)
>>
>>   request_sysdata_file_work_fun()
>>     _sysdata_file_request()
>>       fw_get_filesystem_firmware()
>> 	kernel_read_file_from_path()
>>
>> sysdata_synchronize_request(&cookie);
>>
>> Doesn't look like what your asking for.
>
> No, but its also a generic kernel read issue as I noted in my last
> reply.

Okay, got it.

>>> If we want to overhaul firmware loading support we need to figure
>>> out  how to support case when a driver want to [asynchronously] request
>>> firmware/config/blob and the rest of the system is not ready. Even if we
>>> want kernel to do read/load the data we need userspace to tell kernel
>>> when firmware partition is available, until then the kernel should not
>>> fail the request.
>>
>> I gather from Luis' blog post and comments that he is on the quest
>> on removing userspace support completely.
>
> No, I explained in my last proposed documentation patch series that we cannot
> get rid of the usermode helper.

I stand corrected.

> Its not well understood why so I explained and documented why.

Obviously, I got lost somewhere there :)

> Best we can do is compartamentalize its uses.

Sounds like a plan.

> The sysdata API's main goal rather is to provide a flexible API first,
> compartamentalizing the usermode helper was secondary. But now it seems
> I may just also add devm support too to help simplify code further.

I missed the point that you plan to add usermode helper support to the 
sysdata API.

> What Dmitry notes is an existential issue with kernel_read_file_from_path()
> and we need a common solution for it.

Understood. I guess best thing to keep that discussion in the other 
subthread.

>> Maybe this attempt here could be a step before. Step 1 would be
>> changing request_firmware_nowait() to request_firmware_async so
>> drivers don't have to come up with their own sync primitives, e.g.
>>
>>   cookie = request_firmware_async()
>>   fw_load_wait(cookie)
>
> That's one of the features already part of async mechanism of the sysdata API :)

Yes, I realized that too :)

cheers,
daniel

Thanks for the feedback.

[toc] | [prev] | [next] | [standalone]

#1453617 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >The sysdata API's main goal rather is to provide a flexible API first,
> >compartamentalizing the usermode helper was secondary. But now it seems
> >I may just also add devm support too to help simplify code further.
> 
> I missed the point that you plan to add usermode helper support to
> the sysdata API.

I had no such plans, when I have asked folks so far about "hey are you
really in need for it, OK what for? " and "what extended uses do you
envision?" so I far I have not gotten any replies at all. So -- instead
sysdata currently ignores it.

  Luis

[toc] | [prev] | [next] | [standalone]

#1453624 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>>> The sysdata API's main goal rather is to provide a flexible API first,
>>> compartamentalizing the usermode helper was secondary. But now it seems
>>> I may just also add devm support too to help simplify code further.
>>
>> I missed the point that you plan to add usermode helper support to
>> the sysdata API.
>
> I had no such plans, when I have asked folks so far about "hey are you
> really in need for it, OK what for? " and "what extended uses do you
> envision?" so I far I have not gotten any replies at all. So -- instead
> sysdata currently ignores it.

So you argue for the remoteproc use case with 100+ MB firmware that if 
there is a way to load after pivot_root() (or other additional firmware 
partition shows up) then there is no need at all for usermode helper?

[toc] | [prev] | [next] | [standalone]

#1453647 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>>The sysdata API's main goal rather is to provide a flexible API first,
> >>>compartamentalizing the usermode helper was secondary. But now it seems
> >>>I may just also add devm support too to help simplify code further.
> >>
> >>I missed the point that you plan to add usermode helper support to
> >>the sysdata API.
> >
> >I had no such plans, when I have asked folks so far about "hey are you
> >really in need for it, OK what for? " and "what extended uses do you
> >envision?" so I far I have not gotten any replies at all. So -- instead
> >sysdata currently ignores it.
> 
> So you argue for the remoteproc use case with 100+ MB firmware that
> if there is a way to load after pivot_root() (or other additional
> firmware partition shows up) then there is no need at all for
> usermode helper?

No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
far I have only found using coccinelle grammar 2 explicit users, that's it. My
patch series (not yet merge) then annotates these as valid as I've verified
through their documentation they have some quirky requirement.

Other than these two drivers I'd like hear to valid requirements for it.

The existential issue is a real issue but it does not look impossible to
resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
stuff built-in firmware to avoid this issue, but it does not mean a solution
is not possible.

Remind me -- why can remoteproc not stuff the firmware in initramfs ?

Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
support per enum kernel_read_file_id. For instance we'd have one for
READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
would in turn be used as the system configurable deterministic file for
which to wait for to be present before enabling each enum kernel_read_file_id
type read.

Thoughts ?

  Luis

[toc] | [prev] | [next] | [standalone]

#1455652 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
>>> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>> So you argue for the remoteproc use case with 100+ MB firmware that
>> if there is a way to load after pivot_root() (or other additional
>> firmware partition shows up) then there is no need at all for
>> usermode helper?
>
> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> patch series (not yet merge) then annotates these as valid as I've verified
> through their documentation they have some quirky requirement.

I got that question wrong. It should read something like 'for the 
remoteproc 100+MB there is no need for the user help?'. I've gone 
through your patches and they make perfectly sense too. Maybe I can 
convince you to take a better version of my patch 3 into your queue. And 
I help you converting the exiting drivers. Obviously if you like my help 
at all.

> Other than these two drivers I'd like hear to valid requirements for it.
>
> The existential issue is a real issue but it does not look impossible to
> resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> stuff built-in firmware to avoid this issue, but it does not mean a solution
> is not possible.
>
> Remind me -- why can remoteproc not stuff the firmware in initramfs ?

I don't know. I was just bringing it up with the hope that Bjorn will 
defend it. It seems my tactics didn't work out :)

> Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> support per enum kernel_read_file_id. For instance we'd have one for
> READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> would in turn be used as the system configurable deterministic file for
> which to wait for to be present before enabling each enum kernel_read_file_id
> type read.
>
> Thoughts ?

Not sure if I get you here correctly. Is the 'system configurable 
deterministic file' is a knob which controlled by user space? Or it this 
something you define at compile time?

Hmm, so it would allow to decided to ask a userspace helper or load the 
firmware directly (to be more precised the kernel_read_file_id type). If 
yes, than it is what currently already have just integrated nicely into 
the new sysdata API.

cheers,
daniel

[toc] | [prev] | [next] | [standalone]

#1455879 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> 
> I accept all help and would be glad to make enhancements instead of
> the old API through new API. The biggest thing here first I think is
> adding devm support, that I think should address what seemed to be
> the need to add more code for a transformation into the API. I'd

I am confused. Why do we need devm support, given that devm is only
valid in probe() paths[*] and we do know that we do not want to load
firmware in probe() paths because it may cause blocking?

[*] Yes, I know there are calls to devm* outside of probe() but I am
pretty sure they are buggy unless they explicitly freed with devm* as
well and then there is no point. IN all other cases it is likely wrong
as it messes up with order of freeing resources.

Thanks.

-- 
Dmitry

[toc] | [prev] | [next] | [standalone]

#1455943 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Wed, Aug 03, 2016 at 09:18:21AM -0700, Dmitry Torokhov wrote:
> On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> > 
> > I accept all help and would be glad to make enhancements instead of
> > the old API through new API. The biggest thing here first I think is
> > adding devm support, that I think should address what seemed to be
> > the need to add more code for a transformation into the API. I'd
> 
> I am confused. Why do we need devm support, given that devm is only
> valid in probe() paths[*] and we do know that we do not want to load
> firmware in probe() paths because it may cause blocking?

Its a good point, I hadn't gone on to implement devm support on the sysdata API
yet here so this requirement was not known to me. This certainly would put a
limitation to the idea of using devm then to deal with the firmware for you,
given that not all users of firmware are on probe, and as you note we want to
by default avoid firmware calls on probe since init+probe are called serially
by default unless a driver is using the new async probe. Nevertheless, even if
we had userspace or the driver always asking for async probe, most users of the
firmware API are not on probe anyway, so the gains of using devm to help with
freeing the firmware for the driver on probe would be very limited.

With that in mind, in retrospect then the current sysdata approach to require a
callback for synchronous calls would seem to work around this issue and
generalize a solution given we'd have:

For the sync case:

const struct sysdata_file_desc sysdata_desc = {
	SYSDATA_DEFAULT_SYNC(driver_sync_req_cb, dev),
	.keep = false, /* not explicitly needed as default is false */
};
ret = sysdata_file_request();
...

Behind the scenes firmware_class would call driver_sync_req_cb(),
since that's where we know the firmware will be consumed and since
the driver has explicitly asked that it no longer needs to keep the
firmware around (keep == false), it will free it on behalf of the
driver.

Since current synchronous calls for firmware do not have a callback
this would mean a driver changing to the sysdata API if it wanted
to take advantage of this feature of letting firmware_class free
the firmware for you, you'd need a bit more code than before.

For the asynchronous case this is a bit different given that the
current async firmware API requires a callback, so if keep == false
on the async sysdata API we just remove the release_firmware()
calls when converting over.

Given this, other than the bikeshedding aspects [0] ("sysdata", "driver data",
"firmware), perhaps the sysdata API is done then.

[0] http://phk.freebsd.dk/sagas/bikeshed.html

> [*] Yes, I know there are calls to devm* outside of probe() but I am
> pretty sure they are buggy unless they explicitly freed with devm* as
> well and then there is no point.

Really ? If so that's good to know.. and it should mean grammar could
be used to hunt this down, specially since we have now some grammar
basics to help us check for calls on probe or init. On the grammar
we'd just only complain if a call was used not in a probe path.

> IN all other cases it is likely wrong
> as it messes up with order of freeing resources.

Good to know, thanks. Hopefully the above semantics of the driver
using keep should suffice. Which gets me to think, what if devm
had something similar to white-list uses outside of probe so that
if a keep (or another flag name) was set then its vetting that
the order of freeing of resources is understood and fine.

  Luis

[toc] | [prev] | [next] | [standalone]

#1455894 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Wed, Aug 03, 2016 at 08:57:09AM +0200, Daniel Wagner wrote:
> On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> >On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> >>On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >>>On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>So you argue for the remoteproc use case with 100+ MB firmware that
> >>if there is a way to load after pivot_root() (or other additional
> >>firmware partition shows up) then there is no need at all for
> >>usermode helper?
> >
> >No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> >far I have only found using coccinelle grammar 2 explicit users, that's it. My
> >patch series (not yet merge) then annotates these as valid as I've verified
> >through their documentation they have some quirky requirement.
> 
> I got that question wrong. It should read something like 'for the
> remoteproc 100+MB there is no need for the user help?'.

That's not a question for me but for those who say that the usermode helper
is needed for remoteproc, so far from what folks are saying it seems the only
reason for the usermodehelper was to try to avoid the deterministic issue,
but I suggested a way to resolve that without the usermode helper now so
would be curious to hear if there are any more reasons for it.

> I've gone
> through your patches and they make perfectly sense too. Maybe I can
> convince you to take a better version of my patch 3 into your queue.
> And I help you converting the exiting drivers. Obviously if you like
> my help at all.

I accept all help and would be glad to make enhancements instead of
the old API through new API. The biggest thing here first I think is
adding devm support, that I think should address what seemed to be
the need to add more code for a transformation into the API. I'd
personally only want to add that and be done with an introduction
of the sysdata API. Further changes IMHO are best done atomically
after that on top of it, but I'm happy to queue in the changes.

> >Other than these two drivers I'd like hear to valid requirements for it.
> >
> >The existential issue is a real issue but it does not look impossible to
> >resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> >stuff built-in firmware to avoid this issue, but it does not mean a solution
> >is not possible.
> >
> >Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> 
> I don't know. I was just bringing it up with the hope that Bjorn
> will defend it. It seems my tactics didn't work out :)

OK.

> >Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> >support per enum kernel_read_file_id. For instance we'd have one for
> >READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> >would in turn be used as the system configurable deterministic file for
> >which to wait for to be present before enabling each enum kernel_read_file_id
> >type read.
> >
> >Thoughts ?
> 
> Not sure if I get you here correctly. Is the 'system configurable
> deterministic file' is a knob which controlled by user space? Or it
> this something you define at compile time?

I meant at compile time on the kernel. So CONFIG_READ_READY_SENTINEL
or something like this, and it be a string, which if set then when
the kernel read APIs are used, then a new API could be introduced
that would *only* enable reading through once that sentinel has
been detected by the kernel to allowed through reads. Doing this
per mount / target filesystem is rather cumbersome given possible
overlaps in mounts and also pivot_root() being possible, so instead
targeting simply the fs/exec.c enum kernel_read_file_id would seem
more efficient and clean but we would need a decided upon set of
paths per enum kernel_read_file_id as base (or just one path per
enum kernel_read_file_id). For number of paths I mean the number
of target directories to look for the sentinel per enum kernel_read_file_id,
so for instance for READING_FIRMWARE perhaps just deciding on /lib/firmware/
would suffice, but if this supported multiple paths another option may be
for the sentinel to also be looked for in /lib/firmware/updates/,
/lib/firmware/" UTS_RELEASE -- etc. It would *stop* after finding one
sentinel on any of these paths.

If a system has has CONFIG_READ_READY_SENTINEL it would mean an agreed upon
system configuration has been decided so that at any point in time reads
against READING_FIRMWARE using a new kernel_read_file_from_path_sentinel()
(or something like it) would only allow the read to go through once
the sentinel has been found for READING_FIRMWARE on the agreed upon
paths.

The benefit of the sentintel approach is it avoids complexities with
pivot_root(), and makes the deterministic aspect of the target left
only to a system-configuration enabled target path / file.

This is just an idea. I'd like some FS folks to review.

> Hmm, so it would allow to decided to ask a userspace helper or load
> the firmware directly (to be more precised the kernel_read_file_id
> type). If yes, than it is what currently already have just
> integrated nicely into the new sysdata API.

Sorry, no, the above description is better of what I meant. This
actually would not need to go into the sysdata API, unless of
course we wanted it just as a new "feature" of it, but I don't
think that's needed unless it has some implications behind the
scenes. Given that firmware_class now uses a common core kernel
API for reading files kernel_read_file_from_path() we could
for instance add kernel_read_file_from_sentintel() and only
if CONFIG_READ_READY_SENTINEL() would it block and wait until
the sentinel clears. This should mean being able to make the
change for both the old API and the new proposed sysdata API.
Likewise for other kernel_read_file*() users -- they'd benefit
from it as well.

  Luis

[toc] | [prev] | [next] | [standalone]

#1455962 — Re: [RFC v0 7/8] Input: ims-pcu: use firmware_stat instead of completion

On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> On Wed, Aug 03, 2016 at 08:57:09AM +0200, Daniel Wagner wrote:
> > On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> > >On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> > >>On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> > >>>On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> > >>So you argue for the remoteproc use case with 100+ MB firmware that
> > >>if there is a way to load after pivot_root() (or other additional
> > >>firmware partition shows up) then there is no need at all for
> > >>usermode helper?
> > >
> > >No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> > >far I have only found using coccinelle grammar 2 explicit users, that's it. My
> > >patch series (not yet merge) then annotates these as valid as I've verified
> > >through their documentation they have some quirky requirement.
> > 
> > I got that question wrong. It should read something like 'for the
> > remoteproc 100+MB there is no need for the user help?'.
> 
> That's not a question for me but for those who say that the usermode helper
> is needed for remoteproc, so far from what folks are saying it seems the only
> reason for the usermodehelper was to try to avoid the deterministic issue,
> but I suggested a way to resolve that without the usermode helper now so
> would be curious to hear if there are any more reasons for it.
> 
> > I've gone
> > through your patches and they make perfectly sense too. Maybe I can
> > convince you to take a better version of my patch 3 into your queue.
> > And I help you converting the exiting drivers. Obviously if you like
> > my help at all.
> 
> I accept all help and would be glad to make enhancements instead of
> the old API through new API. The biggest thing here first I think is
> adding devm support, that I think should address what seemed to be
> the need to add more code for a transformation into the API. I'd
> personally only want to add that and be done with an introduction
> of the sysdata API. Further changes IMHO are best done atomically
> after that on top of it, but I'm happy to queue in the changes.
> 
> > >Other than these two drivers I'd like hear to valid requirements for it.
> > >
> > >The existential issue is a real issue but it does not look impossible to
> > >resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> > >stuff built-in firmware to avoid this issue, but it does not mean a solution
> > >is not possible.
> > >
> > >Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> > 
> > I don't know. I was just bringing it up with the hope that Bjorn
> > will defend it. It seems my tactics didn't work out :)
> 
> OK.
> 
> > >Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> > >support per enum kernel_read_file_id. For instance we'd have one for
> > >READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> > >would in turn be used as the system configurable deterministic file for
> > >which to wait for to be present before enabling each enum kernel_read_file_id
> > >type read.
> > >
> > >Thoughts ?
> > 
> > Not sure if I get you here correctly. Is the 'system configurable
> > deterministic file' is a knob which controlled by user space? Or it
> > this something you define at compile time?
> 
> I meant at compile time on the kernel. So CONFIG_READ_READY_SENTINEL
> or something like this, and it be a string, which if set then when
> the kernel read APIs are used, then a new API could be introduced
> that would *only* enable reading through once that sentinel has
> been detected by the kernel to allowed through reads. Doing this
> per mount / target filesystem is rather cumbersome given possible
> overlaps in mounts and also pivot_root() being possible, so instead
> targeting simply the fs/exec.c enum kernel_read_file_id would seem
> more efficient and clean but we would need a decided upon set of
> paths per enum kernel_read_file_id as base (or just one path per
> enum kernel_read_file_id). For number of paths I mean the number
> of target directories to look for the sentinel per enum kernel_read_file_id,
> so for instance for READING_FIRMWARE perhaps just deciding on /lib/firmware/
> would suffice, but if this supported multiple paths another option may be
> for the sentinel to also be looked for in /lib/firmware/updates/,
> /lib/firmware/" UTS_RELEASE -- etc. It would *stop* after finding one
> sentinel on any of these paths.
> 
> If a system has has CONFIG_READ_READY_SENTINEL it would mean an agreed upon
> system configuration has been decided so that at any point in time reads
> against READING_FIRMWARE using a new kernel_read_file_from_path_sentinel()
> (or something like it) would only allow the read to go through once
> the sentinel has been found for READING_FIRMWARE on the agreed upon
> paths.
> 
> The benefit of the sentintel approach is it avoids complexities with
> pivot_root(), and makes the deterministic aspect of the target left
> only to a system-configuration enabled target path / file.
> 
> This is just an idea. I'd like some FS folks to review.
> 
> > Hmm, so it would allow to decided to ask a userspace helper or load
> > the firmware directly (to be more precised the kernel_read_file_id
> > type). If yes, than it is what currently already have just
> > integrated nicely into the new sysdata API.
> 
> Sorry, no, the above description is better of what I meant. This
> actually would not need to go into the sysdata API, unless of
> course we wanted it just as a new "feature" of it, but I don't
> think that's needed unless it has some implications behind the
> scenes. Given that firmware_class now uses a common core kernel
> API for reading files kernel_read_file_from_path() we could
> for instance add kernel_read_file_from_sentintel() and only
> if CONFIG_READ_READY_SENTINEL() would it block and wait until
> the sentinel clears. This should mean being able to make the
> change for both the old API and the new proposed sysdata API.
> Likewise for other kernel_read_file*() users -- they'd benefit
> from it as well.

A file sentinel would implicate a file namespace thing being used on the
filesystem -- to me this just means the Linux distribution / system integrator
would add this per filesystem, but agree this is pretty hacky.  Furthermore
we'd wait forever if the Linux distribution / system integrator forgot to set
the sentinel file. That's not good. To avoid that a generic "root fs ready"
event could be sent from userspace to know when to clear stale reads... but if
that's going to be done best just replace all sentintels with a simple "root fs
ready" which would mean all reads from the kernel are ready. If we wanted
further granularity I suppose we could further have one event per enum
kernel_read_file_id, and a generic all-is-ready one.

To start off with then a simple event from userspace should suffice. But do keep
in mind that granularity might help given that a big iron system might have some
large array of disks to mount during bootup and that may take a long while, and
you likely want to read /lib/firmware way before that filesystem is ready.

Not sure if granularity fixated by enum kernel_read_file_id should suffice,
perhaps given its also enough for LSMs...

This indeed would mean a kernelspace and userspace change, but it would mean 
not having to deal with the usermode helper crap anymore.

Anyway -- these are just ideas, patches welcomed !

  Luis

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.kernel

csiph-web