Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1482067 > unrolled thread

gen_initramfs_list.sh escaping problem or stale dependency file?

Back to article view | Back to linux.kernel

Contents

  gen_initramfs_list.sh escaping problem or stale dependency file? Florian Fainelli <f.fainelli@gmail.com> - 2016-09-13 02:20 +0200
    Re: gen_initramfs_list.sh escaping problem or stale dependency file? Michal Marek <mmarek@suse.com> - 2016-09-13 09:30 +0200
      Re: gen_initramfs_list.sh escaping problem or stale dependency file? Florian Fainelli <f.fainelli@gmail.com> - 2016-09-13 19:30 +0200
      Re: gen_initramfs_list.sh escaping problem or stale dependency file? Florian Fainelli <f.fainelli@gmail.com> - 2016-09-19 22:10 +0200
        Re: gen_initramfs_list.sh escaping problem or stale dependency file? Michal Marek <mmarek@suse.com> - 2016-09-23 09:10 +0200
          [PATCH] initramfs: Escape colons in depfile Michal Marek <mmarek@suse.com> - 2016-09-23 10:40 +0200

#1482067 — gen_initramfs_list.sh escaping problem or stale dependency file?

Hi,

I have a root filesystem embedding filenames that look like these:

/lib/data/<vid>:<pid>

these are essentially files that can be matched against an USB
vendor/product id in an easy way.

Now, the fun part is that this is only a problem when doing the
following (using OpenWrt/LEDE as a build system):

1:
- set CONFIG_INITRAMFS_SOURCE=""
- build kernel modules
- build my user-space tools
- build the kernel image
- reconfigure the kernel to now use an initramfs
- build the kernel w/ initramfs

and then back to step 1 with the kernel build, would I hit this error:

usr/Makefile:64: *** multiple target patterns.  Stop.

which comes from usr/.initramfs_data.cpio.d containing these files
without escaping:

deps_initramfs := ./scripts/gen_initramfs_list.sh \
/exp00/fainelli/openwrt/trunk/build_dir/target-arm-linux-gnueabihf/root-brcmstb
\
/exp00/fainelli/openwrt/trunk/build_dir/target-arm-linux-gnueabihf/root-brcmstb/lib
\
...

/exp00/fainelli/openwrt/trunk/build_dir/target-arm-linux-gnueabihf/root-brcmstb/lib/network/wwan
\
/exp00/fainelli/openwrt/trunk/build_dir/target-arm-linux-gnueabihf/root-brcmstb/lib/network/wwan/19d2:0063
\

Which sorts of make sense here because the file name contains a ":"
which is not escaped, so GNU Make tries to interpret it.

Now the part that does not quite make sense to me is why this file is
even relevant here considering that the first thing we do is set
CONFIG_INITRAMFS_SOURCE="" to disable the initramfs basically.

Any clues what could be wrong here? I am happy to provide any build
drops you may need to reproduce that.

Thanks!
-- 
Florian

[toc] | [next] | [standalone]

#1482202

On Mon, Sep 12, 2016 at 05:12:15PM -0700, Florian Fainelli wrote:
> Hi,
> 
> I have a root filesystem embedding filenames that look like these:
> 
> /lib/data/<vid>:<pid>
> 
> these are essentially files that can be matched against an USB
> vendor/product id in an easy way.
> 
> Now, the fun part is that this is only a problem when doing the
> following (using OpenWrt/LEDE as a build system):
> 
> 1:
> - set CONFIG_INITRAMFS_SOURCE=""
> - build kernel modules
> - build my user-space tools
> - build the kernel image
> - reconfigure the kernel to now use an initramfs
> - build the kernel w/ initramfs
> 
> and then back to step 1 with the kernel build, would I hit this error:
> 
> usr/Makefile:64: *** multiple target patterns.  Stop.
[...]
> Which sorts of make sense here because the file name contains a ":"
> which is not escaped, so GNU Make tries to interpret it.
> 
> Now the part that does not quite make sense to me is why this file is
> even relevant here considering that the first thing we do is set
> CONFIG_INITRAMFS_SOURCE="" to disable the initramfs basically.

It is possible that we read usr/Makefile twice for some reason. But the
real problem is the lack of escaping. Can you try the following
(untested) patch?


diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index 17fa901418ae..5d3188e74101 100755
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -97,7 +97,10 @@ print_mtime() {
 }
 
 list_parse() {
-	[ ! -L "$1" ] && echo "$1 \\" || :
+	if [ -L "$1" ]; then
+		return
+	fi
+	echo "$1" | sed 's/\([:%]\)/\\\1/g; s/$/ \\/'
 }
 
 # for each file print a line in following format

Thanks,
Michal

[toc] | [prev] | [next] | [standalone]

#1482662

On 09/13/2016 12:24 AM, Michal Marek wrote:
> On Mon, Sep 12, 2016 at 05:12:15PM -0700, Florian Fainelli wrote:
>> Hi,
>>
>> I have a root filesystem embedding filenames that look like these:
>>
>> /lib/data/<vid>:<pid>
>>
>> these are essentially files that can be matched against an USB
>> vendor/product id in an easy way.
>>
>> Now, the fun part is that this is only a problem when doing the
>> following (using OpenWrt/LEDE as a build system):
>>
>> 1:
>> - set CONFIG_INITRAMFS_SOURCE=""
>> - build kernel modules
>> - build my user-space tools
>> - build the kernel image
>> - reconfigure the kernel to now use an initramfs
>> - build the kernel w/ initramfs
>>
>> and then back to step 1 with the kernel build, would I hit this error:
>>
>> usr/Makefile:64: *** multiple target patterns.  Stop.
> [...]
>> Which sorts of make sense here because the file name contains a ":"
>> which is not escaped, so GNU Make tries to interpret it.
>>
>> Now the part that does not quite make sense to me is why this file is
>> even relevant here considering that the first thing we do is set
>> CONFIG_INITRAMFS_SOURCE="" to disable the initramfs basically.
> 
> It is possible that we read usr/Makefile twice for some reason. But the
> real problem is the lack of escaping. Can you try the following
> (untested) patch?

This patch works for me:

Tested-by: Florian Fainelli <f.fainelli@gmail.com>

Kind of surprising that this has not showed up before, but maybe people
don't really go through the same steps while re-configuring/re-building
their kernels

Thanks Michal!

> 
> 
> diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
> index 17fa901418ae..5d3188e74101 100755
> --- a/scripts/gen_initramfs_list.sh
> +++ b/scripts/gen_initramfs_list.sh
> @@ -97,7 +97,10 @@ print_mtime() {
>  }
>  
>  list_parse() {
> -	[ ! -L "$1" ] && echo "$1 \\" || :
> +	if [ -L "$1" ]; then
> +		return
> +	fi
> +	echo "$1" | sed 's/\([:%]\)/\\\1/g; s/$/ \\/'
>  }
>  
>  # for each file print a line in following format
> 
> Thanks,
> Michal
> 


-- 
Florian

[toc] | [prev] | [next] | [standalone]

#1486829

On 09/13/2016 12:24 AM, Michal Marek wrote:
> On Mon, Sep 12, 2016 at 05:12:15PM -0700, Florian Fainelli wrote:
>> Hi,
>>
>> I have a root filesystem embedding filenames that look like these:
>>
>> /lib/data/<vid>:<pid>
>>
>> these are essentially files that can be matched against an USB
>> vendor/product id in an easy way.
>>
>> Now, the fun part is that this is only a problem when doing the
>> following (using OpenWrt/LEDE as a build system):
>>
>> 1:
>> - set CONFIG_INITRAMFS_SOURCE=""
>> - build kernel modules
>> - build my user-space tools
>> - build the kernel image
>> - reconfigure the kernel to now use an initramfs
>> - build the kernel w/ initramfs
>>
>> and then back to step 1 with the kernel build, would I hit this error:
>>
>> usr/Makefile:64: *** multiple target patterns.  Stop.
> [...]
>> Which sorts of make sense here because the file name contains a ":"
>> which is not escaped, so GNU Make tries to interpret it.
>>
>> Now the part that does not quite make sense to me is why this file is
>> even relevant here considering that the first thing we do is set
>> CONFIG_INITRAMFS_SOURCE="" to disable the initramfs basically.
> 
> It is possible that we read usr/Makefile twice for some reason. But the
> real problem is the lack of escaping. Can you try the following
> (untested) patch?

Can you submit an official patch for this? Thanks a lot!

> 
> 
> diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
> index 17fa901418ae..5d3188e74101 100755
> --- a/scripts/gen_initramfs_list.sh
> +++ b/scripts/gen_initramfs_list.sh
> @@ -97,7 +97,10 @@ print_mtime() {
>  }
>  
>  list_parse() {
> -	[ ! -L "$1" ] && echo "$1 \\" || :
> +	if [ -L "$1" ]; then
> +		return
> +	fi
> +	echo "$1" | sed 's/\([:%]\)/\\\1/g; s/$/ \\/'
>  }
>  
>  # for each file print a line in following format
> 
> Thanks,
> Michal
> 


-- 
Florian

[toc] | [prev] | [next] | [standalone]

#1489795

On 2016-09-19 22:00, Florian Fainelli wrote:
> On 09/13/2016 12:24 AM, Michal Marek wrote:
>> On Mon, Sep 12, 2016 at 05:12:15PM -0700, Florian Fainelli wrote:
>>> Hi,
>>>
>>> I have a root filesystem embedding filenames that look like these:
>>>
>>> /lib/data/<vid>:<pid>
>>>
>>> these are essentially files that can be matched against an USB
>>> vendor/product id in an easy way.
>>>
>>> Now, the fun part is that this is only a problem when doing the
>>> following (using OpenWrt/LEDE as a build system):
>>>
>>> 1:
>>> - set CONFIG_INITRAMFS_SOURCE=""
>>> - build kernel modules
>>> - build my user-space tools
>>> - build the kernel image
>>> - reconfigure the kernel to now use an initramfs
>>> - build the kernel w/ initramfs
>>>
>>> and then back to step 1 with the kernel build, would I hit this error:
>>>
>>> usr/Makefile:64: *** multiple target patterns.  Stop.
>> [...]
>>> Which sorts of make sense here because the file name contains a ":"
>>> which is not escaped, so GNU Make tries to interpret it.
>>>
>>> Now the part that does not quite make sense to me is why this file is
>>> even relevant here considering that the first thing we do is set
>>> CONFIG_INITRAMFS_SOURCE="" to disable the initramfs basically.
>>
>> It is possible that we read usr/Makefile twice for some reason. But the
>> real problem is the lack of escaping. Can you try the following
>> (untested) patch?
> 
> Can you submit an official patch for this? Thanks a lot!

The % escape is wrong. I'm trying to fix it or drop this escape.

Michal

[toc] | [prev] | [next] | [standalone]

#1489845 — [PATCH] initramfs: Escape colons in depfile

Special characters are problematic in depfiles, but we can fix colons
easily.

Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
---
 scripts/gen_initramfs_list.sh | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index 17fa901418ae..0055b07b03b6 100755
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -97,7 +97,10 @@ print_mtime() {
 }
 
 list_parse() {
-	[ ! -L "$1" ] && echo "$1 \\" || :
+	if [ -L "$1" ]; then
+		return
+	fi
+	echo "$1" | sed 's/:/\\:/g; s/$/ \\/'
 }
 
 # for each file print a line in following format
-- 
2.6.6

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web