Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1257533 > unrolled thread

[PATCH 00/10] merge_config misc reworks and testcases

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/10] merge_config misc reworks and testcases Olof Johansson <olof@lixom.net> - 2015-10-28 01:50 +0100
    [PATCH 09/10] merge_config.sh: allow single configs to be passed in on cmdline Olof Johansson <olof@lixom.net> - 2015-10-28 01:50 +0100
      Re: [PATCH 09/10] merge_config.sh: allow single configs to be passed  in on cmdline Darren Hart <dvhart@infradead.org> - 2015-10-28 08:30 +0100
    [PATCH 08/10] merge_config.sh: use trap for cleanup Olof Johansson <olof@lixom.net> - 2015-10-28 01:50 +0100
      Re: [PATCH 08/10] merge_config.sh: use trap for cleanup Darren Hart <dvhart@infradead.org> - 2015-10-28 08:20 +0100
      Re: [PATCH 08/10] merge_config.sh: use trap for cleanup Darren Hart <dvhart@infradead.org> - 2015-10-28 08:30 +0100
    Re: [PATCH 00/10] merge_config misc reworks and testcases Olof Johansson <olof@lixom.net> - 2015-10-28 01:50 +0100
    Re: [PATCH 00/10] merge_config misc reworks and testcases Darren Hart <dvhart@infradead.org> - 2015-10-28 06:10 +0100
      Re: [PATCH 00/10] merge_config misc reworks and testcases Bruce Ashfield <bruce.ashfield@windriver.com> - 2015-10-28 07:30 +0100
        Re: [PATCH 00/10] merge_config misc reworks and testcases Darren Hart <dvhart@infradead.org> - 2015-10-28 08:10 +0100

#1257533 — [PATCH 00/10] merge_config misc reworks and testcases

Hi,

Somewhat wide distribution list here, since I've added everyone who's
touched the script, with the presumption that those have been the major
users of it. Please make sure none of these changes break your use cases.

I've done some reworks of merge_config.sh. I was quite hesitant to start
this since there are no good ways to see if your changes break others
or not, so the first thing I did was to add some tests. I know this is
highly unorthodox so try not to panic. :-)

As far as what this series does is:

- Adds a way to pass in CONFIG_FOO=<value> on the command line, it gets
  treated as a single-entry fragment file

- The script now prints the warnings on stderr, and returns non-0 when
  something is encountered

- Optionally, it'll also return non-0 when a redundant entry is found. I
  presumed people rely on -r not being a failure so I did this separately

- CONFIG_FOO=n and "# CONFIG_FOO is not set" is now treated the same,
  and using the former doesn't cause an invalid warning when the results
  are checked at the end

- Slightly odd things happened if a fragment contains the same option
  twice: It'd produce a warning that was malformed. Now just ignore that
  and use only the latest value of said option.


-Olof

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1257534 — [PATCH 09/10] merge_config.sh: allow single configs to be passed in on cmdline

Treat CONFIG_FOO=..  on the command line the same way as a single-entry file would.

Signed-off-by: Olof Johansson <olof@lixom.net>
---
 scripts/kconfig/merge_config.sh | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
index b26c0ef..69463dd 100755
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -1,8 +1,8 @@
 #!/bin/sh
-#  merge_config.sh - Takes a list of config fragment values, and merges
-#  them one by one. Provides warnings on overridden values, and specified
-#  values that did not make it to the resulting .config file (due to missed
-#  dependencies or config symbol removal).
+#  merge_config.sh - Takes a list of config fragment filenames or configuration
+#  value, and merges them one by one. Provides warnings on overridden values,
+#  and specified values that did not make it to the resulting .config file
+#  (due to missed dependencies or config symbol removal).
 #
 #  Portions reused from kconf_check and generate_cfg:
 #  http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
@@ -108,7 +108,26 @@ fi
 
 MERGE_LIST=$*
 SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
-TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
+TMP_FILE=$(mktemp $OUTPUT/.tmp.config.XXXXXXXXXX)
+
+CLEAN_FILES=$TMP_FILE
+
+# Process cmdline configs into temporary fragments
+for ENTRY in $MERGE_LIST ; do
+	case $ENTRY in
+	CONFIG*)
+		FF=$(mktemp $OUTPUT/.temp.frag-cmdline.XXXXX)
+		CLEAN_FILES="$CLEAN_FILES $FF"
+		echo $ENTRY > $FF
+		NEW_LIST="$NEW_LIST $FF"
+		;;
+	*)
+		NEW_LIST="$NEW_LIST $ENTRY"
+		;;
+	esac
+done
+
+MERGE_LIST=$NEW_LIST
 
 echo "Using $INITFILE as base"
 cat $INITFILE > $TMP_FILE
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257741 — Re: [PATCH 09/10] merge_config.sh: allow single configs to be passed in on cmdline

On Wed, Oct 28, 2015 at 09:42:10AM +0900, Olof Johansson wrote:
> Treat CONFIG_FOO=..  on the command line the same way as a single-entry file would.
> 
> Signed-off-by: Olof Johansson <olof@lixom.net>
> ---
>  scripts/kconfig/merge_config.sh | 29 ++++++++++++++++++++++++-----
>  1 file changed, 24 insertions(+), 5 deletions(-)
> 
> diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> index b26c0ef..69463dd 100755
> --- a/scripts/kconfig/merge_config.sh
> +++ b/scripts/kconfig/merge_config.sh
> @@ -1,8 +1,8 @@
>  #!/bin/sh
> -#  merge_config.sh - Takes a list of config fragment values, and merges
> -#  them one by one. Provides warnings on overridden values, and specified
> -#  values that did not make it to the resulting .config file (due to missed
> -#  dependencies or config symbol removal).
> +#  merge_config.sh - Takes a list of config fragment filenames or configuration
> +#  value, and merges them one by one. Provides warnings on overridden values,
> +#  and specified values that did not make it to the resulting .config file
> +#  (due to missed dependencies or config symbol removal).
>  #
>  #  Portions reused from kconf_check and generate_cfg:
>  #  http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/kconf_check
> @@ -108,7 +108,26 @@ fi
>  
>  MERGE_LIST=$*
>  SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
> -TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
> +TMP_FILE=$(mktemp $OUTPUT/.tmp.config.XXXXXXXXXX)
> +
> +CLEAN_FILES=$TMP_FILE

Oh, here it is :-) Looks like this should come before 8/10 ?

But, content is good. Multiple files are needed in case CONFIG_* options are
intermixed with file fragments.

Signed-off-by: Darren Hart <dvhart@linux.intel.com>

-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257535 — [PATCH 08/10] merge_config.sh: use trap for cleanup

Use the trap to cleanup even on regular exit.

Signed-off-by: Olof Johansson <olof@lixom.net>
---
 scripts/kconfig/merge_config.sh | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
index 1945b2c..b26c0ef 100755
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -23,10 +23,12 @@
 EXITVAL=0
 
 clean_up() {
-	rm -f $TMP_FILE
+	if [ -n "$CLEAN_FILES" ] ; then
+		rm -f $CLEAN_FILES
+	fi
 	exit $EXITVAL
 }
-trap clean_up HUP INT TERM
+trap clean_up HUP INT TERM EXIT
 
 usage() {
 	echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
@@ -178,4 +180,4 @@ for CFG in $(sed -n "$SED_CONFIG_EXP" $TMP_FILE); do
 	fi >&2
 done
 
-clean_up
+# Note: clean_up will run here due to EXIT being trapped
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257731 — Re: [PATCH 08/10] merge_config.sh: use trap for cleanup

On Wed, Oct 28, 2015 at 09:42:09AM +0900, Olof Johansson wrote:
> Use the trap to cleanup even on regular exit.
> 
> Signed-off-by: Olof Johansson <olof@lixom.net>
> ---
>  scripts/kconfig/merge_config.sh | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> index 1945b2c..b26c0ef 100755
> --- a/scripts/kconfig/merge_config.sh
> +++ b/scripts/kconfig/merge_config.sh
> @@ -23,10 +23,12 @@
>  EXITVAL=0
>  
>  clean_up() {
> -	rm -f $TMP_FILE
> +	if [ -n "$CLEAN_FILES" ] ; then
> +		rm -f $CLEAN_FILES

Hrm... where did CLEAN_FILES come from? I dodn't see it in master and I don't
see it in patches 1-7. Did I miss it?

-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257738 — Re: [PATCH 08/10] merge_config.sh: use trap for cleanup

On Wed, Oct 28, 2015 at 09:42:09AM +0900, Olof Johansson wrote:
> Use the trap to cleanup even on regular exit.
> 
> Signed-off-by: Olof Johansson <olof@lixom.net>

Looks like this should trade spots in the series with 9 where CLEAN_FILES is
defined. Otherwise:

Reviewed-by: Darren Hart <dvhart@linux.intel.com>

> ---
>  scripts/kconfig/merge_config.sh | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
> index 1945b2c..b26c0ef 100755
> --- a/scripts/kconfig/merge_config.sh
> +++ b/scripts/kconfig/merge_config.sh
> @@ -23,10 +23,12 @@
>  EXITVAL=0
>  
>  clean_up() {
> -	rm -f $TMP_FILE
> +	if [ -n "$CLEAN_FILES" ] ; then
> +		rm -f $CLEAN_FILES
> +	fi
>  	exit $EXITVAL
>  }
> -trap clean_up HUP INT TERM
> +trap clean_up HUP INT TERM EXIT
>  
>  usage() {
>  	echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
> @@ -178,4 +180,4 @@ for CFG in $(sed -n "$SED_CONFIG_EXP" $TMP_FILE); do
>  	fi >&2
>  done
>  
> -clean_up
> +# Note: clean_up will run here due to EXIT being trapped
> -- 
> 2.1.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257537

On Wed, Oct 28, 2015 at 9:42 AM, Olof Johansson <olof@lixom.net> wrote:
> Hi,
>
> Somewhat wide distribution list here, since I've added everyone who's
> touched the script, with the presumption that those have been the major
> users of it. Please make sure none of these changes break your use cases.

Actually, I dropped a handful of cc:s on this repost, so this isn't
necessarily true. Also, since Yann didn't respond on the last posting
I'm sending these to you now, Michal. :)

Hopefully Darren, who seems to be one of the major users of
merge_config, can ack these if they pass in his environment.


-Olof
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257657

On Wed, Oct 28, 2015 at 09:42:01AM +0900, Olof Johansson wrote:
> Hi,
> 
> Somewhat wide distribution list here, since I've added everyone who's
> touched the script, with the presumption that those have been the major
> users of it. Please make sure none of these changes break your use cases.

+Bruce. I think you were going to test these with the linux-yocto tooling. Did
you get a chance to run that test? I'd like your thoughts on the two comments
below:

> 
> I've done some reworks of merge_config.sh. I was quite hesitant to start
> this since there are no good ways to see if your changes break others
> or not, so the first thing I did was to add some tests. I know this is
> highly unorthodox so try not to panic. :-)
> 
> As far as what this series does is:
> 
> - Adds a way to pass in CONFIG_FOO=<value> on the command line, it gets
>   treated as a single-entry fragment file
> 
> - The script now prints the warnings on stderr, and returns non-0 when
>   something is encountered

This one might impact linux-yocto usage, Bruce? That said, it seems like the
right thing to do. So I'd still like to see it go in, but we may need to plan to
update the dependent tooling to use it.

> 
> - Optionally, it'll also return non-0 when a redundant entry is found. I
>   presumed people rely on -r not being a failure so I did this separately
> 
> - CONFIG_FOO=n and "# CONFIG_FOO is not set" is now treated the same,
>   and using the former doesn't cause an invalid warning when the results
>   are checked at the end
> 
> - Slightly odd things happened if a fragment contains the same option
>   twice: It'd produce a warning that was malformed. Now just ignore that
>   and use only the latest value of said option.

This one will likely impact usage as well. linux-yocto does want to report when
there is an override, not as an error, but for informational purposes - "Where
does my option get clobbered?"



-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257700

On 10/28/2015 01:02 AM, Darren Hart wrote:
> On Wed, Oct 28, 2015 at 09:42:01AM +0900, Olof Johansson wrote:
>> Hi,
>>
>> Somewhat wide distribution list here, since I've added everyone who's
>> touched the script, with the presumption that those have been the major
>> users of it. Please make sure none of these changes break your use cases.
>
> +Bruce. I think you were going to test these with the linux-yocto tooling. Did
> you get a chance to run that test? I'd like your thoughts on the two comments
> below:

I ran some initial tests, but I didn't end up doing a full
update due to other constraints.

But in the next few weeks, I can do that update and full run.

>
>>
>> I've done some reworks of merge_config.sh. I was quite hesitant to start
>> this since there are no good ways to see if your changes break others
>> or not, so the first thing I did was to add some tests. I know this is
>> highly unorthodox so try not to panic. :-)
>>
>> As far as what this series does is:
>>
>> - Adds a way to pass in CONFIG_FOO=<value> on the command line, it gets
>>    treated as a single-entry fragment file
>>
>> - The script now prints the warnings on stderr, and returns non-0 when
>>    something is encountered
>
> This one might impact linux-yocto usage, Bruce? That said, it seems like the
> right thing to do. So I'd still like to see it go in, but we may need to plan to
> update the dependent tooling to use it.

I don't directly let the merge_config output be visible, but capture it
and then do more processing later. So while this may mean that I have
to update some wrappers to capture stderr, it shouldn't be a big deal.

>
>>
>> - Optionally, it'll also return non-0 when a redundant entry is found. I
>>    presumed people rely on -r not being a failure so I did this separately
>>
>> - CONFIG_FOO=n and "# CONFIG_FOO is not set" is now treated the same,
>>    and using the former doesn't cause an invalid warning when the results
>>    are checked at the end
>>
>> - Slightly odd things happened if a fragment contains the same option
>>    twice: It'd produce a warning that was malformed. Now just ignore that
>>    and use only the latest value of said option.
>
> This one will likely impact usage as well. linux-yocto does want to report when
> there is an override, not as an error, but for informational purposes - "Where
> does my option get clobbered?"

I haven't looked at the patches yet (and I will shortly), but if that
is within a single fragment, I can live with it going away, since it is
easy to check that outside of the merge script.

But if this is a redefinition between fragments, that's something different
and something that I capture and report to users, and yes, I
currently take it from the output of the merge_config run. If it goes
away, I'd have to recreate it somehow.

So if this can at least be maintained as enabled via a parameter, that
would be be ideal. Otherwise, I'll have to recreate the output some
other way.

Bruce

>
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1257728

On Wed, Oct 28, 2015 at 01:30:59AM -0400, Bruce Ashfield wrote:
> On 10/28/2015 01:02 AM, Darren Hart wrote:
> >On Wed, Oct 28, 2015 at 09:42:01AM +0900, Olof Johansson wrote:
> >>- The script now prints the warnings on stderr, and returns non-0 when
> >>   something is encountered
> >
> >This one might impact linux-yocto usage, Bruce? That said, it seems like the
> >right thing to do. So I'd still like to see it go in, but we may need to plan to
> >update the dependent tooling to use it.
> 
> I don't directly let the merge_config output be visible, but capture it
> and then do more processing later. So while this may mean that I have
> to update some wrappers to capture stderr, it shouldn't be a big deal.
> 
> >
> >>
> >>- Optionally, it'll also return non-0 when a redundant entry is found. I
> >>   presumed people rely on -r not being a failure so I did this separately
> >>
> >>- CONFIG_FOO=n and "# CONFIG_FOO is not set" is now treated the same,
> >>   and using the former doesn't cause an invalid warning when the results
> >>   are checked at the end
> >>
> >>- Slightly odd things happened if a fragment contains the same option
> >>   twice: It'd produce a warning that was malformed. Now just ignore that
> >>   and use only the latest value of said option.
> >
> >This one will likely impact usage as well. linux-yocto does want to report when
> >there is an override, not as an error, but for informational purposes - "Where
> >does my option get clobbered?"
> 
> I haven't looked at the patches yet (and I will shortly), but if that
> is within a single fragment, I can live with it going away, since it is
> easy to check that outside of the merge script.
> 
> But if this is a redefinition between fragments, that's something different
> and something that I capture and report to users, and yes, I
> currently take it from the output of the merge_config run. If it goes
> away, I'd have to recreate it somehow.
> 
> So if this can at least be maintained as enabled via a parameter, that
> would be be ideal. Otherwise, I'll have to recreate the output some
> other way.

It still reports redundancies across different fragments. It just fixes the grep
so it doesn't display two options from the same file.

-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web