Re: Escaping of line breaks for text replacement

From	dave.gma+news002@googlemail.com.invalid (Dave Sines)
Newsgroups	comp.unix.programmer
Subject	Re: Escaping of line breaks for text replacement
Date	2014-03-06 22:59 +0000
Organization	A noiseless patient Spider
Message-ID	<k7bnuaxen3.ln2@perseus.wenlock-data.co.uk> (permalink)
References	(2 earlier) <bnj8u3FhmfmU1@mid.individual.net> <17peuaxd22.ln2@perseus.wenlock-data.co.uk> <bnr2u1F83b4U1@mid.individual.net> <6msmuax4g2.ln2@perseus.wenlock-data.co.uk> <bns1opFepjlU1@mid.individual.net>

Show all headers | View raw

Markus Elfring <Markus.Elfring@web.de> wrote:
>> Using some dummy input files derived from elsewhere in this thread and a
>> Makefile modelled on the above fragment, I'm seeing inconsistent results
>> when make invokes different versions of bash (and bash-as-/bin/sh).
> 
> Thanks for your interest and feedback.
> 
> Would you like to tell a bit more about the bash versions you try out?

4.3.0, 4.2.37, 4.2.20, 4.1.9, 4.0.35, 3.2.48, 3.1.17, 2.05b.7.

Here's a demo Makefile (it creates r.txt and t.txt in the working directory)

#v+
SED = sed

REPL = r.txt
TMPL = t.txt

ESCAPING1 = XY=$$(< $(REPL) )                 \
        && echo $$BASH_VERSION                \
        && XY=$${XY//$$'\n'/$$'\\\\\\n'}      \
        && $(SED) "s%\# placeholder%$${XY}%"

ESCAPING2 = XY=$$(< $(REPL) )                 \
        && echo $$BASH_VERSION                \
        && $(SED) "s%\# placeholder%$${XY//$$'\n'/$$'\\\\\\n'}%"

works:
	$(ESCAPING1) $(TMPL)

fails:
	$(ESCAPING2) $(TMPL)

$(REPL):
	printf 'ABC\nXYZ\n' > $(REPL)

$(TMPL):
	printf '++++\n# placeholder\n++++\n' > $(TMPL)

.SILENT:
#v-

/bin/sh is a symlink to bash.  /bin/bash and /bin/bash-4.3 are bash
4.3.0(1)-release.  make is GNU Make 4.0.  sed is GNU sed 4.2.2.

$ make works
4.3.0(1)-release
++++
ABC\
XYZ
++++
$ make fails
4.3.0(1)-release
sed: -e expression #1, char 19: unterminated `s' command
Makefile:20: recipe for target 'fails' failed
make: *** [fails] Error 1

Note that the "fails" target works when SHELL is not /bin/sh:

$ make SHELL=/bin/bash-4.3 fails
4.3.0(1)-release
++++
ABC\
XYZ
++++

With /bin/sh as a symlink to /bin/bash-4.2.37.

$ make works
4.2.37(1)-release
++++
ABC\
XYZ
++++
$ make fails
4.2.37(1)-release
++++
ABC\nXYZ
++++
$ make SHELL=/bin/bash-4.2.37 fails
4.2.37(1)-release
++++
ABC\nXYZ
++++

Bash versions 3.1.17, 3.2.48, 4.0.35, 4.1.9, and 4.2.20 match the behaviour
of 4.2.37.

With /bin/sh linked to bash 2.05b the output from the "fails" target
target differs slightly:

$ make works
2.05b.0(1)-release
++++
ABC\
XYZ
++++
$ make fails
2.05b.0(1)-release
++++
ABC'\n'XYZ
++++

>> Moving the shell substitution with escape expansions out of the double-
>> quoted string seems to fix this.
> 
> I hoped that I could avoid an intermediate variable assignment. I am
> surprised that I might stumble on a portability issue here.

Is awk acceptable?  I *think* the following fragment should work in your
Makefile (<https://systeme.lip6.fr/pipermail/cocci/2014-March/000680.html>).

#v+
AWKPROC = awk '                                                         \
        NR == FNR { sub(/[|]/, " ") ; s = s n $$0 ; n = "\\\n" ; next } \
        /\# Alternation placeholder/ { $$0 = s }                        \
        { print }'

$(LIST_PATTERN): $(RESULT_FUNCTIONS_WITH_PREFIX)
	$(AWKPROC) $< $(LIST_PATTERN_TEMPLATE) > $@

$(PATCH_PATTERN): $(RESULT_FUNCTIONS_WITH_PREFIX)
	$(AWKPROC) $< $(PATCH_PATTERN_TEMPLATE) > $@
#v-

Change the assignment 'n = "\\\n"' to 'n = "\n"' if you don't want the
newlines within the substitution to be escaped in the output files.

Thread

Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-02 14:00 +0100
  Re: Escaping of line breaks for text replacement jt@toerring.de (Jens Thoms Toerring) - 2014-03-02 15:49 +0000
    Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-03 08:38 +0100
  Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-02 16:52 +0000
    Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-02 17:53 +0000
      Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-02 20:27 +0000
        Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-03 16:17 +0000
          Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-03 17:44 +0000
            Re: Escaping of line breaks for text replacement Geoff Clare <geoff@clare.See-My-Signature.invalid> - 2014-03-04 13:30 +0000
    Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-02 20:50 +0100
      Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-03 16:52 +0000
    Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-03 09:24 +0100
    Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-03 12:55 +0100
      Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-03 17:02 +0000
        Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-06 12:02 +0100
          Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-06 18:51 +0000
            Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-06 20:48 +0100
              Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-06 22:59 +0000
                Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-07 06:38 +0000
                Re: Escaping of line breaks for text replacement Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-03-07 13:33 +0000
                Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-07 15:08 +0000
                Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-07 11:23 +0100
                Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-07 14:43 +0000
                Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-07 17:23 +0100
                Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-07 19:53 +0000
                Bash 4.3.0 escape expansion (was: Re: Escaping of line breaks for text replacement) dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-08 12:26 +0000
                Re: Bash 4.3.0 escape expansion (was: Re: Escaping of line breaks for text replacement) Kaz Kylheku <kaz@kylheku.com> - 2014-03-08 20:05 +0000
                Re: Bash 4.3.0 escape expansion Ben Bacarisse <ben.usenet@bsb.me.uk> - 2014-03-09 03:48 +0000
                Re: Bash 4.3.0 escape expansion dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-09 13:21 +0000
  Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-02 18:11 +0000
    Re: Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-02 20:08 +0100
      Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-02 20:35 +0000
        Re: Which content should be escaped? Markus Elfring <Markus.Elfring@web.de> - 2014-03-03 10:30 +0100
          Re: Which content should be escaped? dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-04 14:04 +0000
            Re: Which content should be escaped? Kaz Kylheku <kaz@kylheku.com> - 2014-03-04 17:32 +0000
  Re: Escaping of line breaks for text replacement Siri Cruz <chine.bleu@yahoo.com> - 2014-03-07 02:47 -0800

csiph-web