Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #4121 > unrolled thread

Escaping of line breaks for text replacement

Back to article view | Back to comp.programming

Contents

  Escaping of line breaks for text replacement Markus Elfring <Markus.Elfring@web.de> - 2014-03-02 14:00 +0100
    Re: Escaping of line breaks for text replacement jt@toerring.de (Jens Thoms Toerring) - 2014-03-02 15:49 +0000
    Re: Escaping of line breaks for text replacement dave.gma+news002@googlemail.com.invalid (Dave Sines) - 2014-03-02 16:52 +0000
    Re: Escaping of line breaks for text replacement Kaz Kylheku <kaz@kylheku.com> - 2014-03-02 18:11 +0000
    Re: Escaping of line breaks for text replacement Siri Cruz <chine.bleu@yahoo.com> - 2014-03-07 02:47 -0800

#4121 — Escaping of line breaks for text replacement

Hello,

I get the following result if I try out such a command on my openSUSE system.

elfring@Sonne:~> $SHELL --version && sed --version && LINE='echo ---' && $LINE \
&& DRAFT='template1.txt' && cat $DRAFT && $LINE \
&& INPUT='replacement1.txt' && cat $INPUT && XY=$(< $INPUT) && $LINE \
&& LANG=C sed "s/# placeholder/$XY/" $DRAFT
GNU bash, Version 4.2.45(1)-release (x86_64-suse-linux-gnu)
   ...
sed (GNU sed) 4.2.2
   ...
---
+++++++++++++
# placeholder
*************
---
ABC
XYZ
---
sed: -e expression #1, char 19: unterminated `s' command


I guess that the shown error message is an indication that I should add a bit of
escaping to the content of the variable "XY" which I inserted previously. I have
tried to achieve this with shell pattern substitution.

elfring@Sonne:~> XY=$(< $INPUT) && XY=${XY//\\\n/./} && echo $XY
ABC XYZ


I wonder why I do not get this approach to work so far with some additional
backslashes here. How should newlines be escaped for this purpose?

Would you like to share any further ideas?

Regards,
Markus

[toc] | [next] | [standalone]

#4122

In comp.unix.programmer Markus Elfring <Markus.Elfring@web.de> wrote:
> Hello,

> I get the following result if I try out such a command on my openSUSE system.

> elfring@Sonne:~> $SHELL --version && sed --version && LINE='echo ---' && $LINE \
> && DRAFT='template1.txt' && cat $DRAFT && $LINE \
> && INPUT='replacement1.txt' && cat $INPUT && XY=$(< $INPUT) && $LINE \
> && LANG=C sed "s/# placeholder/$XY/" $DRAFT
> GNU bash, Version 4.2.45(1)-release (x86_64-suse-linux-gnu)
>    ...
> sed (GNU sed) 4.2.2
>    ...
> ---
> +++++++++++++
> # placeholder
> *************
> ---
> ABC
> XYZ
> ---
> sed: -e expression #1, char 19: unterminated `s' command


> I guess that the shown error message is an indication that I should add a
> bit of escaping to the content of the variable "XY" which I inserted
> previously. I have tried to achieve this with shell pattern substitution.

Yes the problem is the linefeed in the replacement you pass to sed
in the end. You need to reolace the linefeeds read from the "re-
placements1.txt" file by '\\n' for sed to start to like them. One
way to do it would be e.g.

$SHELL --version && sed --version && LINE='echo ---' && $LINE &&
DRAFT='template1.txt' && cat $DRAFT && $LINE && INPUT='replacement1.txt' &&
cat $INPUT && XY=`sed ':a;N;$!ba;s/\n/\\\n/g' $INPUT` && $LINE && LANG=C sed
"s/# placeholder/$XY/" $DRAFT

With

XY=`sed ':a;N;$!ba;s/\n/\\\n/g' $INPUT`

you set XY to the files contents with linefeeds replaced by
'\\n' which is then acceptable to sed.

                               Regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [prev] | [next] | [standalone]

#4123

[ Followup-To: set to comp.unix.programmer ]

Markus Elfring <Markus.Elfring@web.de> wrote:
> Hello,
> 
> I get the following result if I try out such a command on my
> openSUSE system.
> 
> elfring@Sonne:~> $SHELL --version && sed --version &&
> LINE='echo ---' && $LINE && DRAFT='template1.txt' && cat $DRAFT &&
> $LINE && INPUT='replacement1.txt' && cat $INPUT && XY=$(< $INPUT) &&
> $LINE && LANG=C sed "s/# placeholder/$XY/" $DRAFT

> GNU bash, Version 4.2.45(1)-release (x86_64-suse-linux-gnu)
>   ...
> sed (GNU sed) 4.2.2
>   ...
> ---
> +++++++++++++
> # placeholder
> *************
> ---
> ABC
> XYZ
> ---
> sed: -e expression #1, char 19: unterminated `s' command

DRAFT=template1.txt
REPLACEMENT=replacement1.txt

sed '/^# placeholder/{
r '"$REPLACEMENT"'
d
}' "$DRAFT"

Note the change from single to double quotes and back -- the script is
composed of three adjacent strings which will be concatenated by the
shell to form a single argument to sed.

> 
> I guess that the shown error message is an indication that I should
> add a bit of escaping to the content of the variable "XY" which I
> inserted previously. I have tried to achieve this with shell pattern
> substitution.
> 
> elfring@Sonne:~> XY=$(< $INPUT) && XY=${XY//\\\n/./} && echo $XY
> ABC XYZ
> 
> I wonder why I do not get this approach to work so far with some
> additional backslashes here. How should newlines be escaped for
> this purpose?

Use the shell's backslash-escape expansion ($'what\never').

It's not just newlines, any characters in the replacement text which
match sed's s command delimiter ('/' by default) will also need to
be escaped.

  XY=$(< "$REPLACEMENT" )
  XY=${XY//$'\n'/$'\\\n'}
  XY=${XY//\//\\\/}

  sed "s/# placeholder/$XY/" "$DRAFT"

[toc] | [prev] | [next] | [standalone]

#4124

["Followup-To:" header set to comp.unix.programmer.]
On 2014-03-02, Markus Elfring <Markus.Elfring@web.de> wrote:
> Hello,
>
> I get the following result if I try out such a command on my openSUSE system.
>
> elfring@Sonne:~> $SHELL --version && sed --version && LINE='echo ---' && $LINE \
> && DRAFT='template1.txt' && cat $DRAFT && $LINE \
> && INPUT='replacement1.txt' && cat $INPUT && XY=$(< $INPUT) && $LINE \
> && LANG=C sed "s/# placeholder/$XY/" $DRAFT
> GNU bash, Version 4.2.45(1)-release (x86_64-suse-linux-gnu)
>    ...
> sed (GNU sed) 4.2.2
>    ...
> ---
> +++++++++++++
> # placeholder
> *************
> ---
> ABC
> XYZ
> ---
> sed: -e expression #1, char 19: unterminated `s' command

Unix already has a mechanism for interpolating variables into multi-line
text, called a "here document".

Here, it behooves us to use it, rather than reach for sed tricks
to do variable substitution.

Live demo:

$ cat template.txt 
my terminal is $TERM
the number of lines is $LINES
the number of colums is $COLUMNS

Now: dynamically wrap the here-doc syntax around the contents of template.txt
and evaluate it:

$ eval "cat <<EOF
$(cat template.txt)
EOF"
my terminal is xterm
the number of lines is 44
the number of colums is 80

You can have newlines in the variables that are substituted.

If you put the here-doc prologue and epilogue "cat <<EOF" and "EOF" the
template itself, then it's just a straight eval "$(cat template.txt)".

Remember to escape dollar signs in the here doc. It has to follow the
rules for a here doc.

If you ever do use sed for variable substitution have a clear delimiter on the
variables.  For instance, call it @REPLACEMENT@, not replacement,
for reasons which should be obvious.

[toc] | [prev] | [next] | [standalone]

#4128

> I wonder why I do not get this approach to work so far with some additional
> backslashes here. How should newlines be escaped for this purpose?
> 
> Would you like to share any further ideas?

Don't use sed if sed doesn't do what you want. Use tclsh, perl, etc

@ cat <<':eof:'>/tmp/a
abc def pqr stu
ijk mno abc
def mno def
:eof:


@ tclsh /dev/stdin /tmp/a <<':eof:'

set ch [open [lindex $argv 0]]
set file [read $ch]
close $ch

#   Change
#       abc def pqr stu
#       ijk mno abc
#       def mno def
#   to
#       abc def pqr stu
#       ijk mno + mno def

regsub -all {abc\ndef} $file {+} file

#   Change
#       abc def pqr stu
#       ijk mno + mno def
#   to
#       X X pqr stu
#       ijk mno + mno X

regsub -all {[a-f]+} $file {X} file

set ch [open [lindex $argv 0] w]
puts $ch $file
close $ch

:eof:

-- 
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted.
'I desire mercy, not sacrifice.'

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming

csiph-web