Path: csiph.com!weretis.net!feeder6.news.weretis.net!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail
From: Stephane Chazelas <stephane.chazelas@gmail.com>
Newsgroups: gnu.bash.bug
Subject: Re: Locale not Obeyed by Parameter Expansion with Pattern Substitution
Date: Mon, 18 Nov 2019 20:46:26 +0000
Lines: 52
Approved: bug-bash@gnu.org
Message-ID: <mailman.1890.1574109997.13325.bug-bash@gnu.org>
References: <0acc4767-e87f-f163-b39e-f137effdfea2.ref@sbcglobal.net> <0acc4767-e87f-f163-b39e-f137effdfea2@sbcglobal.net> <20191118204626.33smprow5zae4apl@chaz.gmail.com>
NNTP-Posting-Host: lists.gnu.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Cc: bug-bash@gnu.org
To: Chris Carlen <crobc@sbcglobal.net>
Envelope-to: bug-bash@gnu.org
Content-Disposition: inline
In-Reply-To: <0acc4767-e87f-f163-b39e-f137effdfea2@sbcglobal.net>
User-Agent: NeoMutt/20171215
Precedence: list
Xref: csiph.com gnu.bash.bug:15610

2019-11-17 01:25:31 -0800, Chris Carlen:
[...]
> # write 'REVERSE PILCROW SIGN' to B, then repeat as above:
> printf -v B '\u204B'
> set -- ${B//?()/ }
> echo "${@@Q}"       #-> $'\342' $'\201' $'\213'
> 
> # NOTE: Since there is only one character (under the UTF-8 locale),
> # this should have set only the first positional parameter with the
> # character REVERSE PILCROW SIGN, not split it into bytes (AFAIK).
[...]

Yes, the question is where to resume searching after a match of
an empty string in ${var//pattern/replacement}.

Note that it's even worse in ksh93 where bash copied that syntax
from:

$ A=$'\u2048\u2048' ksh93 -c 'printf "%q\n" "${A//?()/:}"'
$':\u[2048]:\x81:\x88:\u[2048]:\x81:\x88:'

(here with ksh93u+)

Then there's the question of what

${B/$'\201'/}

should do. Should that $'\201' match the byte component of the encoding of
U+204B?

It seems to me that zsh's approach is best:

$ A=$'\u2048\201\u2048' zsh  -c "printf '%q\n' \"\${A//$'\201'/:}\""
⁈:⁈

That is replace that \201 byte, except when it's part of a
properly encoded character.

Compare with:

$ A=$'\u2048\201\u2048' bash  -c "printf '%q\n' \"\${A//$'\201'/:}\""
$'\342:\210:\342:\210'

$ A=$'\u2048\201\u2048' ksh93  -c "printf '%q\n' \"\${A//$'\201'/:}\""
$'\u[2048]:\x88:\u[2048]:\x88'

(or yash which can't deal with that \201 byte at all as it can't
form a valid character).

-- 
Stephane