Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #82569

Re: Character encoding conversion in wide string literals

Path csiph.com!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From "Alf P. Steinbach" <alf.p.steinbach@gmail.com>
Newsgroups comp.lang.c++
Subject Re: Character encoding conversion in wide string literals
Date Tue, 7 Dec 2021 18:59:53 +0100
Organization A noiseless patient Spider
Lines 38
Message-ID <soo7es$r6e$1@dont-email.me> (permalink)
References <sonh15$g9j$1@gioia.aioe.org> <sonvh8$vc9$1@dont-email.me> <soo39s$rsi$1@dont-email.me>
Mime-Version 1.0
Content-Type text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding 7bit
Injection-Date Tue, 7 Dec 2021 17:59:56 -0000 (UTC)
Injection-Info reader02.eternal-september.org; posting-host="082f8b1fdbf905e3e36c03435b0eccad"; logging-data="27854"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX180CCoD+pSTJif5VCOsGWh0"
User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.3.2
Cancel-Lock sha1:/gm7SvR02IrK9/yKpK0eth8iv/o=
In-Reply-To <soo39s$rsi$1@dont-email.me>
Content-Language en-US
Xref csiph.com comp.lang.c++:82569

Show key headers only | View raw

On 7 Dec 2021 17:48, James Kuyper wrote:
> On 12/7/21 10:44 AM, Alf P. Steinbach wrote:
> ...
>> aware: the main problem is that the C and C++ standards do not conform
>> to reality in their requirement that a `wchar_t` value should suffice to
>> encode all possible code points in the wide character set.
> 
> The purpose of the C and C++ standards is prescriptive, not descriptive.
> It's therefore missing the point to criticize them for not conforming to
> reality. Rather, you should say that some popular implementations fail
> to conform to the standards.

No, in this case it's the standard's fault. They failed to standardize 
existing practice and instead standardized a completely unreasonable 
requirement, given that 16-bit `wchar_t` was established as the API 
foundation in the most widely used OS on the platform, something that 
could not easily be changed. In particular this was the C standard 
committee: their choice here was as reasonable and practical as their 
choice of not supporting pointers outside of original (sub-) array.

It was idiotic. It was simple blunders. But inn both cases, as I recall, 
they tried to cover up the blunder by writing a rationale; they took the 
blunders to heart and made them into great obstacles, to not lose face.


>> In Windows wide text is UTF-16, with 16-bit `wchar_t`. Which means that
>> some emojis etc. that appear as a single character and constitute one
>> 21-bit code point, can become a pair of two `wchar_t` values, an UTF-16
>> "surrogate pair".
> 
> The C++ standard explicitly addresses that point, though the C standard
> does not.

Happy to hear that but some more specific information would be welcome.


- Alf

Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Character encoding conversion in wide string literals Juha Nieminen <nospam@thanks.invalid> - 2021-12-07 11:37 +0000
  Re: Character encoding conversion in wide string literals "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2021-12-07 16:44 +0100
    Re: Character encoding conversion in wide string literals James Kuyper <jameskuyper@alumni.caltech.edu> - 2021-12-07 11:48 -0500
      Re: Character encoding conversion in wide string literals "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2021-12-07 18:59 +0100
        Re: Character encoding conversion in wide string literals James Kuyper <jameskuyper@alumni.caltech.edu> - 2021-12-07 13:55 -0500
        Re: Character encoding conversion in wide string literals Öö Tiib <ootiib@hot.ee> - 2021-12-07 16:39 -0800
        Re: Character encoding conversion in wide string literals David Brown <david.brown@hesbynett.no> - 2021-12-08 09:18 +0100
    Re: Character encoding conversion in wide string literals Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-12-07 09:41 -0800
      Re: Character encoding conversion in wide string literals "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2021-12-07 19:07 +0100
        Re: Character encoding conversion in wide string literals Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-12-07 12:21 -0800
      Re: Character encoding conversion in wide string literals Paavo Helde <eesnimi@osa.pri.ee> - 2021-12-07 20:32 +0200
        Re: Character encoding conversion in wide string literals James Kuyper <jameskuyper@alumni.caltech.edu> - 2021-12-07 13:55 -0500
        Re: Character encoding conversion in wide string literals Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2021-12-07 12:26 -0800
  Re: Character encoding conversion in wide string literals James Kuyper <jameskuyper@alumni.caltech.edu> - 2021-12-07 11:38 -0500
    Re: Character encoding conversion in wide string literals Manfred <noname@add.invalid> - 2021-12-07 19:07 +0100

csiph-web