Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #67678 > unrolled thread

How to get standard paths?!?

Back to article view | Back to comp.lang.c++

Contents

  How to get standard paths?!? Szyk Cech <szykcech@spoko.pl> - 2019-08-17 13:58 +0200
    Re: How to get standard paths?!? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-17 14:02 +0200
      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 14:05 +0200
        Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 15:11 +0200
          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 15:59 +0200
            Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 16:20 +0200
              Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 16:28 +0200
                Re: How to get standard paths?!? Ralf Goertz <me@myprovider.invalid> - 2019-08-17 16:37 +0200
                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 16:54 +0200
                    Re: How to get standard paths?!? Paavo Helde <myfirstname@osa.pri.ee> - 2019-08-17 18:37 +0300
                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 17:54 +0200
                        Re: How to get standard paths?!? Paavo Helde <myfirstname@osa.pri.ee> - 2019-08-17 19:55 +0300
                          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 19:15 +0200
                            Re: How to get standard paths?!? Paavo Helde <myfirstname@osa.pri.ee> - 2019-08-17 20:39 +0300
                              Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 19:44 +0200
                                Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 20:10 +0200
                                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 08:58 +0200
                                    Re: How to get standard paths?!? Bart <bc@freeuk.com> - 2019-08-18 10:16 +0100
                                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 11:27 +0200
                              Re: How to get standard paths?!? Bo Persson <bo@bo-persson.se> - 2019-08-17 21:07 +0200
                                Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 21:41 +0200
                    Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 17:40 +0200
                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 17:52 +0200
                        Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 18:00 +0200
                          Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 18:01 +0200
                            Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 18:26 +0200
                              Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 20:20 +0200
                                Re: How to get standard paths?!? Keith Thompson <kst-u@mib.org> - 2019-08-17 16:33 -0700
                                  Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 11:34 +0200
                                Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 08:48 +0200
                                  Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 11:34 +0200
                                    Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 12:57 +0200
                                      Re: How to get standard paths?!? Bart <bc@freeuk.com> - 2019-08-18 13:07 +0100
                                        Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 15:49 +0200
                                        Re: How to get standard paths?!? Bo Persson <bo@bo-persson.se> - 2019-08-18 20:33 +0200
                                          Re: How to get standard paths?!? Bart <bc@freeuk.com> - 2019-08-18 20:51 +0100
                                            Re: How to get standard paths?!? Bo Persson <bo@bo-persson.se> - 2019-08-19 00:04 +0200
                                              Re: How to get standard paths?!? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-25 05:35 -0700
                                                Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-25 15:02 +0200
                                                  Re: How to get standard paths?!? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-25 06:38 -0700
                                                    Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-25 15:46 +0200
                                                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-25 15:54 +0200
                                            Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-19 08:52 +0200
                                              Re: How to get standard paths?!? "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> - 2019-08-19 09:32 +0200
                                                Re: How to get standard paths?!? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-22 12:37 -0700
                                              Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-19 10:26 +0200
                        Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 18:00 +0200
                          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 18:06 +0200
                            Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 20:24 +0200
                              Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 08:49 +0200
                                Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 11:46 +0200
                                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 12:59 +0200
                        Re: How to get standard paths?!? scott@slp53.sl.home (Scott Lurndal) - 2019-08-18 15:40 +0000
                          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 17:44 +0200
                    Re: How to get standard paths?!? scott@slp53.sl.home (Scott Lurndal) - 2019-08-18 15:38 +0000
                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 17:46 +0200
                Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 17:56 +0200
                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 18:05 +0200
                    Re: How to get standard paths?!? Hergen Lehmann <hlehmann.expires.5-11@snafu.de> - 2019-08-17 19:47 +0200
                      Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 20:46 +0200
                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 08:54 +0200
                        Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 12:07 +0200
                          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 13:05 +0200
                            Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 14:48 +0200
                              Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 15:51 +0200
                                Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 16:43 +0200
                                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 17:02 +0200
                    Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 20:41 +0200
                      Re: How to get standard paths?!? Keith Thompson <kst-u@mib.org> - 2019-08-17 16:46 -0700
                        Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 12:22 +0200
                      Re: How to get standard paths?!? Robert Wessel <robertwessel2@yahoo.com> - 2019-08-17 22:57 -0500
                      Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 08:57 +0200
                        Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-18 12:24 +0200
                          Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 13:07 +0200
                            Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-18 13:13 +0200
            Re: How to get standard paths?!? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-17 11:22 -0700
              Re: How to get standard paths?!? Bo Persson <bo@bo-persson.se> - 2019-08-17 21:21 +0200
                Re: How to get standard paths?!? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-08-17 22:25 -0700
                Re: How to get standard paths?!? Robert Wessel <robertwessel2@yahoo.com> - 2019-08-19 01:19 -0500
                  Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-19 08:25 +0200
    Re: How to get standard paths?!? Bonita Montero <Bonita.Montero@gmail.com> - 2019-08-17 14:02 +0200
    Re: How to get standard paths?!? David Brown <david.brown@hesbynett.no> - 2019-08-17 15:13 +0200
    Re: How to get standard paths?!? Sam <sam@email-scan.com> - 2019-08-17 09:21 -0400
    Re: How to get standard paths?!? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-08-19 20:57 +0000

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

#67942

> In N4659:
> Section 25.5.1 p1:  "The contents and meaning of the header <clocale>
> are the same as the C standard library header <locale.h>."
> Section 2 (Normative references) p2:  "The library described in Clause 7
> of ISO/IEC 9899:2011 is hereinafter called the /C standard library/."

And where does the C-standard say that the with of wchar_t has to match
the platform?

[toc] | [prev] | [next] | [standalone]

#67943

>> In N4659:
>> Section 25.5.1 p1:  "The contents and meaning of the header <clocale>
>> are the same as the C standard library header <locale.h>."
>> Section 2 (Normative references) p2:  "The library described in Clause 7
>> of ISO/IEC 9899:2011 is hereinafter called the /C standard library/."

> And where does the C-standard say that the with of wchar_t has to match
> the platform?

Here, the current 2019 C-draft:
"3.7.3: wide character: value representable by an object of type
wchar_t, capable of representing any character in the current locale"

[toc] | [prev] | [next] | [standalone]

#67775

On 18/08/2019 21:51, Bart wrote:
> On 18/08/2019 19:33, Bo Persson wrote:
>> On 2019-08-18 at 14:07, Bart wrote:
>>> On 18/08/2019 11:57, Bonita Montero wrote:
>>>>>> Unicode is defined to have a maximum of 21 bit codepoints.
>>>>
>>>>> Yes.  And how to you fit those 21 bits into a 16-bit wchar_t ?
>>>>
>>>> That's not necessary if you use UTF-16.
>>>>
>>>
>>>
>>> Show me how to make use of UTF-16 here, because it doesn't seem to
>>> work on Windows:
>>>
>>>      #include <wchar_t>
>>>
>>>      wchar_t c;
>>>      c = 1000000;         // character code 1000000
>>>
>>>      printf("c = %d 0x%X", c, c);
>>>
>>> Displays:
>>>
>>>      c = 16960 0x4240
>>>
>>> Not this:
>>>
>>>      c = 1000000 0xF4240
>>
>> So we can only guess that this character code is not part of the
>> "supported locale" used in your program.
>>
>> Nowhere does the language standard require C++ to support all
>> available OS locales. It could be a very small subset.

It is not the language that supports locales, or the compiler - it is
the /implementation/, which includes libraries and any other bits and
pieces needed for creating runnable programs.

Certainly a C++ (or C) compiler for Windows could choose only to support
some locales, such as those for which a 16-bit wchar_t is sufficient.
That might make sense for a small or custom compiler.  It would make no
sense at all for the main platform compiler for the system, and the
compiler that is (presumably) used to compile Windows itself.  And -
unsurprisingly - MSVC /does/ support locales with multi-unit characters
when encoded with UTF-16.

> 
> A small subset where none of the locale's alphabets has character codes
> above 65535.
> 
> Anyway, the locale need have nothing to do with the ability to be able
> to read, write or process arbitrary Unicode characters and strings.

Correct.

> 
> Otherwise countries such as the UK and US could get by with just ASCII.

How naïve of you to think so :-)

Some /programs/ can get away with just ASCII.  No countries can, not
even the USA.

Many programs could get away with UCS-2, however - these cover most
current scripts other than CJK.

> 
> But if they are going beyond a 7-bit character, then why not beyond
> 16-bit too? Actually, the first few alphabets beyond codepoint 65535
> appear to be ancient scripts with no meaningful locale anyway.

Many of the ancient scripts are below 0xffff Unicode code points - i.e.,
in the BMP.  The major use of higher planes is for CJK characters for
Chinese, Japanese and Korean.

[toc] | [prev] | [next] | [standalone]

#67778

On 19.08.2019 08:52, David Brown wrote:
> On 18/08/2019 21:51, Bart wrote:
>> On 18/08/2019 19:33, Bo Persson wrote:
>>> On 2019-08-18 at 14:07, Bart wrote:
>>>> On 18/08/2019 11:57, Bonita Montero wrote:
>>>>>>> Unicode is defined to have a maximum of 21 bit codepoints.
>>>>>
>>>>>> Yes.  And how to you fit those 21 bits into a 16-bit wchar_t ?
>>>>>
>>>>> That's not necessary if you use UTF-16.
>>>>>
>>>>
>>>>
>>>> Show me how to make use of UTF-16 here, because it doesn't seem to
>>>> work on Windows:
>>>>
>>>>       #include <wchar_t>
>>>>
>>>>       wchar_t c;
>>>>       c = 1000000;         // character code 1000000
>>>>
>>>>       printf("c = %d 0x%X", c, c);
>>>>
>>>> Displays:
>>>>
>>>>       c = 16960 0x4240
>>>>
>>>> Not this:
>>>>
>>>>       c = 1000000 0xF4240
>>>
>>> So we can only guess that this character code is not part of the
>>> "supported locale" used in your program.
>>>
>>> Nowhere does the language standard require C++ to support all
>>> available OS locales. It could be a very small subset.
> 
> It is not the language that supports locales, or the compiler - it is
> the /implementation/, which includes libraries and any other bits and
> pieces needed for creating runnable programs.

The above discussion is IMHO not meaningful.

The issue about what the standard means when it requires support for all 
characters in the execution character set, has been discussed at length 
before, multiple times, both here and e.g. on SO.

There is no good complete resolution, but there is the /observation/ 
that since the standard is out of tune with reality in this respect, it 
can't very well be viewed as authoritative. Viewing it as authoritative 
on an issue where it's demonstrably just wrong, where all Windows 
compilers are in breach of its silly requirement, would be religious. 
Since it's ignored by the compiler writers, others should do as them.


> Certainly a C++ (or C) compiler for Windows could choose only to support
> some locales, such as those for which a 16-bit wchar_t is sufficient.
> That might make sense for a small or custom compiler.  It would make no
> sense at all for the main platform compiler for the system, and the
> compiler that is (presumably) used to compile Windows itself.  And -
> unsurprisingly - MSVC /does/ support locales with multi-unit characters
> when encoded with UTF-16.

Again.

Plus the observation that locales are mostly irrelevant for Unicode.

There are some slight dependencies on locales, e.g. for Turkish I's, but 
those can reasonably be argued are /defects in Unicode/. One can simply 
choose to not support those marginal locale dependencies. Encoding text 
for a current locale is anyway super braindead in modern software.


>> A small subset where none of the locale's alphabets has character codes
>> above 65535.
>>
>> Anyway, the locale need have nothing to do with the ability to be able
>> to read, write or process arbitrary Unicode characters and strings.
> 
> Correct.
> 
>>
>> Otherwise countries such as the UK and US could get by with just ASCII.
> 
> How naïve of you to think so :-)
> 
> Some /programs/ can get away with just ASCII.  No countries can, not
> even the USA.
> 
> Many programs could get away with UCS-2, however - these cover most
> current scripts other than CJK.
> 
>>
>> But if they are going beyond a 7-bit character, then why not beyond
>> 16-bit too? Actually, the first few alphabets beyond codepoint 65535
>> appear to be ancient scripts with no meaningful locale anyway.
> 
> Many of the ancient scripts are below 0xffff Unicode code points - i.e.,
> in the BMP.  The major use of higher planes is for CJK characters for
> Chinese, Japanese and Korean.

Emojis.

  Cheers!,

- Alf

[toc] | [prev] | [next] | [standalone]

#67886

"Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> writes:

> On 19.08.2019 08:52, David Brown wrote:
>
>> On 18/08/2019 21:51, Bart wrote:
>>
>>> On 18/08/2019 19:33, Bo Persson wrote:
>>>
>>>> On 2019-08-18 at 14:07, Bart wrote:
>>>>
>>>>> On 18/08/2019 11:57, Bonita Montero wrote:
>>>>>
>>>>>>>> Unicode is defined to have a maximum of 21 bit codepoints.
>>>>>>>
>>>>>>> Yes.  And how to you fit those 21 bits into a 16-bit wchar_t ?
>>>>>>
>>>>>> That's not necessary if you use UTF-16.
>>>>>
>>>>> Show me how to make use of UTF-16 here, because it doesn't seem
>>>>> to work on Windows:
>>>>>
>>>>>   #include <wchar_t>
>>>>>
>>>>>   wchar_t c;
>>>>>   c = 1000000; // character code 1000000
>>>>>
>>>>>   printf("c = %d 0x%X", c, c);
>>>>>
>>>>> Displays:
>>>>>
>>>>>   c = 16960 0x4240
>>>>>
>>>>> Not this:
>>>>>
>>>>>   c = 1000000 0xF4240
>>>>
>>>> So we can only guess that this character code is not part of the
>>>> "supported locale" used in your program.
>>>>
>>>> Nowhere does the language standard require C++ to support all
>>>> available OS locales.  It could be a very small subset.
>>
>> It is not the language that supports locales, or the compiler - it
>> is the /implementation/, which includes libraries and any other
>> bits and pieces needed for creating runnable programs.
>
> The above discussion is IMHO not meaningful.
>
> The issue about what the standard means when it requires support for
> all characters in the execution character set, has been discussed at
> length before, multiple times, both here and e.g. on SO.
>
> There is no good complete resolution, but there is the /observation/
> that since the standard is out of tune with reality in this respect,
> it can't very well be viewed as authoritative.  [...]

It's Microsoft that is out of tune.  And getting moreso as time
goes on.

[toc] | [prev] | [next] | [standalone]

#67780

> Certainly a C++ (or C) compiler for Windows could choose only to support
> some locales, such as those for which a 16-bit wchar_t is sufficient.
> That might make sense for a small or custom compiler.  It would make no
> sense at all for the main platform compiler for the system, and the
> compiler that is (presumably) used to compile Windows itself.  And -
> unsurprisingly - MSVC /does/ support locales with multi-unit characters
> when encoded with UTF-16.

MSVC++ wchar_t's size matches Windows.h's WCHAR-type; both are 16 bits.
So it is the right decision to have a wchar_t with this with.

[toc] | [prev] | [next] | [standalone]

#67699

On 17/08/2019 17:52, Bonita Montero wrote:
>> wchar_t has /never/ been specifically about Unicode.  It means "wide 
>> character", it is supposed to big enough to hold a character of any 
>> character set supported by the implementation.  For example, a Chinese 
>> system might support ASCII and Big5, which is a 16-bit encoding - it 
>> could therefore have a 16-bit wchar_t.
> 
> Unrealistic that it will be used for anything different than Unicode.

wchar_t existed from long before Unicode became the standard choice. 
But I agree that for future systems, it is unrealistic for wchar_t to be 
used with anything other than UTF-32 encodings.

> 
>> For systems that support Unicode, wchar_t is required to be at least 
>> 32-bit.
> 
> No, it's implementation-defined.

It is required to be big enough for any of the compiler's wide-character 
execution set.  Since Windows supports Unicode (regardless of the 
encodings used), a Windows compiler must be able to hold any Unicode 
code point in a wchar_t - i.e., wchar_t must be a minimum of 32 bit.

[toc] | [prev] | [next] | [standalone]

#67702

> It is required to be big enough for any of the compiler's wide-character 
> execution set.  Since Windows supports Unicode (regardless of the 
> encodings used), a Windows compiler must be able to hold any Unicode 
> code point in a wchar_t - i.e., wchar_t must be a minimum of 32 bit.

Win32 uses UTF-16 and there will never be more populated codepoints
than UTF-16 can cover; so where's the problem?

[toc] | [prev] | [next] | [standalone]

#67712

On 17/08/2019 18:06, Bonita Montero wrote:
>> It is required to be big enough for any of the compiler's 
>> wide-character execution set.  Since Windows supports Unicode 
>> (regardless of the encodings used), a Windows compiler must be able to 
>> hold any Unicode code point in a wchar_t - i.e., wchar_t must be a 
>> minimum of 32 bit.
> 
> Win32 uses UTF-16 and there will never be more populated codepoints
> than UTF-16 can cover; so where's the problem?

There is no problem with using UTF-16 for strings - except that it is an 
encoding that combines all the disadvantages of UTF-8 with all the 
disadvantages of UTF-32 with none of their advantages.  But you cannot 
store a arbitrary Unicode /character/ in a single 16-bit object. 
Critically, you cannot store a Unicode CJK (Chinese, Japanese, Korean) 
ideograph in a single 16-bit wchar_t, despite these characters being 
supported by Windows.  You /can/ store them in a char32_t, or a 32-bit 
wchar_t.

[toc] | [prev] | [next] | [standalone]

#67728

> There is no problem with using UTF-16 for strings - except that it is
> an encoding that combines all the disadvantages of UTF-8 with all the 
> disadvantages of UTF-32 with none of their advantages.  But you cannot 
> store a arbitrary Unicode /character/ in a single 16-bit object. 

Therefore you have UTF-16.

[toc] | [prev] | [next] | [standalone]

#67738

On 18/08/2019 08:49, Bonita Montero wrote:
>> There is no problem with using UTF-16 for strings - except that it is
>> an encoding that combines all the disadvantages of UTF-8 with all the 
>> disadvantages of UTF-32 with none of their advantages.  But you cannot 
>> store a arbitrary Unicode /character/ in a single 16-bit object. 
> 
> Therefore you have UTF-16.
> 

wchar_t is not for UTF-16 - each single wchar_t object should store a 
complete character.  That is what the type means - look it up.

Strings or arrays of wchar_t can hold UTF-16 encoded data, which will 
work in practice but is an abuse of wchar_t that goes against the 
standards.  ("char16_t" is the type you want here, which is distinct 
from wchar_t despite being the same size on Windows.)

Until you can find a way to store the /single/ character "𓃀" (the 
hieroglyph for "B") in a /single/ 16-bit wchar_t, not a string or array, 
then 16-bit wchar_t is too small.

[toc] | [prev] | [next] | [standalone]

#67744

> Strings or arrays of wchar_t can hold UTF-16 encoded data, which will 
> work in practice but is an abuse of wchar_t that goes against the 
> standards.  ("char16_t" is the type you want here, which is distinct 
> from wchar_t despite being the same size on Windows.)

There's no violation of the standard. wchar_t is implementation-defined.

[toc] | [prev] | [next] | [standalone]

#67761

Bonita Montero <Bonita.Montero@gmail.com> writes:
>> wchar_t has /never/ been specifically about Unicode.  It means "wide 
>> character", it is supposed to big enough to hold a character of any 
>> character set supported by the implementation.  For example, a Chinese 
>> system might support ASCII and Big5, which is a 16-bit encoding - it 
>> could therefore have a 16-bit wchar_t.
>
>Unrealistic that it will be used for anything different than Unicode.

It (wchar_t) predates unicode by a few years.

[toc] | [prev] | [next] | [standalone]

#67762

>> Unrealistic that it will be used for anything different than Unicode.

> It (wchar_t) predates unicode by a few years.

That doesn't change the above.

[toc] | [prev] | [next] | [standalone]

#67760

Bonita Montero <Bonita.Montero@gmail.com> writes:
>> #include <iostream>
>> 
>> int main() {
>>      std::cout<<sizeof(wchar_t)<<std::endl;
>> }
>> 
>>> g++ -o wchar wchar.cc
>>> ./wchar
>> 4
>>> g++ -v
>> Using built-in specs.
>> COLLECT_GCC=g++
>> COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-suse-linux/9/lto-wrapper
>> OFFLOAD_TARGET_NAMES=hsa:nvptx-none
>> Target: x86_64-suse-linux
>> …
>
>Hard, I wouldn't have expected this. I think this isn't a clever
>decision. For internal processing the representation doesn't matter
>but for compatiblity-reasons it should match the most commun Unicode
>representation.

wchar_t in Unix (198x)  predates Windows 3.1 (1992) by several years.   

Windows is the outlier here, not unix.

[toc] | [prev] | [next] | [standalone]

#67763

>> Hard, I wouldn't have expected this. I think this isn't a clever
>> decision. For internal processing the representation doesn't matter
>> but for compatiblity-reasons it should match the most commun Unicode
>> representation.

> wchar_t in Unix (198x)  predates Windows 3.1 (1992) by several years.

That doesn't affect the above because in the beginning, UCS-2 was the
standard.

[toc] | [prev] | [next] | [standalone]

#67697

On 17/08/2019 16:28, Bonita Montero wrote:
>> MS was early in the "Unicode" game by using UCS-2 in Windows NT.  They 
>> get credit for trying, IMHO.  But it quickly became apparent that 16 
>> bits was not sufficient, and anyone who was not tied to Windows used 
>> 32-bit utf-32 when they needed "one object is one character", 
>> typically for internal use only, and utf-8 when they wanted strings.  
>> If only MS had taken the hit and made the changes while they still 
>> could, it would have avoided the hideous mess that resulted with the 
>> painful 16-bit types.  16-bit is too short to hold all the Unicode 
>> characters, but long enough to have all the disadvantages of wasted 
>> space and endianness issues.  And now we are left with the jumble that 
>> is five different character types in C++ (not including signed and 
>> unsigned versions), MS' influence putting utf-16 nonsense into the C++ 
>> standards, and of course other platform-independent languages (Java, 
>> Python) and libraries (QT) using utf-16 or ucs-16 to be compatible 
>> with MS's error.
> 
> We were not talking about UTF-whatever. We were talking about wchar_t
> or more precisely std::u16sting. You can store 16 bit characters in the
> latter as well as UTF-16-seqences in this.

You can store any character encoding supported by the compiler in a 
wchar_t, unless you are using Windows, which is broken and can't store 
Unicode characters in a wchar_t because it is too small.

You can store any UTF-16 code unit in a char16_t.  That does not mean 
you can store any UTF-16 character in a char16_t - you can only store 
those that fit in one unit.  And in a std::u16string, you can store any 
UTF-16 string of characters.

You can store any UTF-32 code unit in a char32_t.  That covers all 
Unicode code points, but there are Unicode characters that are made of 
combinations of code points.  And in a std::u32string, you can store any 
UTF-32 string of characters.

And you can store any UTF-8 code unit in a char8_t.


In a wchar_t, you can (except on Windows) store any character from the 
"execution wide-character set" for the compiler.  That does not need to 
be Unicode.

On most modern systems, wchar_t matches char32_t.  But it is not a 
requirement.


> We were talking about that you can almost rely on the fact that wchar_t
> is 16 bits wide.

You are wrong.  You can rely on wchar_t being 32-bit, except on Windows 
and a few small 8-bit systems that often have very limited support for 
wchar_t at all.

> There might be ancient CPU-architectures where the re-
> gisters aren't 8, 16, ... bits wide but you can be certain that no one
> would implement a conforming C++-compiler for these systems.

People use C++ all the time on 8-bit and 16-bit processors - /new/ 
designs, not ancient ones.  They often don't have a full C++ library, 
but they are freestanding systems and don't need the full library to be 
compliant.  (They might be non-compliant in other ways.)

Windows compilers are invariably non-compliant regarding wchar_t because 
it is only 16-bit on that platform, when it is required to be 32-bit.

[toc] | [prev] | [next] | [standalone]

#67701

> You can store any character encoding supported by the compiler in a 
> wchar_t, unless you are using Windows, which is broken and can't store 
> Unicode characters in a wchar_t because it is too small.

The size of wchar_t is implementation-dependent, so Windows isn't broken
here. And you have a limited number of codepoints in UTF-16 over UTF-32,
but even UTF-16 has such a huge number of codepoints that this will
never be exhausted.
And as there will never be more codepoints in Unicode as the "limited"
range of UTF-16 can cover Windows isn't broken here.

> On most modern systems, wchar_t matches char32_t.
> But it is not a  requirement.

That was where David is wrong.

> People use C++ all the time on 8-bit and 16-bit processors - /new/ 
> designs, not ancient ones.  They often don't have a full C++ library, 
> but they are freestanding systems and don't need the full library to
> be compliant.  (They might be non-compliant in other ways.)

I wasn't talking about the library but basic datatypes.

> Windows compilers are invariably non-compliant regarding wchar_t because 
> it is only 16-bit on that platform, when it is required to be 32-bit.

UTF-16 is suitable for all codepoints that will ever be populated.

[toc] | [prev] | [next] | [standalone]

#67708

Am 17.08.19 um 18:05 schrieb Bonita Montero:
>> You can store any character encoding supported by the compiler in a 
>> wchar_t, unless you are using Windows, which is broken and can't store 
>> Unicode characters in a wchar_t because it is too small.
> 
> The size of wchar_t is implementation-dependent, so Windows isn't broken
> here.

Yes, the size of wchar_t is implementation-dependent, so it's not 
formally broken.

But the Windows implementation is technically broken, as the programmer 
can neither rely on the assumption, that any given character can be 
stored into a wchar_t, nor that iterating over a string will correctly 
return the stored characters, nor that insert/erase/substring operations 
will cut the string properly - which completely defies the purpose of 
wchar_t.

If i have to code multi-byte awareness into each and every string 
operation anyways, there is no point of using wstring instead of string 
and the much more common UTF8 encoding.

And it is getting even worse, as many Windows libraries and Windows 
applications are not aware of the fact, that a Windows wstring is 
supposed to be UTF16. The code is assuming one codepoint per string 
position and fails in an unpredictable way, if the input data acutally 
contains codepoints from the upper pages.

>> On most modern systems, wchar_t matches char32_t.
>> But it is not a  requirement.
> 
> That was where David is wrong.

He's not.

[toc] | [prev] | [next] | [standalone]

#67714

On 17/08/2019 19:47, Hergen Lehmann wrote:
> Am 17.08.19 um 18:05 schrieb Bonita Montero:
>>> You can store any character encoding supported by the compiler in a 
>>> wchar_t, unless you are using Windows, which is broken and can't 
>>> store Unicode characters in a wchar_t because it is too small.
>>
>> The size of wchar_t is implementation-dependent, so Windows isn't broken
>> here.
> 
> Yes, the size of wchar_t is implementation-dependent, so it's not 
> formally broken.

It is for Windows, because Windows supports Unicode as an execution 
character set (the encoding is irrelevant), and 16-bit wchar_t is not 
big enough.  Anything 21 bits or higher, that fulfils the other 
requirements in the standard, would do.

> 
> But the Windows implementation is technically broken, as the programmer 
> can neither rely on the assumption, that any given character can be 
> stored into a wchar_t, nor that iterating over a string will correctly 
> return the stored characters, nor that insert/erase/substring operations 
> will cut the string properly - which completely defies the purpose of 
> wchar_t.
> 

Yes.

16-bit wchar_t made sense when Windows supported UCS-2, which is a 
16-bit subset of Unicode, and is where Unicode started.  But Unicode 
moved on in 1996, and Windows did not.

> If i have to code multi-byte awareness into each and every string 
> operation anyways, there is no point of using wstring instead of string 
> and the much more common UTF8 encoding.
> 

Yes.

> And it is getting even worse, as many Windows libraries and Windows 
> applications are not aware of the fact, that a Windows wstring is 
> supposed to be UTF16. The code is assuming one codepoint per string 
> position and fails in an unpredictable way, if the input data acutally 
> contains codepoints from the upper pages.
> 
>>> On most modern systems, wchar_t matches char32_t.
>>> But it is not a  requirement.
>>
>> That was where David is wrong.
> 
> He's not.
> 

[toc] | [prev] | [next] | [standalone]

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

Back to top | Article view | comp.lang.c++

csiph-web