Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #161499 > unrolled thread

Re: Locale-related questions

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Locale-related questions Martin Str|mberg <ams@ludd.luth.se> - 2015-12-03 08:40 +0100
    Re: Locale-related questions Nicolas George <george@nsup.org> - 2015-12-03 09:10 +0100
      Re: Locale-related questions "Neal P. Murphy" <neal.p.murphy@alum.wpi.edu> - 2015-12-03 09:30 +0100
        Re: Locale-related questions Nicolas George <george@nsup.org> - 2015-12-03 09:40 +0100
        Re: Locale-related questions <tomas@tuxteam.de> - 2015-12-03 09:40 +0100

#161499 — Re: Locale-related questions

In article <qAfPA-1Jf-13@gated-at.bofh.it> Nicolas George <george@nsup.org> wrote:
> wchar_t portably. For starters, the i4s at microsoft decided that 64k
> characters should be enough for everyone, so if your cross-platform includes
> microsoftisms, you can not use wchar_t to represent an Unicode code point.
> The i4s at sun had other interesting ideas on how to make the coding for
> wchar_t itself depend on the locale.

I understand the reference to (the mess from) MS (although at that
time everyone in America did think 64k would be enough characters for
everyone, didn't they?), but not the one to Sun.

Were they the ones that made sure that C's wchar_t is standardized as
it it? Care to enlighten me?


-- 
MartinS

[toc] | [next] | [standalone]

#161502

[Multipart message — attachments visible in raw view] — view raw

Le tridi 13 frimaire, an CCXXIV, Martin Str|mberg a écrit :
> I understand the reference to (the mess from) MS (although at that
> time everyone in America did think 64k would be enough characters for
> everyone, didn't they?), but not the one to Sun.
> 
> Were they the ones that made sure that C's wchar_t is standardized as
> it it? Care to enlighten me?

Out of memory, it was more than thirteen years ago. They may have fixed
their design, but I doubt it given Solaris' religion on backwards
compatibility.

With Solaris' implementation of wchar_t (i.e. in the libc, but of course,
outside the Linux world, the libc and the kernel come from the same origin
and have the same name), the value for non-ASCII characters depends on the
locale.

More precisely, the value of wchar_t with UTF-8 locales is the Unicode code
point (same as GNU and any sane implementation), but with non-UTF-8 locales,
it is something else, usually derived from the octet value with markers in
the high order bits.

For example, for the character "U+00C9 LATIN CAPITAL LETTER E WITH ACUTE",
with LC_CTYPE=en_US.UTF-8, the wchar_t value is 0x000000c9, but with
LC_CTYPE=en_US (i.e. ISO-8859-1), the wchar_t value is something like
0x010000c9.

I do not remember if I tested the case of non-UTF-8 non-ISO-8859-1 locales.
My guess is, for "U+0399 GREEK CAPITAL LETTER IOTA", it is 0x00000399 for
LC_CTYPE=en_US.UTF-8, but for el_GR.ISO-8859-7, it would be something like
0x070000c9 (the ISO-8859-7 code for U+0399 is 0xc9).

In practice, that means that you can not use wchar_t values to query Unicode
databases, and that if the locale changes during the run of your program all
your wchar_t values become invalid.

(Sun is also responsible for making java's char type 16-bits and strings
UTF-16. This is another case of headdesk, or possibly headwall, although
entirely unrelated to the wchar_t issue. For those who do not know,
basically UTF-16 manages to combine almost all the drawbacks of UTF-8 with
almost all the drawbacks of UCS-4, plus a whole bunch of drawbacks of its
own.)

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#161504

On Thu, 3 Dec 2015 09:01:40 +0100
Nicolas George <george@nsup.org> wrote:

> (Sun is also responsible for making java's char type 16-bits and strings
> UTF-16. This is another case of headdesk, or possibly headwall, although
> entirely unrelated to the wchar_t issue. For those who do not know,
> basically UTF-16 manages to combine almost all the drawbacks of UTF-8 with
> almost all the drawbacks of UCS-4, plus a whole bunch of drawbacks of its
> own.)

This is a fine example of putting one's thoughts into words perfectly, leaving no doubt as to the intent. It's usually difficult, but clearly not impossible. :)

N

[toc] | [prev] | [next] | [standalone]

#161506

[Multipart message — attachments visible in raw view] — view raw

Le tridi 13 frimaire, an CCXXIV, tomas@tuxteam.de a écrit :
> Yeah, agreed, that's quote-worthy. Nicolas: can I put that into my
> quotations book? With attribution, of course!

No problem. I am flattered.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#161507

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Dec 03, 2015 at 03:25:38AM -0500, Neal P. Murphy wrote:
> On Thu, 3 Dec 2015 09:01:40 +0100
> Nicolas George <george@nsup.org> wrote:
> 
> > (Sun is also responsible for making java's char type 16-bits and strings
> > UTF-16. This is another case of headdesk, or possibly headwall, although
> > entirely unrelated to the wchar_t issue. For those who do not know,
> > basically UTF-16 manages to combine almost all the drawbacks of UTF-8 with
> > almost all the drawbacks of UCS-4, plus a whole bunch of drawbacks of its
> > own.)
> 
> This is a fine example of putting one's thoughts into words perfectly, leaving no doubt as to the intent. It's usually difficult, but clearly not impossible. :)

Yeah, agreed, that's quote-worthy. Nicolas: can I put that into my
quotations book? With attribution, of course!

regards
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlZf9XMACgkQBcgs9XrR2kaAVgCePFlxDvCcddGmyRMTBVBWnYAm
LuEAnRck0VXC8btHnwArnd3zzxohudaB
=eqXT
-----END PGP SIGNATURE-----

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web