Re: Unicode in C Sources

From	David Brown <david.brown@hesbynett.no>
Newsgroups	comp.lang.c
Subject	Re: Unicode in C Sources
Date	2020-02-03 22:25 +0100
Organization	A noiseless patient Spider
Message-ID	<r1a347$em1$1@dont-email.me> (permalink)
References	(4 earlier) <r178u5$eqm$2@dont-email.me> <r17c1n$cnp$1@dont-email.me> <r17h80$ecf$1@dont-email.me> <r192n7$1qh$1@dont-email.me> <r19neh$2lc$1@dont-email.me>

Show all headers | View raw

On 03/02/2020 19:06, James Kuyper wrote:
> On 2/3/20 7:12 AM, David Brown wrote:
>> On 02/02/2020 23:07, James Kuyper wrote:
>>> On 2/2/20 3:39 PM, David Brown wrote:
> ...
>>>> Basically, you had to use \u1234 or \U12345678 syntax for your Unicode
>>>> letters in identifiers.  So if you wanted "const double π = 3.14;",
>>>> you'd have to write "const double \u03C0 = 3.14;".  That is with the
>>>> "-fextended-identifiers" on to enable the feature.
>>>
>>> That's not an example of "support for extended identifiers" - it doesn't
>>> involve the use of a single extended character. That's simply of meeting
>>> the standard's requirements for the handling of UCN's, while failing to
>>> support extended characters in identifiers. I was expecting, from your
>>> comment, something much more bizarre than that.
>>
>> Now I'm confused.  This "π" identifier /is/ an example of an extended
>> identifier.
> 
> No, on an implementation which supports it, 'π' is an extended
> character, and an identifier containing it, such as "aππle" would be an
> extended identifier. However, \u03C0 is an UCN, not an extended character.
> 

Yes, \u03C0 is a UCN - "Universal character name", described in 6.4.3

The term "extended identifier" is not actually defined in the standard, 
as far as I can see - the phrase "extended identifier" is only found in 
the list of major changes to C99.  But the section on identifiers, 
6.4.2.1, says that an identifier is made up of digits (except for the 
first character), non-digits (a-z, A-Z, _), universal character names, 
and other implementation-defined characters.

An "extended identifier" must, I think, mean an identifier which has a 
UCN or an identifier which has an additional implementation-defined 
character (which must map to UCN's).

Basically, \u and \U sequences are source code character set independent 
ways to write additional Unicode characters that can be used in 
identifiers.  They play a somewhat similar role to trigraphs, in that 
they let you have extended identifiers in the source code without 
needing to express the characters directly.

Or are you saying that "extended identifier" is only about UCN's in 
identifier names?

To me, this is about user convenience and compatibility.  UCN's in 
identifiers, on their own, is clearly a useless concept.  If you have to 
use them, they add nothing but obfuscation to the code - they only make 
sense in combination with editors or pre-processors that let you use 
Unicode characters.  They are a back-stop, to let you access extended 
identifiers even if you can't use an extended character set.

> Since C99, the C standard requires a conforming implementation to accept
> UCNs, regardless of whether or not they support any extended characters.
> In character constants and string literals, what it requires is quite
> minimal - during translation phase 5, they are "converted to the
> corresponding member of the execution character set; if there is no
> corresponding member, it is converted to an implementation-defined
> member other than the null (wide) character." (5.1.1.2p5). Note, in
> particular, that a conforming implementation could convert all UCNs to
> the same "implementation-defined member".
> 
> No such conversion is described for UCNs in identifiers. I believe that
> implementations should treat extended characters in identifiers as
> equivalent to the implementation-defined UCNs that those characters are
> mapped to - but I'm having trouble locating the text in the standard
> that says so.
> 
> For members of the extended character set, "An implementation may allow
> multibyte characters that are not part of the basic source character set
> to appear in identifiers; which characters and their correspondence to
> universal character names is implementation-defined."  (6.4.2.1p3).
> Note, in particular, that an implementation need not allow any character
> that's not in the basic source character set.
> 
>>   My point is that gcc supports extended identifiers with
>> Unicode characters in their names (following the rules in Annex D, which
>> I believe allows π).
> 
> No, the rules in Annex D mandate support for \u03C0, which is a UCN, not
> an extended character. They don't cover 'π', which (if supported) would
> have to be an extended character, but is not a UCN. gcc is allowed to
> decide whether 'π' is in its extended character set, and if it is in
> that set, is permitted to choose which UCN it considers π equivalent to.
> It would be crazy to consider it equivalent to anything other than
> \u03C0, but being crazy in that sense wouldn't render an implementation
> non-conforming.
> 

I haven't suggested that gcc is non-conforming here.  It (up to gcc 10) 
appears to support extended identifiers in a fully conforming way while 
managing to be close to useless in practice due to the lack of support 
for extended characters i identifiers.  Really, that is all I meant be 
"interesting" support - technically correct and practically useless.

Thread

Unicode in C Sources F Russell <fr@random.info> - 2020-02-01 17:12 +0000
  Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-01 17:37 +0000
    Re: Unicode in C Sources Anton Shepelev <anton.txt@gmail.com> - 2020-02-01 20:43 +0300
      Re: Unicode in C Sources Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-02-01 10:57 -0800
        Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 16:58 +0100
          Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-02 17:05 +0000
            Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 19:52 +0100
              Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 19:20 +0000
            Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 12:38 +0000
              Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 09:16 -0500
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 15:32 +0100
                Re: Unicode in C Sources Öö Tiib <ootiib@hot.ee> - 2020-02-03 09:01 -0800
                Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 21:15 +0000
              Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:06 +0000
              Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:19 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-04 02:50 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 08:16 +0100
    Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-01 18:10 +0000
      Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-01 15:19 -0500
        Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:13 +0100
          Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:46 +0100
            Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 00:24 +0000
          Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-02 08:14 -0500
            Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 17:59 +0100
              Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:44 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 21:49 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 16:58 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:33 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:38 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 10:25 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 19:11 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 15:59 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 16:51 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:05 -0500
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:17 -0800
      Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 17:09 +0100
        Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:46 -0500
          Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 21:39 +0100
            Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 17:07 -0500
              Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 13:12 +0100
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:46 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 16:47 +0000
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:36 -0800
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 23:56 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:06 -0500
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:25 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:29 -0800
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:30 -0800
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-19 14:19 -0800
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-04-02 07:50 -0700
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-04-02 08:16 -0700
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 22:33 +0000
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:05 -0500
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:10 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:05 +0100
                Re: Unicode in C Sources already5chosen@yahoo.com - 2020-02-04 09:38 -0800
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:46 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:07 -0500
            Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-02 14:54 -0800
      Re: Unicode in C Sources Siri Cruise <chine.bleu@yahoo.com> - 2020-02-03 15:10 -0800
  Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 07:47 +0100
    Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 14:00 +0100
      Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:45 +0100
        Re: Unicode in C Sources Guillaume <message@bottle.org> - 2020-02-03 17:54 +0100
        Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:36 +0100
          Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 16:33 +0100
            Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 16:45 +0100
              Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-04 13:02 -0800
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 23:18 +0100
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 11:07 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:49 +0100
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-16 06:23 -0800
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 10:54 +0100
              Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:51 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 14:03 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:31 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 15:59 +0000
                Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-05 16:02 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 16:59 +0000
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 18:02 +0000
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 19:37 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 19:37 +0100
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 21:02 +0100
                Re: Unicode in C Sources Ian Collins <ian-news@hotmail.com> - 2020-02-06 10:34 +1300
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:10 +0000
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-06 13:53 +0100
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-06 13:26 +0000
                Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-06 07:27 -0500
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:03 +0000
                Re: Unicode in C Sources Mikko Rauhala <mjr@iki.fi> - 2020-02-06 10:46 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 21:47 +0100
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 22:50 +0000
                Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-07 00:25 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-07 02:07 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:26 +0100
      Re: Unicode in C Sources antispam@math.uni.wroc.pl - 2020-04-18 03:58 +0000

csiph-web