Re: Unicode in C Sources

From	Manfred <noname@add.invalid>
Newsgroups	comp.lang.c
Subject	Re: Unicode in C Sources
Date	2020-02-02 17:59 +0100
Organization	Aioe.org NNTP Server
Message-ID	<r16v5c$1bff$1@gioia.aioe.org> (permalink)
References	(1 earlier) <87h80axl7s.fsf@bsb.me.uk> <5ajZF.602301$CO7.252756@fx36.am4> <R3lZF.110132$gW1.31048@fx03.iad> <r150nk$nsg$1@gioia.aioe.org> <1XzZF.178637$hD4.62031@fx47.iad>

Show all headers | View raw

On 2/2/2020 2:14 PM, Richard Damon wrote:
>>>> If not, then what /is/ allowed, as there could be hundreds of 
>>>> thousands of allowable characters?
>>>
>>> The C standard allows the implementation to define what set of 
>>> characters it allows in the source of a program. The basic character 
>>> set includes those characters you list as allowed to be in identifiers.
>>>
>>> The Standard also allows the implementation to allow additional 
>>> characters in an extended character set (which can easily be all of 
>>> Unicode if desired).
>>>
>>> The Standard also defines Universal Characters (using \u9999 or 
>>> \U00012345 notation) which basically allow unicode values to be 
>>> specified, and defines a list of which of these must and must not be 
>>> allowed in identifier names. If the implementation allows general 
>>> Unicode characters in its character set, then those match ranges 
>>> should probably be allowed / disallowed.
>>>
>>> The ranges of explicitly allowed codes:
>>> 00A8, 00AA, 00AD, 00AF, 00B2–00B5, 00B7–00BA, 00BC–00BE, 00C0–00D6, 
>>> 00D8–00F6, 00F8–00FF, 0100–167F, 1681–180D, 180F–1FFF, 200B–200D, 
>>> 202A–202E, 203F–2040, 2054, 2060–206F, 2070–218F, 2460–24FF, 
>>> 2776–2793, 2C00–2DFF, 2E80–2FFF, 3004–3007, 3021–302F, 3031–303F, 
>>> 3040–D7FF, F900–FD3D, FD40–FDCF, FDF0–FE44, FE47–FFFD, 10000–1FFFD, 
>>> 20000–2FFFD, 30000–3FFFD, 40000–4FFFD, 50000–5FFFD, 60000–6FFFD, 
>>> 70000– 7FFFD, 80000–8FFFD, 90000–9FFFD, A0000–AFFFD, B0000–BFFFD, 
>>> C0000–CFFFD, D0000–DFFFD, E0000–EFFFD
>>>
>>> The ranges of explicitly disallowed codes:
>>> 0300–036F, 1DC0–1DFF, 20D0–20FF, FE20–FE2F
>>>
>>> Note the codes D800-DFFF (the UTF-16 Surrogates) are explicitly not 
>>> allowed as characters.
>>>
>>> Codes not mentioned I suppose are implementation dependent. I would 
>>> have to check carefully, but many of the gaps are unassigned (or 
>>> assigned as not characters) in Unicode.
>>
>> This is in n1570 6.4.2, D.1 and D.2, but it does not explain why gcc 
>> refuses this code: the Greek characters span 0370-03FF, which fall in 
>> the 0100–167F range of allowed universal character names.
> 
> If written in the (ugly) \u notation then it is supposed to accept them, 
> but as the actual Unicode character itself, it is a function of what the 
> implementation defines as it extended source character set.
> 
> My understanding is that the \u notation is designed that for a compiler 
> that does NOT accept full unicode source, you could have a pre-filter 
> phase that processes the source and converts the Unicode characters into 
> Universal characters (the \u and \U notation). It is even written so 
> this could be part of the early phases of the compiler.

In fact the wording used in C++ standard is more explicit on this 
(translation phase 1), although my understanding of the wording of C 
standard mandates that the characters in D.1 be accepted in identifiers 
as well (with the exception of D.2 as initial)

For the record, here is the gcc bug report:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67224

Thread

Unicode in C Sources F Russell <fr@random.info> - 2020-02-01 17:12 +0000
  Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-01 17:37 +0000
    Re: Unicode in C Sources Anton Shepelev <anton.txt@gmail.com> - 2020-02-01 20:43 +0300
      Re: Unicode in C Sources Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-02-01 10:57 -0800
        Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 16:58 +0100
          Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-02 17:05 +0000
            Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 19:52 +0100
              Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 19:20 +0000
            Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 12:38 +0000
              Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 09:16 -0500
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 15:32 +0100
                Re: Unicode in C Sources Öö Tiib <ootiib@hot.ee> - 2020-02-03 09:01 -0800
                Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 21:15 +0000
              Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:06 +0000
              Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:19 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-04 02:50 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 08:16 +0100
    Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-01 18:10 +0000
      Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-01 15:19 -0500
        Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:13 +0100
          Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:46 +0100
            Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 00:24 +0000
          Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-02 08:14 -0500
            Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 17:59 +0100
              Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:44 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 21:49 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 16:58 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:33 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:38 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 10:25 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 19:11 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 15:59 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 16:51 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:05 -0500
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:17 -0800
      Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 17:09 +0100
        Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:46 -0500
          Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 21:39 +0100
            Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 17:07 -0500
              Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 13:12 +0100
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:46 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 16:47 +0000
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:36 -0800
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 23:56 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:06 -0500
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:25 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:29 -0800
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:30 -0800
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-19 14:19 -0800
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-04-02 07:50 -0700
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-04-02 08:16 -0700
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 22:33 +0000
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:05 -0500
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:10 -0500
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:05 +0100
                Re: Unicode in C Sources already5chosen@yahoo.com - 2020-02-04 09:38 -0800
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:46 +0100
                Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:07 -0500
            Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-02 14:54 -0800
      Re: Unicode in C Sources Siri Cruise <chine.bleu@yahoo.com> - 2020-02-03 15:10 -0800
  Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 07:47 +0100
    Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 14:00 +0100
      Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:45 +0100
        Re: Unicode in C Sources Guillaume <message@bottle.org> - 2020-02-03 17:54 +0100
        Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:36 +0100
          Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 16:33 +0100
            Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 16:45 +0100
              Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-04 13:02 -0800
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 23:18 +0100
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 11:07 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:49 +0100
                Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-16 06:23 -0800
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 10:54 +0100
              Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:51 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 14:03 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:31 +0100
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 15:59 +0000
                Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-05 16:02 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 16:59 +0000
                Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 18:02 +0000
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 19:37 +0100
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 19:37 +0100
                Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 21:02 +0100
                Re: Unicode in C Sources Ian Collins <ian-news@hotmail.com> - 2020-02-06 10:34 +1300
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:10 +0000
                Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-06 13:53 +0100
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-06 13:26 +0000
                Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-06 07:27 -0500
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:03 +0000
                Re: Unicode in C Sources Mikko Rauhala <mjr@iki.fi> - 2020-02-06 10:46 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 21:47 +0100
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 22:50 +0000
                Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-07 00:25 +0000
                Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-07 02:07 +0000
                Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:26 +0100
      Re: Unicode in C Sources antispam@math.uni.wroc.pl - 2020-04-18 03:58 +0000

csiph-web