Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #149611 > unrolled thread
| Started by | F Russell <fr@random.info> |
|---|---|
| First post | 2020-02-01 17:12 +0000 |
| Last post | 2020-04-18 03:58 +0000 |
| Articles | 20 on this page of 96 — 22 participants |
Back to article view | Back to comp.lang.c
Unicode in C Sources F Russell <fr@random.info> - 2020-02-01 17:12 +0000
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-01 17:37 +0000
Re: Unicode in C Sources Anton Shepelev <anton.txt@gmail.com> - 2020-02-01 20:43 +0300
Re: Unicode in C Sources Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-02-01 10:57 -0800
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 16:58 +0100
Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-02 17:05 +0000
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 19:52 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 19:20 +0000
Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 12:38 +0000
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 09:16 -0500
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 15:32 +0100
Re: Unicode in C Sources Öö Tiib <ootiib@hot.ee> - 2020-02-03 09:01 -0800
Re: Unicode in C Sources Jorgen Grahn <grahn+nntp@snipabacken.se> - 2020-02-03 21:15 +0000
Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:06 +0000
Re: Unicode in C Sources F Russell <fr@random.info> - 2020-02-04 02:19 +0000
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-04 02:50 +0000
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 08:16 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-01 18:10 +0000
Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-01 15:19 -0500
Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:13 +0100
Re: Unicode in C Sources Manfred <invalid@add.invalid> - 2020-02-02 00:46 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-02 00:24 +0000
Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-02 08:14 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 17:59 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:44 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-02 21:49 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 16:58 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:33 +0100
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 15:38 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 10:25 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-03 19:11 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 15:59 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 16:51 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:05 -0500
Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:17 -0800
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 17:09 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 14:46 -0500
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-02 21:39 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-02 17:07 -0500
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 13:12 +0100
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:46 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 16:47 +0000
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:36 -0800
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 23:56 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:06 -0500
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:25 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 13:29 -0800
Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-19 12:30 -0800
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-19 14:19 -0800
Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-04-02 07:50 -0700
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-04-02 08:16 -0700
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-03 22:33 +0000
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:05 -0500
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-03 23:10 -0500
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:05 +0100
Re: Unicode in C Sources already5chosen@yahoo.com - 2020-02-04 09:38 -0800
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-04 18:46 +0100
Re: Unicode in C Sources James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-02-04 16:07 -0500
Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-02 14:54 -0800
Re: Unicode in C Sources Siri Cruise <chine.bleu@yahoo.com> - 2020-02-03 15:10 -0800
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 07:47 +0100
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 14:00 +0100
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-03 16:45 +0100
Re: Unicode in C Sources Guillaume <message@bottle.org> - 2020-02-03 17:54 +0100
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-03 22:36 +0100
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 16:33 +0100
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-04 16:45 +0100
Re: Unicode in C Sources Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-02-04 13:02 -0800
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-04 23:18 +0100
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 11:07 +0100
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:49 +0100
Re: Unicode in C Sources Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-02-16 06:23 -0800
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 10:54 +0100
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 14:51 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 14:03 +0000
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:31 +0100
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 15:59 +0000
Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-05 16:02 +0000
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 16:59 +0000
Re: Unicode in C Sources Bart <bc@freeuk.com> - 2020-02-05 18:02 +0000
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 19:37 +0100
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-05 19:37 +0100
Re: Unicode in C Sources jacobnavia <jacob@jacob.remcomp.fr> - 2020-02-05 21:02 +0100
Re: Unicode in C Sources Ian Collins <ian-news@hotmail.com> - 2020-02-06 10:34 +1300
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:10 +0000
Re: Unicode in C Sources Manfred <noname@add.invalid> - 2020-02-06 13:53 +0100
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-06 13:26 +0000
Re: Unicode in C Sources Richard Damon <Richard@Damon-Family.org> - 2020-02-06 07:27 -0500
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 23:03 +0000
Re: Unicode in C Sources Mikko Rauhala <mjr@iki.fi> - 2020-02-06 10:46 +0000
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 21:47 +0100
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-05 22:50 +0000
Re: Unicode in C Sources scott@slp53.sl.home (Scott Lurndal) - 2020-02-07 00:25 +0000
Re: Unicode in C Sources Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-02-07 02:07 +0000
Re: Unicode in C Sources David Brown <david.brown@hesbynett.no> - 2020-02-05 16:26 +0100
Re: Unicode in C Sources antispam@math.uni.wroc.pl - 2020-04-18 03:58 +0000
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
| From | Manfred <invalid@add.invalid> |
|---|---|
| Date | 2020-02-02 00:46 +0100 |
| Message-ID | <r152ka$vqr$1@gioia.aioe.org> |
| In reply to | #149627 |
On 2/2/20 12:13 AM, Manfred wrote:
> On 2/1/20 9:19 PM, Richard Damon wrote:
>> On 2/1/20 1:10 PM, Bart wrote:
>>> On 01/02/2020 17:37, Ben Bacarisse wrote:
>>>> F Russell <fr@random.info> writes:
>>>>
>>>>> Should it be possible to write C source files in Unicode?
>>>>>
>>>>> Now, only former-ASCII characters are allowed but it would
>>>>> be nice to be able to write:
>>>>
>>>> No, it is allowed now:
>>>>
>>>> $ cat t.c
>>>> #include <stdio.h>
>>>> #include <math.h>
>>>>
>>>> int main()
>>>> {
>>>> double φ = 0.5, ω2 = 0.2, θ = -0.5, ω3 = 0.1;
>>>> printf("%g\n", cos(φ) * ω2 - sin(θ) * ω3);
>>>> }
>>>> $ clang t.c -lm
>>>> $ ./a.out
>>>> 0.223459
>>>
>>> That works with clang on rextester.com, but not with gcc? (Same with
>>> bcc, where the problem is the first byte of the phi character when
>>> using utf8.)
>>>
>>> Doesn't C restrict identifier names to [A-Z a-z 0-9 _] ?
>>>
>>> If not, then what /is/ allowed, as there could be hundreds of
>>> thousands of allowable characters?
>>
>> The C standard allows the implementation to define what set of
>> characters it allows in the source of a program. The basic character
>> set includes those characters you list as allowed to be in identifiers.
>>
>> The Standard also allows the implementation to allow additional
>> characters in an extended character set (which can easily be all of
>> Unicode if desired).
>>
>> The Standard also defines Universal Characters (using \u9999 or
>> \U00012345 notation) which basically allow unicode values to be
>> specified, and defines a list of which of these must and must not be
>> allowed in identifier names. If the implementation allows general
>> Unicode characters in its character set, then those match ranges
>> should probably be allowed / disallowed.
>>
>> The ranges of explicitly allowed codes:
>> 00A8, 00AA, 00AD, 00AF, 00B2–00B5, 00B7–00BA, 00BC–00BE, 00C0–00D6,
>> 00D8–00F6, 00F8–00FF, 0100–167F, 1681–180D, 180F–1FFF, 200B–200D,
>> 202A–202E, 203F–2040, 2054, 2060–206F, 2070–218F, 2460–24FF,
>> 2776–2793, 2C00–2DFF, 2E80–2FFF, 3004–3007, 3021–302F, 3031–303F,
>> 3040–D7FF, F900–FD3D, FD40–FDCF, FDF0–FE44, FE47–FFFD, 10000–1FFFD,
>> 20000–2FFFD, 30000–3FFFD, 40000–4FFFD, 50000–5FFFD, 60000–6FFFD,
>> 70000– 7FFFD, 80000–8FFFD, 90000–9FFFD, A0000–AFFFD, B0000–BFFFD,
>> C0000–CFFFD, D0000–DFFFD, E0000–EFFFD
>>
>> The ranges of explicitly disallowed codes:
>> 0300–036F, 1DC0–1DFF, 20D0–20FF, FE20–FE2F
>>
>> Note the codes D800-DFFF (the UTF-16 Surrogates) are explicitly not
>> allowed as characters.
>>
>> Codes not mentioned I suppose are implementation dependent. I would
>> have to check carefully, but many of the gaps are unassigned (or
>> assigned as not characters) in Unicode.
>
> This is in n1570 6.4.2, D.1 and D.2, but it does not explain why gcc
> refuses this code: the Greek characters span 0370-03FF, which fall in
> the 0100–167F range of allowed universal character names.
As it happens, this seems to be a long lasting bug in gcc, fixed in gcc 10.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2020-02-02 00:24 +0000 |
| Message-ID | <TEoZF.298801$7i5.274736@fx35.am4> |
| In reply to | #149628 |
On 01/02/2020 23:46, Manfred wrote: > On 2/2/20 12:13 AM, Manfred wrote: >> On 2/1/20 9:19 PM, Richard Damon wrote: >>> The ranges of explicitly allowed codes: >>> 00A8, 00AA, 00AD, 00AF, 00B2–00B5, 00B7–00BA, 00BC–00BE, 00C0–00D6, >>> 00D8–00F6, 00F8–00FF, 0100–167F, 1681–180D, 180F–1FFF, 200B–200D, >>> 202A–202E, 203F–2040, 2054, 2060–206F, 2070–218F, 2460–24FF, >>> 2776–2793, 2C00–2DFF, 2E80–2FFF, 3004–3007, 3021–302F, 3031–303F, >>> 3040–D7FF, F900–FD3D, FD40–FDCF, FDF0–FE44, FE47–FFFD, 10000–1FFFD, >>> 20000–2FFFD, 30000–3FFFD, 40000–4FFFD, 50000–5FFFD, 60000–6FFFD, >>> 70000– 7FFFD, 80000–8FFFD, 90000–9FFFD, A0000–AFFFD, B0000–BFFFD, >>> C0000–CFFFD, D0000–DFFFD, E0000–EFFFD >>> >>> The ranges of explicitly disallowed codes: >>> 0300–036F, 1DC0–1DFF, 20D0–20FF, FE20–FE2F >>> >>> Note the codes D800-DFFF (the UTF-16 Surrogates) are explicitly not >>> allowed as characters. >>> >>> Codes not mentioned I suppose are implementation dependent. I would >>> have to check carefully, but many of the gaps are unassigned (or >>> assigned as not characters) in Unicode. >> >> This is in n1570 6.4.2, D.1 and D.2, but it does not explain why gcc >> refuses this code: the Greek characters span 0370-03FF, which fall in >> the 0100–167F range of allowed universal character names. > > As it happens, this seems to be a long lasting bug in gcc, fixed in gcc 10. It works also with a couple of other C compilers (tcc and lccwin). I've modified my bcc compiler to accept that program, but via the expedient of taking any byte over 127 as a legal character for an identifier. At some point, I'll do it properly. (Maybe. It involves properly decoding utf8 sequences into unicode values, checking if the values are within those ranges, then storing the 2-4 byte sequence as part of the identifier. Messy, and right inside what is normally a very fast identifier loop.)
[toc] | [prev] | [next] | [standalone]
| From | Richard Damon <Richard@Damon-Family.org> |
|---|---|
| Date | 2020-02-02 08:14 -0500 |
| Message-ID | <1XzZF.178637$hD4.62031@fx47.iad> |
| In reply to | #149627 |
On 2/1/20 6:13 PM, Manfred wrote:
> On 2/1/20 9:19 PM, Richard Damon wrote:
>> On 2/1/20 1:10 PM, Bart wrote:
>>> On 01/02/2020 17:37, Ben Bacarisse wrote:
>>>> F Russell <fr@random.info> writes:
>>>>
>>>>> Should it be possible to write C source files in Unicode?
>>>>>
>>>>> Now, only former-ASCII characters are allowed but it would
>>>>> be nice to be able to write:
>>>>
>>>> No, it is allowed now:
>>>>
>>>> $ cat t.c
>>>> #include <stdio.h>
>>>> #include <math.h>
>>>>
>>>> int main()
>>>> {
>>>> double φ = 0.5, ω2 = 0.2, θ = -0.5, ω3 = 0.1;
>>>> printf("%g\n", cos(φ) * ω2 - sin(θ) * ω3);
>>>> }
>>>> $ clang t.c -lm
>>>> $ ./a.out
>>>> 0.223459
>>>
>>> That works with clang on rextester.com, but not with gcc? (Same with
>>> bcc, where the problem is the first byte of the phi character when
>>> using utf8.)
>>>
>>> Doesn't C restrict identifier names to [A-Z a-z 0-9 _] ?
>>>
>>> If not, then what /is/ allowed, as there could be hundreds of
>>> thousands of allowable characters?
>>
>> The C standard allows the implementation to define what set of
>> characters it allows in the source of a program. The basic character
>> set includes those characters you list as allowed to be in identifiers.
>>
>> The Standard also allows the implementation to allow additional
>> characters in an extended character set (which can easily be all of
>> Unicode if desired).
>>
>> The Standard also defines Universal Characters (using \u9999 or
>> \U00012345 notation) which basically allow unicode values to be
>> specified, and defines a list of which of these must and must not be
>> allowed in identifier names. If the implementation allows general
>> Unicode characters in its character set, then those match ranges
>> should probably be allowed / disallowed.
>>
>> The ranges of explicitly allowed codes:
>> 00A8, 00AA, 00AD, 00AF, 00B2–00B5, 00B7–00BA, 00BC–00BE, 00C0–00D6,
>> 00D8–00F6, 00F8–00FF, 0100–167F, 1681–180D, 180F–1FFF, 200B–200D,
>> 202A–202E, 203F–2040, 2054, 2060–206F, 2070–218F, 2460–24FF,
>> 2776–2793, 2C00–2DFF, 2E80–2FFF, 3004–3007, 3021–302F, 3031–303F,
>> 3040–D7FF, F900–FD3D, FD40–FDCF, FDF0–FE44, FE47–FFFD, 10000–1FFFD,
>> 20000–2FFFD, 30000–3FFFD, 40000–4FFFD, 50000–5FFFD, 60000–6FFFD,
>> 70000– 7FFFD, 80000–8FFFD, 90000–9FFFD, A0000–AFFFD, B0000–BFFFD,
>> C0000–CFFFD, D0000–DFFFD, E0000–EFFFD
>>
>> The ranges of explicitly disallowed codes:
>> 0300–036F, 1DC0–1DFF, 20D0–20FF, FE20–FE2F
>>
>> Note the codes D800-DFFF (the UTF-16 Surrogates) are explicitly not
>> allowed as characters.
>>
>> Codes not mentioned I suppose are implementation dependent. I would
>> have to check carefully, but many of the gaps are unassigned (or
>> assigned as not characters) in Unicode.
>
> This is in n1570 6.4.2, D.1 and D.2, but it does not explain why gcc
> refuses this code: the Greek characters span 0370-03FF, which fall in
> the 0100–167F range of allowed universal character names.
If written in the (ugly) \u notation then it is supposed to accept them,
but as the actual Unicode character itself, it is a function of what the
implementation defines as it extended source character set.
My understanding is that the \u notation is designed that for a compiler
that does NOT accept full unicode source, you could have a pre-filter
phase that processes the source and converts the Unicode characters into
Universal characters (the \u and \U notation). It is even written so
this could be part of the early phases of the compiler.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-02 17:59 +0100 |
| Message-ID | <r16v5c$1bff$1@gioia.aioe.org> |
| In reply to | #149632 |
On 2/2/2020 2:14 PM, Richard Damon wrote: >>>> If not, then what /is/ allowed, as there could be hundreds of >>>> thousands of allowable characters? >>> >>> The C standard allows the implementation to define what set of >>> characters it allows in the source of a program. The basic character >>> set includes those characters you list as allowed to be in identifiers. >>> >>> The Standard also allows the implementation to allow additional >>> characters in an extended character set (which can easily be all of >>> Unicode if desired). >>> >>> The Standard also defines Universal Characters (using \u9999 or >>> \U00012345 notation) which basically allow unicode values to be >>> specified, and defines a list of which of these must and must not be >>> allowed in identifier names. If the implementation allows general >>> Unicode characters in its character set, then those match ranges >>> should probably be allowed / disallowed. >>> >>> The ranges of explicitly allowed codes: >>> 00A8, 00AA, 00AD, 00AF, 00B2–00B5, 00B7–00BA, 00BC–00BE, 00C0–00D6, >>> 00D8–00F6, 00F8–00FF, 0100–167F, 1681–180D, 180F–1FFF, 200B–200D, >>> 202A–202E, 203F–2040, 2054, 2060–206F, 2070–218F, 2460–24FF, >>> 2776–2793, 2C00–2DFF, 2E80–2FFF, 3004–3007, 3021–302F, 3031–303F, >>> 3040–D7FF, F900–FD3D, FD40–FDCF, FDF0–FE44, FE47–FFFD, 10000–1FFFD, >>> 20000–2FFFD, 30000–3FFFD, 40000–4FFFD, 50000–5FFFD, 60000–6FFFD, >>> 70000– 7FFFD, 80000–8FFFD, 90000–9FFFD, A0000–AFFFD, B0000–BFFFD, >>> C0000–CFFFD, D0000–DFFFD, E0000–EFFFD >>> >>> The ranges of explicitly disallowed codes: >>> 0300–036F, 1DC0–1DFF, 20D0–20FF, FE20–FE2F >>> >>> Note the codes D800-DFFF (the UTF-16 Surrogates) are explicitly not >>> allowed as characters. >>> >>> Codes not mentioned I suppose are implementation dependent. I would >>> have to check carefully, but many of the gaps are unassigned (or >>> assigned as not characters) in Unicode. >> >> This is in n1570 6.4.2, D.1 and D.2, but it does not explain why gcc >> refuses this code: the Greek characters span 0370-03FF, which fall in >> the 0100–167F range of allowed universal character names. > > If written in the (ugly) \u notation then it is supposed to accept them, > but as the actual Unicode character itself, it is a function of what the > implementation defines as it extended source character set. > > My understanding is that the \u notation is designed that for a compiler > that does NOT accept full unicode source, you could have a pre-filter > phase that processes the source and converts the Unicode characters into > Universal characters (the \u and \U notation). It is even written so > this could be part of the early phases of the compiler. In fact the wording used in C++ standard is more explicit on this (translation phase 1), although my understanding of the wording of C standard mandates that the characters in D.1 be accepted in identifiers as well (with the exception of D.2 as initial) For the record, here is the gcc bug report: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67224
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-02 14:44 -0500 |
| Message-ID | <r178r3$eqm$1@dont-email.me> |
| In reply to | #149639 |
On 2/2/20 11:59 AM, Manfred wrote: > On 2/2/2020 2:14 PM, Richard Damon wrote: ... >> If written in the (ugly) \u notation then it is supposed to accept them, >> but as the actual Unicode character itself, it is a function of what the >> implementation defines as it extended source character set. >> >> My understanding is that the \u notation is designed that for a compiler >> that does NOT accept full unicode source, you could have a pre-filter >> phase that processes the source and converts the Unicode characters into >> Universal characters (the \u and \U notation). It is even written so >> this could be part of the early phases of the compiler. > > In fact the wording used in C++ standard is more explicit on this > (translation phase 1), although my understanding of the wording of C > standard mandates that the characters in D.1 be accepted in identifiers > as well (with the exception of D.2 as initial) C++ does not allow an implementation-defined set of extended characters to appear in an identifier, and it does not allow UCNs at the start of an identifier. C allows both. In C, The UCNs listed in D.1 must be accepted. Whether or not the corresponding extended characters (if any) are accepted is entirely implementation-defined.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-02 21:49 +0100 |
| Message-ID | <r17clc$1boh$1@gioia.aioe.org> |
| In reply to | #149648 |
On 2/2/2020 8:44 PM, James Kuyper wrote: > On 2/2/20 11:59 AM, Manfred wrote: >> On 2/2/2020 2:14 PM, Richard Damon wrote: > ... >>> If written in the (ugly) \u notation then it is supposed to accept them, >>> but as the actual Unicode character itself, it is a function of what the >>> implementation defines as it extended source character set. >>> >>> My understanding is that the \u notation is designed that for a compiler >>> that does NOT accept full unicode source, you could have a pre-filter >>> phase that processes the source and converts the Unicode characters into >>> Universal characters (the \u and \U notation). It is even written so >>> this could be part of the early phases of the compiler. >> >> In fact the wording used in C++ standard is more explicit on this >> (translation phase 1), although my understanding of the wording of C >> standard mandates that the characters in D.1 be accepted in identifiers >> as well (with the exception of D.2 as initial) > > C++ does not allow an implementation-defined set of extended characters > to appear in an identifier, and it does not allow UCNs at the start of > an identifier. Almost correct: C++ allows UCNs at the start of an identifier, except those in table 3 [lex.name] (which is the counterpart of D.2) C allows both. > > In C, The UCNs listed in D.1 must be accepted. Whether or not the > corresponding extended characters (if any) are accepted is entirely > implementation-defined. > This is the part that is not explicit (to me) in the C standard, when I read translation phase 1.
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-02 16:58 -0500 |
| Message-ID | <r17gn4$447$1@dont-email.me> |
| In reply to | #149654 |
On 2/2/20 3:49 PM, Manfred wrote: > On 2/2/2020 8:44 PM, James Kuyper wrote: >> On 2/2/20 11:59 AM, Manfred wrote: >>> On 2/2/2020 2:14 PM, Richard Damon wrote: >> ... >>>> If written in the (ugly) \u notation then it is supposed to accept them, >>>> but as the actual Unicode character itself, it is a function of what the >>>> implementation defines as it extended source character set. >>>> >>>> My understanding is that the \u notation is designed that for a compiler >>>> that does NOT accept full unicode source, you could have a pre-filter >>>> phase that processes the source and converts the Unicode characters into >>>> Universal characters (the \u and \U notation). It is even written so >>>> this could be part of the early phases of the compiler. >>> >>> In fact the wording used in C++ standard is more explicit on this >>> (translation phase 1), although my understanding of the wording of C >>> standard mandates that the characters in D.1 be accepted in identifiers >>> as well (with the exception of D.2 as initial) >> >> C++ does not allow an implementation-defined set of extended characters >> to appear in an identifier, and it does not allow UCNs at the start of >> an identifier. > Almost correct: C++ allows UCNs at the start of an identifier, except > those in table 3 [lex.name] (which is the counterpart of D.2) > > C allows both. I'd forgotten (if in fact, I ever knew) that C imposes the same restrictions on initial UCNs. I should have reviewed the wording more carefully before posting that comment. The part about an "implementation-defined set of extended characters" is correct, and was my main point. >> In C, The UCNs listed in D.1 must be accepted. Whether or not the >> corresponding extended characters (if any) are accepted is entirely >> implementation-defined. >> > This is the part that is not explicit (to me) in the C standard, when I > read translation phase 1. The only thing that the description of translation phase 1 says which is relevant is "Physical source file multibyte characters are mapped, in an implementation-defined manner, to the source character set." Keeping mind that the extended source character set is a subset of the extended source character set, this implies that extended characters are permitted in C programs, but doesn't say anything to address the issue of where they are permitted. The relevant issue is the syntax of identifiers. The relevant wording is in 6.4.2.1, which identifies "universal-character-name" and "other implementation-defined characters" as two of the permitted forms that an "identifier-nondigit" can take. 6.4.2.1p3 says "An implementation may allow multibyte characters that are not part of the basic source character set to appear in identifiers; which characters and their correspondence to universal character names is implementation-defined."
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-03 15:33 +0100 |
| Message-ID | <r19avh$7l4$1@gioia.aioe.org> |
| In reply to | #149657 |
On 2/2/2020 10:58 PM, James Kuyper wrote: > On 2/2/20 3:49 PM, Manfred wrote: >> On 2/2/2020 8:44 PM, James Kuyper wrote: >>> On 2/2/20 11:59 AM, Manfred wrote: >>>> On 2/2/2020 2:14 PM, Richard Damon wrote: >>> ... >>>>> If written in the (ugly) \u notation then it is supposed to accept them, >>>>> but as the actual Unicode character itself, it is a function of what the >>>>> implementation defines as it extended source character set. >>>>> >>>>> My understanding is that the \u notation is designed that for a compiler >>>>> that does NOT accept full unicode source, you could have a pre-filter >>>>> phase that processes the source and converts the Unicode characters into >>>>> Universal characters (the \u and \U notation). It is even written so >>>>> this could be part of the early phases of the compiler. >>>> >>>> In fact the wording used in C++ standard is more explicit on this >>>> (translation phase 1), although my understanding of the wording of C >>>> standard mandates that the characters in D.1 be accepted in identifiers >>>> as well (with the exception of D.2 as initial) >>> >>> C++ does not allow an implementation-defined set of extended characters >>> to appear in an identifier, and it does not allow UCNs at the start of >>> an identifier. >> Almost correct: C++ allows UCNs at the start of an identifier, except >> those in table 3 [lex.name] (which is the counterpart of D.2) >> >> C allows both. > > I'd forgotten (if in fact, I ever knew) that C imposes the same > restrictions on initial UCNs. I should have reviewed the wording more > carefully before posting that comment. The part about an > "implementation-defined set of extended characters" is correct, and was > my main point. > >>> In C, The UCNs listed in D.1 must be accepted. Whether or not the >>> corresponding extended characters (if any) are accepted is entirely >>> implementation-defined. >>> >> This is the part that is not explicit (to me) in the C standard, when I >> read translation phase 1. > > The only thing that the description of translation phase 1 says which is > relevant is "Physical source file multibyte characters are mapped, in an > implementation-defined manner, to the source character set." And that's different from the C++ standard which adds: "Any source file character not in the basic source character set (5.3) is replaced by the universal-character-name that designates that character". This makes clear that after translation phase 1 any extended character is represented in UCN form, which is used in the syntax of identifiers. In the same clause, C++ also makes clear that "The set of physical source file characters accepted is implementation-defined." Keeping > mind that the extended source character set is a subset of the extended > source character set, this implies that extended characters are > permitted in C programs, but doesn't say anything to address the issue > of where they are permitted. I can't parse "extended source character set is a subset of the extended source character set" > > The relevant issue is the syntax of identifiers. The relevant wording is > in 6.4.2.1, which identifies "universal-character-name" and "other > implementation-defined characters" as two of the permitted forms that an > "identifier-nondigit" can take. 6.4.2.1p3 says "An implementation may > allow multibyte characters that are not part of the basic source > character set to appear in identifiers; which characters and their > correspondence to universal character names is implementation-defined." > The first sentence in that clause is "Each universal character name in an identifier shall designate a character whose encoding in ISO/IEC 10646 falls into one of the ranges specified in D.1." which suggests that D.1 is the set of allowed characters in identifiers. However, I understand that you are saying that in combination with the last sentence the condition of belonging to D.1. is a necessary but not sufficient condition to be allowed in an identifier. This seems to parse, but then how does it work with the possibility of the character being one of "other implementation-defined characters"? This ends up with D.1. being substantially useless.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-03 15:38 +0100 |
| Message-ID | <r19ba2$7l4$2@gioia.aioe.org> |
| In reply to | #149672 |
On 2/3/2020 3:33 PM, Manfred wrote: > This ends up with D.1. being substantially useless. Unless "other implementation-defined characters" means characters that are not (yet) part of the Unicode standard.
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-03 10:25 -0500 |
| Message-ID | <r19e0i$5nv$1@dont-email.me> |
| In reply to | #149672 |
On 2/3/20 9:33 AM, Manfred wrote: > On 2/2/2020 10:58 PM, James Kuyper wrote: >> On 2/2/20 3:49 PM, Manfred wrote: >>> On 2/2/2020 8:44 PM, James Kuyper wrote: >>>> On 2/2/20 11:59 AM, Manfred wrote: >>>>> On 2/2/2020 2:14 PM, Richard Damon wrote: ... >> The only thing that the description of translation phase 1 says which is >> relevant is "Physical source file multibyte characters are mapped, in an >> implementation-defined manner, to the source character set." > > And that's different from the C++ standard which adds: "Any source file > character not in the basic source character set (5.3) is replaced by the > universal-character-name that designates that character". OK - either I'd forgotten about that, or it's relatively new and I didn't notice the change (I stopped paying close attention to changes in the C++ standard around 2002, and only started paying attention again recently, when I was assigned to a project that uses C++). > Keeping >> mind that the extended source character set is a subset of the extended >> source character set, this implies that extended characters are >> permitted in C programs, but doesn't say anything to address the issue >> of where they are permitted. > I can't parse "extended source character set is a subset of the extended > source character set" Sorry, that was intended to say "The extended source character set is a subset of the source character set". >> The relevant issue is the syntax of identifiers. The relevant wording is >> in 6.4.2.1, which identifies "universal-character-name" and "other >> implementation-defined characters" as two of the permitted forms that an >> "identifier-nondigit" can take. 6.4.2.1p3 says "An implementation may >> allow multibyte characters that are not part of the basic source >> character set to appear in identifiers; which characters and their >> correspondence to universal character names is implementation-defined." >> > > The first sentence in that clause is "Each universal character name in > an identifier shall designate a character whose encoding in ISO/IEC > 10646 falls into one of the ranges specified in D.1." which suggests > that D.1 is the set of allowed characters in identifiers. No, it only means that the UCNs described in D.1 are in the set, not that they constitute the entire set. Which is good, because D.1 deliberately excludes the UCNs that corresponding the basic source character set. It would be pretty odd if you weren't allowed to use "identifier" as an identifier - it consists entirely of basic source characters. > However, I understand that you are saying that in combination with the > last sentence the condition of belonging to D.1. is a necessary but not > sufficient condition to be allowed in an identifier. Correct. Basic source characters, and an implementation-defined set of extended characters, are also allowed. > This seems to parse, but then how does it work with the possibility of > the character being one of "other implementation-defined characters"? > This ends up with D.1. being substantially useless. No, D.1 is a list of the permitted UCNs, not extended characters. '\u03C0' is covered by D.1, but 'π' is not. Implementations are permitted to define the map between extended source characters and UCNs. They are required to recognize extended source characters in identifiers as matching the corresponding UCNS. They are required to convert UCNs in character strings and string literals to the corresponding member of the execution character set (if any) during translation phase 5. But extended characters and UCNs are two distinct things, and you'll misunderstand many of the things the standard says about them if you misinterpret them as being the same.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-03 19:11 +0100 |
| Message-ID | <r19nnk$86a$1@gioia.aioe.org> |
| In reply to | #149677 |
On 2/3/2020 4:25 PM, James Kuyper wrote: > On 2/3/20 9:33 AM, Manfred wrote: >> On 2/2/2020 10:58 PM, James Kuyper wrote: >>> On 2/2/20 3:49 PM, Manfred wrote: >>>> On 2/2/2020 8:44 PM, James Kuyper wrote: >>>>> On 2/2/20 11:59 AM, Manfred wrote: >>>>>> On 2/2/2020 2:14 PM, Richard Damon wrote: > ... >>> The only thing that the description of translation phase 1 says which is >>> relevant is "Physical source file multibyte characters are mapped, in an >>> implementation-defined manner, to the source character set." >> >> And that's different from the C++ standard which adds: "Any source file >> character not in the basic source character set (5.3) is replaced by the >> universal-character-name that designates that character". > > OK - either I'd forgotten about that, or it's relatively new and I > didn't notice the change (I stopped paying close attention to changes in > the C++ standard around 2002, and only started paying attention again > recently, when I was assigned to a project that uses C++). > [...] > >>> The relevant issue is the syntax of identifiers. The relevant wording is >>> in 6.4.2.1, which identifies "universal-character-name" and "other >>> implementation-defined characters" as two of the permitted forms that an >>> "identifier-nondigit" can take. 6.4.2.1p3 says "An implementation may >>> allow multibyte characters that are not part of the basic source >>> character set to appear in identifiers; which characters and their >>> correspondence to universal character names is implementation-defined." >>> >> >> The first sentence in that clause is "Each universal character name in >> an identifier shall designate a character whose encoding in ISO/IEC >> 10646 falls into one of the ranges specified in D.1." which suggests >> that D.1 is the set of allowed characters in identifiers. > > No, it only means that the UCNs described in D.1 are in the set, not > that they constitute the entire set. Which is good, because D.1 > deliberately excludes the UCNs that corresponding the basic source > character set. It would be pretty odd if you weren't allowed to use > "identifier" as an identifier - it consists entirely of basic source > characters. I meant "the set of allowed /extended/ characters in identifiers", I thought this was obvious since we are only talking about extended chars here. > >> However, I understand that you are saying that in combination with the >> last sentence the condition of belonging to D.1. is a necessary but not >> sufficient condition to be allowed in an identifier. > > Correct. Basic source characters, and an implementation-defined set of > extended characters, are also allowed. > >> This seems to parse, but then how does it work with the possibility of >> the character being one of "other implementation-defined characters"? >> This ends up with D.1. being substantially useless. > > No, D.1 is a list of the permitted UCNs, not extended characters. > '\u03C0' is covered by D.1, but 'π' is not. This is exactly my point with the C++ standard being more explicit (and more clear): C++ translation phase 1 converts all extended chars into UCNs, so that in the following 'π' is replaced by '\u03C0' in identifiers (and the rest of the source in general, the exception being raw string literals) C leaves this unspecified. Implementations are > permitted to define the map between extended source characters and UCNs. > They are required to recognize extended source characters in identifiers > as matching the corresponding UCNS. They are required to convert UCNs in > character strings and string literals to the corresponding member of the > execution character set (if any) during translation phase 5. But > extended characters and UCNs are two distinct things, and you'll > misunderstand many of the things the standard says about them if you > misinterpret them as being the same. > I know they are different things. Nevertheless I think it is pretty ambiguous that 'π' and '\u03C0' behave differently.
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-03 15:59 -0500 |
| Message-ID | <r1a1kg$54v$1@dont-email.me> |
| In reply to | #149690 |
On 2/3/20 1:11 PM, Manfred wrote: > On 2/3/2020 4:25 PM, James Kuyper wrote: >> On 2/3/20 9:33 AM, Manfred wrote: ... >>> The first sentence in that clause is "Each universal character name in >>> an identifier shall designate a character whose encoding in ISO/IEC >>> 10646 falls into one of the ranges specified in D.1." which suggests >>> that D.1 is the set of allowed characters in identifiers. >> >> No, it only means that the UCNs described in D.1 are in the set, not >> that they constitute the entire set. Which is good, because D.1 >> deliberately excludes the UCNs that corresponding the basic source >> character set. It would be pretty odd if you weren't allowed to use >> "identifier" as an identifier - it consists entirely of basic source >> characters. > I meant "the set of allowed /extended/ characters in identifiers", I > thought this was obvious since we are only talking about extended chars > here. No, that doesn't fit either. Code containing a UCN not on that list has undefined behavior, since it violates the "shall" in 6.4.2.1p3, the clause cited above. However, an implementation is allowed to define the behavior that the standard itself leaves undefined, by allowing a UCN that is not in that list. A fully conforming implementation is also allowed to support extended characters whose Unicode encoding is not on that list (whether or not it allows the corresponding UCN). It's also allowed to support extended characters that have no Unicode encoding (though such characters would be hard to find). Both of those have implementation-defined behavior rather than undefined behavior. ... >>> This seems to parse, but then how does it work with the possibility of >>> the character being one of "other implementation-defined characters"? >>> This ends up with D.1. being substantially useless. >> >> No, D.1 is a list of the permitted UCNs, not extended characters. >> '\u03C0' is covered by D.1, but 'π' is not. > > This is exactly my point with the C++ standard being more explicit (and > more clear): C++ translation phase 1 converts all extended chars into > UCNs, so that in the following 'π' is replaced by '\u03C0' in > identifiers (and the rest of the source in general, the exception being > raw string literals) > C leaves this unspecified. More precisely, it fails to mandate such a conversion. There's an important difference. 5.1.1.2p1 says: "Physical source file multibyte characters are mapped, in an implementation-defined manner, to the source character set (introducing new-line characters for end-of-line indicators) if necessary." If you read the implementation's documentation of that mapping, and create a physical source file that will be mapped to particular source code characters, an implementation's freedom to transform the source code beyond that point is limited to cases where the standard gives explicit permission to do so. If you create a physical source code file that will, when mapped to source code characters, contain the sequence of characters \u03C0, an implementation is NOT free to convert that sequence into 'π', or vice-versa. This matters, because both #error directives and use of the stringizing operator in the expansion of a macro are defined in terms of taking the source code characters, in the form that they have at the end of translation phase 3, and doing something with them that would have observable behavior. > Implementations are >> permitted to define the map between extended source characters and UCNs. >> They are required to recognize extended source characters in identifiers >> as matching the corresponding UCNS. They are required to convert UCNs in >> character strings and string literals to the corresponding member of the >> execution character set (if any) during translation phase 5. But >> extended characters and UCNs are two distinct things, and you'll >> misunderstand many of the things the standard says about them if you >> misinterpret them as being the same. >> > > I know they are different things. Nevertheless I think it is pretty > ambiguous that 'π' and '\u03C0' behave differently. For #error directives and stringizing, I think it's quite clear that they behave differently. However, after translation phase 4, I have to agree with you in the case of identifiers. I was positive that the standard contained text somewhere requiring implementations to treat an identifier that contains an extended character to be treated as equivalent to an identifier that replaced that character with the (implementation-defined) corresponding UCN - but I've been unable to locate such a clause. I also believe that the case of the characters used in a UCN don't affect how identifiers match. Neither does using \U000003C0 rather than \U03C0 - but I can't locate corresponding text in the standard. In character constants and string literals, however, UCNs disappear during translation phase 5. If a source extended character and a UCN both get mapped to the same execution character during that phase, they must behave equivalently from that point onward.
[toc] | [prev] | [next] | [standalone]
| From | Manfred <noname@add.invalid> |
|---|---|
| Date | 2020-02-04 16:51 +0100 |
| Message-ID | <r1c3t4$174b$1@gioia.aioe.org> |
| In reply to | #149693 |
On 2/3/2020 9:59 PM, James Kuyper wrote:
> On 2/3/20 1:11 PM, Manfred wrote:
>> On 2/3/2020 4:25 PM, James Kuyper wrote:
>>> On 2/3/20 9:33 AM, Manfred wrote:
> ...
>>>> The first sentence in that clause is "Each universal character name in
>>>> an identifier shall designate a character whose encoding in ISO/IEC
>>>> 10646 falls into one of the ranges specified in D.1." which suggests
>>>> that D.1 is the set of allowed characters in identifiers.
>>>
>>> No, it only means that the UCNs described in D.1 are in the set, not
>>> that they constitute the entire set. Which is good, because D.1
>>> deliberately excludes the UCNs that corresponding the basic source
>>> character set. It would be pretty odd if you weren't allowed to use
>>> "identifier" as an identifier - it consists entirely of basic source
>>> characters.
>> I meant "the set of allowed /extended/ characters in identifiers", I
>> thought this was obvious since we are only talking about extended chars
>> here.
>
> No, that doesn't fit either. Code containing a UCN not on that list has
> undefined behavior,
I'm sorry, just for the sake of sanity I will repeat my sentence from above:
"... which suggests that D.1 is the set of allowed (implied: extended)
characters in identifiers".
I have no idea why you brought up the basic character set first, and now
UCNs /not/ in D.1.
since it violates the "shall" in 6.4.2.1p3, the
> clause cited above. However, an implementation is allowed to define the
> behavior that the standard itself leaves undefined, by allowing a UCN
> that is not in that list.
> A fully conforming implementation is also allowed to support extended
> characters whose Unicode encoding is not on that list (whether or not it
> allows the corresponding UCN). It's also allowed to support extended
> characters that have no Unicode encoding (though such characters would
> be hard to find). Both of those have implementation-defined behavior
> rather than undefined behavior.
>
[...]
>>
>> I know they are different things. Nevertheless I think it is pretty
>> ambiguous that 'π' and '\u03C0' behave differently.
>
> For #error directives and stringizing, I think it's quite clear that
> they behave differently.
For the record (I know this one implementation and not the standard,
just saying):
$ cat u.c
#include <stdio.h>
#define STR(x) #x
int main()
{
printf("π: %s\n\\u03C0: %s\n",
STR(π),
STR(\u03C0));
}
$ cc -std=c11 -Wall -pedantic u.c && ./a.out
π: π
\u03C0: π
>
> However, after translation phase 4, I have to agree with you in the case
> of identifiers. I was positive that the standard contained text
> somewhere requiring implementations to treat an identifier that contains
> an extended character to be treated as equivalent to an identifier that
> replaced that character with the (implementation-defined) corresponding
> UCN - but I've been unable to locate such a clause. I also believe that
> the case of the characters used in a UCN don't affect how identifiers
> match. Neither does using \U000003C0 rather than \U03C0 - but I can't
> locate corresponding text in the standard.
At least we agree on something ;)
>
> In character constants and string literals, however, UCNs disappear
> during translation phase 5. If a source extended character and a UCN
> both get mapped to the same execution character during that phase, they
> must behave equivalently from that point onward.
>
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-04 16:05 -0500 |
| Message-ID | <r1cma2$5vd$1@dont-email.me> |
| In reply to | #149728 |
On 2/4/20 10:51 AM, Manfred wrote: > On 2/3/2020 9:59 PM, James Kuyper wrote: >> On 2/3/20 1:11 PM, Manfred wrote: >>> On 2/3/2020 4:25 PM, James Kuyper wrote: >>>> On 2/3/20 9:33 AM, Manfred wrote: >> ... >>>>> The first sentence in that clause is "Each universal character name in >>>>> an identifier shall designate a character whose encoding in ISO/IEC >>>>> 10646 falls into one of the ranges specified in D.1." which suggests >>>>> that D.1 is the set of allowed characters in identifiers. >>>> >>>> No, it only means that the UCNs described in D.1 are in the set, not >>>> that they constitute the entire set. Which is good, because D.1 >>>> deliberately excludes the UCNs that corresponding the basic source >>>> character set. It would be pretty odd if you weren't allowed to use >>>> "identifier" as an identifier - it consists entirely of basic source >>>> characters. >>> I meant "the set of allowed /extended/ characters in identifiers", I >>> thought this was obvious since we are only talking about extended chars >>> here. >> >> No, that doesn't fit either. Code containing a UCN not on that list has >> undefined behavior, > > I'm sorry, just for the sake of sanity I will repeat my sentence from above: > "... which suggests that D.1 is the set of allowed (implied: extended) > characters in identifiers". > I have no idea why you brought up the basic character set first, I mentioned basic character set characters because I took your words to mean exactly what they said: "D.1 is the set of allowed characters in identifiers.". It didn't occur to me, until you explained otherwise, that what you actual meant was "D.1 is the set of allowed extended characters in identifiers." There's two reasons for this: the word "extended" didn't appear in your sentence, and is not implied by the context (6.2.4.1 doesn't say anything about extended characters until two sentences after the one you were talking about). The second reason was that D.1 doesn't list any extended characters; it only lists "hexadecimal code values that are valid in universal character names". > and now UCNs /not/ in D.1. I mentioned UCNs not in D.1 to make the distinction between what 6.4.2.1p3 says about D.1, and your incorrect conclusions. D.1 is only about UCNs, not extended characters. What 6.4.2.1p3 says is that UCNs shall fall into one of the ranges specified in D.1. Violating a shall that occurs outside of a constraints section means that the behavior is undefined. That exact same meaning can be expressed negatively by saying that UCNs that don't fall into one of those ranges have undefined behavior. All I was doing was re-wording 6.4.2.1p3 to emphasize it's failure to impose the requirements on extended characters that you were inferring from it. >>> I know they are different things. Nevertheless I think it is pretty >>> ambiguous that 'π' and '\u03C0' behave differently. >> >> For #error directives and stringizing, I think it's quite clear that >> they behave differently. > > For the record (I know this one implementation and not the standard, > just saying): I'm not surprised by the existence of an implementation that disagrees with my interpretation of the standard on this point. I think it's clear, but manifestly not everyone agrees with me on that point.
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2020-02-19 12:17 -0800 |
| Message-ID | <86h7zme3gs.fsf@linuxsc.com> |
| In reply to | #149728 |
Manfred <noname@add.invalid> writes: > On 2/3/2020 9:59 PM, James Kuyper wrote: > >> [long discussion of UCNs and extended characters in identifiers] Let me see if I can clarify what the C standard prescribes here. The source character set is divided into a basic character set and a set of extended characters, which is disjoint from the basic character set. Somewhat confusingly, the term "extended character set" includes both the basic character set and the set of extended characters. The set of extended characters is locale-specific. That means it is implementation-defined, but the definition is allowed to depend on which locale is in force. A universal character name is sequence of characters, all of which are part of the basic character set, in one of two particular forms: six characters of the form \unnnn, or ten characters of the form \Unnnnnnnn, where all the n's are hexidecimal digits. Not all sequences of this form are valid UCNs: The value of the n's must be greater than DFFF, or less than D800 and greater than 009F, or 0024, or 0040, or 0060 (which correspond to dollar, at sign, and backquote, respectively); any seeming UCN that transgresses this rule must provoke a diagnostic. The mapping from UCNs to characters is well-defined, by taking the "short identifier" (the hex digits) of the UCN, and looking up the short identifier in ISO/IEC 10646. All universal character names are sequences of characters in the basic character set, and thus are disjoint from the set of extended characters. The character a UCN represents might or might not be the same as an extended character (more on that farther down), but the UCN itself is not any extended character. An implementation must allow some UCNs to be used in identifiers. In particular, it must allow any UCN in the D.1 ranges to be used in any position after the first, and it must allow any UCN in the D.1 ranges but not in the D.2 ranges in the first position. My reading of the C standard is that these range limitations are meant to be imposed on the implementation, so any "identifer" that uses a UCN not meeting these criteria must provoke a diagnostic. An implementation /may/ allow some members of the set of extended characters to be used in identifiers. Which extended characters are allowed, and (presumably) which extended characters are allowed initially, is implementation-defined. On top of that, how or whether any extended character corresponds to a UCN is implementation-defined. There is no restriction on which extended characters, if any, an implementation may allow to be part of an identifier (remembering that an extended character may not belong to the basic character set), as long as they are documented in the implementation-supplied documentation for implementation-defined behavior. Does all this help clear up the relationship between universal character names and extended characters?
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2020-02-02 17:09 +0100 |
| Message-ID | <r16s70$tdh$2@dont-email.me> |
| In reply to | #149620 |
On 01/02/2020 19:10, Bart wrote:
> On 01/02/2020 17:37, Ben Bacarisse wrote:
>> F Russell <fr@random.info> writes:
>>
>>> Should it be possible to write C source files in Unicode?
>>>
>>> Now, only former-ASCII characters are allowed but it would
>>> be nice to be able to write:
>>
>> No, it is allowed now:
>>
>> $ cat t.c
>> #include <stdio.h>
>> #include <math.h>
>>
>> int main()
>> {
>> double φ = 0.5, ω2 = 0.2, θ = -0.5, ω3 = 0.1;
>> printf("%g\n", cos(φ) * ω2 - sin(θ) * ω3);
>> }
>> $ clang t.c -lm
>> $ ./a.out
>> 0.223459
>
> That works with clang on rextester.com, but not with gcc? (Same with
> bcc, where the problem is the first byte of the phi character when using
> utf8.)
>
gcc's support for extended identifiers is, let us say, "interesting".
It supports them, but not in a way that any sane person would want to use.
> Doesn't C restrict identifier names to [A-Z a-z 0-9 _] ?
>
> If not, then what /is/ allowed, as there could be hundreds of thousands
> of allowable characters?
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-02 14:46 -0500 |
| Message-ID | <r178u5$eqm$2@dont-email.me> |
| In reply to | #149637 |
On 2/2/20 11:09 AM, David Brown wrote: ... > gcc's support for extended identifiers is, let us say, "interesting". > It supports them, but not in a way that any sane person would want to use. Well, that description got me interested. Would you care to expand on that comment?
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2020-02-02 21:39 +0100 |
| Message-ID | <r17c1n$cnp$1@dont-email.me> |
| In reply to | #149649 |
On 02/02/2020 20:46, James Kuyper wrote: > On 2/2/20 11:09 AM, David Brown wrote: > ... >> gcc's support for extended identifiers is, let us say, "interesting". >> It supports them, but not in a way that any sane person would want to use. > > Well, that description got me interested. Would you care to expand on > that comment? > Basically, you had to use \u1234 or \U12345678 syntax for your Unicode letters in identifiers. So if you wanted "const double π = 3.14;", you'd have to write "const double \u03C0 = 3.14;". That is with the "-fextended-identifiers" on to enable the feature. It might be technically correct to say gcc supports Unicode extended identifiers, but in practice it is not particularly convenient. However, I gather that gcc 10 should support them properly, which will be interesting. I am still sceptical about how useful they would be, but I am sure many non-English speakers will want to use their local alphabets to some extent. (Here in Norway, people generally program in English - the lack of åøæ in identifiers is not usually a big issue.) <https://gcc.gnu.org/wiki/FAQ#What_is_the_status_of_adding_the_UTF-8_support_for_identifier_names_in_GCC.3F> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67224>
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2020-02-02 17:07 -0500 |
| Message-ID | <r17h80$ecf$1@dont-email.me> |
| In reply to | #149652 |
On 2/2/20 3:39 PM, David Brown wrote: > On 02/02/2020 20:46, James Kuyper wrote: >> On 2/2/20 11:09 AM, David Brown wrote: >> ... >>> gcc's support for extended identifiers is, let us say, "interesting". >>> It supports them, but not in a way that any sane person would want to use. >> >> Well, that description got me interested. Would you care to expand on >> that comment? >> > > Basically, you had to use \u1234 or \U12345678 syntax for your Unicode > letters in identifiers. So if you wanted "const double π = 3.14;", > you'd have to write "const double \u03C0 = 3.14;". That is with the > "-fextended-identifiers" on to enable the feature. That's not an example of "support for extended identifiers" - it doesn't involve the use of a single extended character. That's simply of meeting the standard's requirements for the handling of UCN's, while failing to support extended characters in identifiers. I was expecting, from your comment, something much more bizarre than that. > I am still sceptical about how useful they would be, but I am sure many > non-English speakers will want to use their local alphabets to some > extent. (Here in Norway, people generally program in English - the lack > of åøæ in identifiers is not usually a big issue.) If support were guaranteed, rather than implementation-defined, there's a fair number of cases where I'd be inclined to take advantage of that feature. As a native English speaker, whose employers and clients have always been native English speakers, I have little or no use of extended characters for that purpose. However, since my code often implements mathematical and scientific equations, it would be attractive to use actual Greek letters in identifiers, rather than the English names of those letters.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2020-02-03 13:12 +0100 |
| Message-ID | <r192n7$1qh$1@dont-email.me> |
| In reply to | #149658 |
On 02/02/2020 23:07, James Kuyper wrote: > On 2/2/20 3:39 PM, David Brown wrote: >> On 02/02/2020 20:46, James Kuyper wrote: >>> On 2/2/20 11:09 AM, David Brown wrote: >>> ... >>>> gcc's support for extended identifiers is, let us say, "interesting". >>>> It supports them, but not in a way that any sane person would want to use. >>> >>> Well, that description got me interested. Would you care to expand on >>> that comment? >>> >> >> Basically, you had to use \u1234 or \U12345678 syntax for your Unicode >> letters in identifiers. So if you wanted "const double π = 3.14;", >> you'd have to write "const double \u03C0 = 3.14;". That is with the >> "-fextended-identifiers" on to enable the feature. > > That's not an example of "support for extended identifiers" - it doesn't > involve the use of a single extended character. That's simply of meeting > the standard's requirements for the handling of UCN's, while failing to > support extended characters in identifiers. I was expecting, from your > comment, something much more bizarre than that. Now I'm confused. This "π" identifier /is/ an example of an extended identifier. My point is that gcc supports extended identifiers with Unicode characters in their names (following the rules in Annex D, which I believe allows π). But gcc does not allow using UTF-8 or other Unicode encodings to write these characters in the source code - they are only allowed using \u or \U escape codes. > >> I am still sceptical about how useful they would be, but I am sure many >> non-English speakers will want to use their local alphabets to some >> extent. (Here in Norway, people generally program in English - the lack >> of åøæ in identifiers is not usually a big issue.) > > If support were guaranteed, rather than implementation-defined, there's > a fair number of cases where I'd be inclined to take advantage of that > feature. As a native English speaker, whose employers and clients have > always been native English speakers, I have little or no use of extended > characters for that purpose. However, since my code often implements > mathematical and scientific equations, it would be attractive to use > actual Greek letters in identifiers, rather than the English names of > those letters. > I think there is a lot of potential use for additional characters in code that is only used in a small group - either because the group needs to work with a non-English language, or because within the group you decide that your code would be clearer with some Greek letters for mathematical code (or whatever suits). Then you as a group can decide on which characters to use, and how you will type them. It is when code is shared or re-used amongst a wider audience that problems will occur. You'll get people who are annoyed because someone wrote π and they can't type it sensibly, or because they can't figure out which "beta" to use from βꞵ𝛽𝜷ß (yes, the last one is not actually a beta), or because capital beta Β looks a lot like B to the reader, but is completely different to the compiler. And if you have a small enough group where some Unicode letters would help, and not have any of these disadvantages, then I expect you will be in a position to define exactly which compiler you will use, and implementation-defined behaviour could be perfectly acceptable.
[toc] | [prev] | [next] | [standalone]
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
Back to top | Article view | comp.lang.c
csiph-web