Path: csiph.com!eternal-september.org!feeder.eternal-september.org!reader01.eternal-september.org!.POSTED!not-for-mail From: Tim Rentsch Newsgroups: comp.lang.c Subject: Re: Unicode in C Sources Date: Wed, 19 Feb 2020 12:17:55 -0800 Organization: A noiseless patient Spider Lines: 60 Message-ID: <86h7zme3gs.fsf@linuxsc.com> References: <87h80axl7s.fsf@bsb.me.uk> <5ajZF.602301$CO7.252756@fx36.am4> <1XzZF.178637$hD4.62031@fx47.iad> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: reader02.eternal-september.org; posting-host="24fb35cc0bcbe230f7108cd93f73ffe7"; logging-data="30894"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/A1WKUqSAo+oXDJ3cjTTEF346y2a/DKZk=" User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.4 (gnu/linux) Cancel-Lock: sha1:Ro6hQu51tyvDAZg7GIaq0HuY7rY= sha1:IxhpqQ2Sjj5li3KI26B1REpVHCM= Xref: csiph.com comp.lang.c:150084 Manfred writes: > On 2/3/2020 9:59 PM, James Kuyper wrote: > >> [long discussion of UCNs and extended characters in identifiers] Let me see if I can clarify what the C standard prescribes here. The source character set is divided into a basic character set and a set of extended characters, which is disjoint from the basic character set. Somewhat confusingly, the term "extended character set" includes both the basic character set and the set of extended characters. The set of extended characters is locale-specific. That means it is implementation-defined, but the definition is allowed to depend on which locale is in force. A universal character name is sequence of characters, all of which are part of the basic character set, in one of two particular forms: six characters of the form \unnnn, or ten characters of the form \Unnnnnnnn, where all the n's are hexidecimal digits. Not all sequences of this form are valid UCNs: The value of the n's must be greater than DFFF, or less than D800 and greater than 009F, or 0024, or 0040, or 0060 (which correspond to dollar, at sign, and backquote, respectively); any seeming UCN that transgresses this rule must provoke a diagnostic. The mapping from UCNs to characters is well-defined, by taking the "short identifier" (the hex digits) of the UCN, and looking up the short identifier in ISO/IEC 10646. All universal character names are sequences of characters in the basic character set, and thus are disjoint from the set of extended characters. The character a UCN represents might or might not be the same as an extended character (more on that farther down), but the UCN itself is not any extended character. An implementation must allow some UCNs to be used in identifiers. In particular, it must allow any UCN in the D.1 ranges to be used in any position after the first, and it must allow any UCN in the D.1 ranges but not in the D.2 ranges in the first position. My reading of the C standard is that these range limitations are meant to be imposed on the implementation, so any "identifer" that uses a UCN not meeting these criteria must provoke a diagnostic. An implementation /may/ allow some members of the set of extended characters to be used in identifiers. Which extended characters are allowed, and (presumably) which extended characters are allowed initially, is implementation-defined. On top of that, how or whether any extended character corresponds to a UCN is implementation-defined. There is no restriction on which extended characters, if any, an implementation may allow to be part of an identifier (remembering that an extended character may not belong to the basic character set), as long as they are documented in the implementation-supplied documentation for implementation-defined behavior. Does all this help clear up the relationship between universal character names and extended characters?