Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #148065 > unrolled thread

How to parse attributes?

Back to article view | Back to comp.lang.c

Contents

  How to parse attributes? Philipp Klaus Krause <pkk@spth.de> - 2019-11-20 09:41 +0100
    Re: How to parse attributes? David Brown <david.brown@hesbynett.no> - 2019-11-20 10:05 +0100
      Re: How to parse attributes? Philipp Klaus Krause <pkk@spth.de> - 2019-11-20 11:05 +0100
        Re: How to parse attributes? David Brown <david.brown@hesbynett.no> - 2019-11-20 12:35 +0100
    Re: How to parse attributes? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-11-20 11:06 +0000
    Re: How to parse attributes? Öö Tiib <ootiib@hot.ee> - 2019-11-20 23:16 -0800
      Re: How to parse attributes? Philipp Klaus Krause <pkk@spth.de> - 2019-11-21 10:39 +0100
        Re: How to parse attributes? Öö Tiib <ootiib@hot.ee> - 2019-11-21 02:51 -0800
          Re: How to parse attributes? James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-11-21 05:00 -0800
        Re: How to parse attributes? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-11-22 02:21 +0000
          Re: How to parse attributes? Keith Thompson <kst-u@mib.org> - 2019-11-21 20:41 -0800
            Re: How to parse attributes? Philipp Klaus Krause <pkk@spth.de> - 2019-11-22 09:15 +0100
            Re: How to parse attributes? David Brown <david.brown@hesbynett.no> - 2019-11-22 09:48 +0100
              Re: How to parse attributes? "A. K." <minforth@arcor.de> - 2019-11-22 02:00 -0800
                Re: How to parse attributes? David Brown <david.brown@hesbynett.no> - 2019-11-22 11:41 +0100
                Re: How to parse attributes? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-11-24 14:02 +0000
                  Re: How to parse attributes? raltbos@xs4all.nl (Richard Bos) - 2019-11-25 16:21 +0000
              Re: How to parse attributes? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-11-22 17:46 +0000
                Re: How to parse attributes? Keith Thompson <kst-u@mib.org> - 2019-11-22 10:53 -0800
                  Re: How to parse attributes? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-11-23 00:08 +0000
      Re: How to parse attributes? scott@slp53.sl.home (Scott Lurndal) - 2019-11-21 15:57 +0000
      Re: How to parse attributes? Bonita Montero <Bonita.Montero@gmail.com> - 2019-11-22 08:21 +0100
      Re: How to parse attributes? Bart <bc@freeuk.com> - 2019-11-22 12:11 +0000

Page 1 of 2  [1] 2  Next page →

#148065 — How to parse attributes?

C2X has attributes.
Implementation in the front-end seems straigtforward, except for one
bit: Keywords are treated as normal identifiers in attributes.

E.g. the following

int i;

void f(void)
{
	[[const::int]] i++;
}

is valid C.

So far, a plain lex / yacc parser, would recognize keywords as tokens in
the lexer, where we don't know if we're in an attribute-specifier.
That makes it hard to allow keywords in attributes. Of course one could
create lots of rules for yacc for all the individual keywords, but that
doesn't look like a clean solution to me.

Is there an easy solution that I overlooked?

Philipp

[toc] | [next] | [standalone]

#148066

On 20/11/2019 09:41, Philipp Klaus Krause wrote:
> C2X has attributes.

Does it?  I don't see anything in the N2346 draft of C2a.

> Implementation in the front-end seems straigtforward, except for one
> bit: Keywords are treated as normal identifiers in attributes.
> 
> E.g. the following
> 
> int i;
> 
> void f(void)
> {
> 	[[const::int]] i++;
> }
> 
> is valid C.
> 
> So far, a plain lex / yacc parser, would recognize keywords as tokens in
> the lexer, where we don't know if we're in an attribute-specifier.
> That makes it hard to allow keywords in attributes. Of course one could
> create lots of rules for yacc for all the individual keywords, but that
> doesn't look like a clean solution to me.
> 
> Is there an easy solution that I overlooked?
> 
> Philipp
> 

I know C++ has attributes, but C++ compilers have long ago given up on
lex/yacc type parsers.  Most modern C compilers are also C++ compilers,
so I suppose fitting lex/yacc style is not a consideration for new
features in C these days.

[toc] | [prev] | [next] | [standalone]

#148069

Am 20.11.19 um 10:05 schrieb David Brown:
> On 20/11/2019 09:41, Philipp Klaus Krause wrote:
>> C2X has attributes.
> 
> Does it?  I don't see anything in the N2346 draft of C2a.
> 

I use N2455.
>>
> 
> I know C++ has attributes, but C++ compilers have long ago given up on
> lex/yacc type parsers.  Most modern C compilers are also C++ compilers,
> so I suppose fitting lex/yacc style is not a consideration for new
> features in C these days.
> 

There are far more C compilers than C++ compilers.

Philipp

[toc] | [prev] | [next] | [standalone]

#148074

On 20/11/2019 11:05, Philipp Klaus Krause wrote:
> Am 20.11.19 um 10:05 schrieb David Brown:
>> On 20/11/2019 09:41, Philipp Klaus Krause wrote:
>>> C2X has attributes.
>>
>> Does it?  I don't see anything in the N2346 draft of C2a.
>>
> 
> I use N2455.
>>>
>>
>> I know C++ has attributes, but C++ compilers have long ago given up on
>> lex/yacc type parsers.  Most modern C compilers are also C++ compilers,
>> so I suppose fitting lex/yacc style is not a consideration for new
>> features in C these days.
>>
> 
> There are far more C compilers than C++ compilers.
> 

There are.  But how many of these are modern C compilers that are
regularly updated with the latest standards?  The vast majority of C
compilers out there barely handle C99.  When you look at C compilers
that support C11 (at least roughly), I think you will find only a few,
and most of these also support C++.  You have gcc and clang, other PC or
*nix toolchains like Intel, Pathscale, IBM, and the "big" embedded
developers like IAR and Green Hills.  They all do C++.  Most use EDG's
front ends.  (In fact, many plain C compilers use third-party front
ends, not their own lex/yacc parsers.)

[toc] | [prev] | [next] | [standalone]

#148072

Philipp Klaus Krause <pkk@spth.de> writes:

> C2X has attributes.
> Implementation in the front-end seems straigtforward, except for one
> bit: Keywords are treated as normal identifiers in attributes.
>
> E.g. the following
>
> int i;
>
> void f(void)
> {
> 	[[const::int]] i++;
> }
>
> is valid C.
>
> So far, a plain lex / yacc parser, would recognize keywords as tokens in
> the lexer, where we don't know if we're in an attribute-specifier.
> That makes it hard to allow keywords in attributes. Of course one could
> create lots of rules for yacc for all the individual keywords, but that
> doesn't look like a clean solution to me.
>
> Is there an easy solution that I overlooked?

My first thought is that it should be simple.  Can you not have the grammar
rules for attributes use an "attribute_id" rule that includes keywords?

Maybe that this what you are trying to avoid...  If so I'd try to handle
this with lex start conditions.  It should be relatively simple to treat
symbols inside [[ ... ]] differently at the lexical level, but I've not
checked the C specification.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#148086

On Wednesday, 20 November 2019 10:41:27 UTC+2, Philipp Klaus Krause  wrote:
> C2X has attributes.
> Implementation in the front-end seems straigtforward, except for one
> bit: Keywords are treated as normal identifiers in attributes.
> 
> E.g. the following
> 
> int i;
> 
> void f(void)
> {
> 	[[const::int]] i++;
> }
> 
> is valid C.
> 
> So far, a plain lex / yacc parser, would recognize keywords as tokens in
> the lexer, where we don't know if we're in an attribute-specifier.
> That makes it hard to allow keywords in attributes. Of course one could
> create lots of rules for yacc for all the individual keywords, but that
> doesn't look like a clean solution to me.
> 
> Is there an easy solution that I overlooked?

I haven't had reasons to make lex/yacc parser for 15 years but I
vaguely remember that it wasn't difficult for to make lexer to realize
that we are in /* const::int */ comment or that we are in 
"  const::int " string literal so what is the  problem for it to
realize that we are in [[ const::int ]] attribute now?

[toc] | [prev] | [next] | [standalone]

#148087

Am 21.11.19 um 08:16 schrieb Öö Tiib:
> 
> I haven't had reasons to make lex/yacc parser for 15 years but I
> vaguely remember that it wasn't difficult for to make lexer to realize
> that we are in /* const::int */ comment or that we are in 
> "  const::int " string literal so what is the  problem for it to
> realize that we are in [[ const::int ]] attribute now?
> 

I guess the main problem is that I am not really familiar with lex /
yacc. The situation with the attribute is a bit different from string
literals though:

1) Inside a string literal, everything is just text. The inside of the
attribute still should be tokenized.

2) [ is atoken that has many uses in C. We don't really know we're in an
attribute while in lex, that knowledge is only there yacc.

Philipp

[toc] | [prev] | [next] | [standalone]

#148088

On Thursday, 21 November 2019 11:39:51 UTC+2, Philipp Klaus Krause  wrote:
> Am 21.11.19 um 08:16 schrieb Öö Tiib:
> > 
> > I haven't had reasons to make lex/yacc parser for 15 years but I
> > vaguely remember that it wasn't difficult for to make lexer to realize
> > that we are in /* const::int */ comment or that we are in 
> > "  const::int " string literal so what is the  problem for it to
> > realize that we are in [[ const::int ]] attribute now?
> > 
> 
> I guess the main problem is that I am not really familiar with lex /
> yacc. 

What I used more than decade ago were Flex and Bison and
have forgot most details so am likely not best expert there
too. But back then it felt simple. When single monolithic parser
gets too huge to parse everything then use cascade of several
parsers.
 
    
> The situation with the attribute is a bit different from string
> literals though:
> 
> 1) Inside a string literal, everything is just text. The inside of the
> attribute still should be tokenized.

That is not strictly true. Inside string literal we have also certain 
tokens (escape sequences).  Similarly between and [[ ]] we can
have tokens that may be keywords or other names. 

> 2) [ is atoken that has many uses in C. We don't really know we're in an
> attribute while in lex, that knowledge is only there yacc.

Note that lex is greedy so / is token with many usages and * is token with
many usages but /* blah blah */ can be also made token. 

[toc] | [prev] | [next] | [standalone]

#148092

On Thursday, November 21, 2019 at 5:51:38 AM UTC-5, Öö Tiib wrote:
> On Thursday, 21 November 2019 11:39:51 UTC+2, Philipp Klaus Krause  wrote:
...
> > The situation with the attribute is a bit different from string
> > literals though:
> > 
> > 1) Inside a string literal, everything is just text. The inside of the
> > attribute still should be tokenized.
> 
> That is not strictly true. Inside string literal we have also certain 
> tokens (escape sequences).  Similarly between and [[ ]] we can
> have tokens that may be keywords or other names.

Those are not tokens in the sense that the C++ standard defines the
term. See sections 5.4, 5.5, and 5.6 for the full details. Escape
sequences are part of the definition of character literals (5.13.3) and
string literals (5.13.5), and since two of those sequences are \' and
\", it is necessary to parse them correctly in order to determine where
character and string literals end - but it is the literals themselves,
and not the escape sequences that they contain, which are considered to
be tokens as the term is defined by the C++ standard.

[toc] | [prev] | [next] | [standalone]

#148111

Philipp Klaus Krause <pkk@spth.de> writes:

> I guess the main problem is that I am not really familiar with lex /
> yacc. The situation with the attribute is a bit different from string
> literals though:
>
> 1) Inside a string literal, everything is just text. The inside of the
> attribute still should be tokenized.

... and parsed.

> 2) [ is atoken that has many uses in C. We don't really know we're in an
> attribute while in lex, that knowledge is only there yacc.

It would have been simpler to handle if [[ had been made a new
punctuator.  That would not be odd; both + and ++ are distinct tokens.
However, it seems an attribute specification starts with two separate [
tokens, so if you need to use lex start conditions to alter the lexing
inside [[ and ]], you will need to use a yacc action to trigger that.
Ask in a yacc/bison group to get the details.

But I am not 100% sure you have to do this.  What is your source for
keywords being regarded as identifiers?  In the usual C grammar, the
rule for "identifier" is very general (the usual letters and digits
definitions), but keywords are considered to be a separate category.
Thus the grammar item "identifier" really means "things that have the
right form but which aren't keywords".

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#148112

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Philipp Klaus Krause <pkk@spth.de> writes:
>> I guess the main problem is that I am not really familiar with lex /
>> yacc. The situation with the attribute is a bit different from string
>> literals though:
>>
>> 1) Inside a string literal, everything is just text. The inside of the
>> attribute still should be tokenized.
>
> ... and parsed.
>
>> 2) [ is atoken that has many uses in C. We don't really know we're in an
>> attribute while in lex, that knowledge is only there yacc.
>
> It would have been simpler to handle if [[ had been made a new
> punctuator.  That would not be odd; both + and ++ are distinct tokens.
> However, it seems an attribute specification starts with two separate [
> tokens, so if you need to use lex start conditions to alter the lexing
> inside [[ and ]], you will need to use a yacc action to trigger that.
> Ask in a yacc/bison group to get the details.

Two consecutive [ tokens can't appear in a valid C program (prior to the
presumed introduction of attributes in C20), but two consecutive ]
tokens can.  Making [[ but not ]] a single token would have been odd.
Making both [[ and ]] single token without any other changes would break
expressions like a[b[i]].

There is precedent in C++ for dealing with nested <> template
delimiters, but that was needed because >> is already a token.  Any
problems with [[ and ]] didn't require such a solution.

> But I am not 100% sure you have to do this.  What is your source for
> keywords being regarded as identifiers?  In the usual C grammar, the
> rule for "identifier" is very general (the usual letters and digits
> definitions), but keywords are considered to be a separate category.
> Thus the grammar item "identifier" really means "things that have the
> right form but which aren't keywords".

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2455.pdf (C20 draft)
6.7.11.1p3:

    An attribute specifier that contains no attributes has no
    effect. The order in which attribute tokens appear in an
    attribute list is not significant. If a keyword (6.4.1) that
    satisfies the syntactic requirements of an identifier (6.4.2) is
    contained in an attribute token, it is considered an identifier.
    A strictly conforming program using a standard attribute remains
    strictly conforming in the

N2454 is the newest working draft, released November 18, 2019.
N2455 is the same thing with markings for changes relative to C17.

I'm not sure why they didn't just forbid keywords in attribute tokens.
Perhaps there are attributes in existing implementations (as extensions)
that use keywords.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#148114

Am 22.11.19 um 05:41 schrieb Keith Thompson:
> Perhaps there are attributes in existing implementations (as extensions)
> that use keywords.
> 

There are. E.g. const in gcc.

Philipp

[toc] | [prev] | [next] | [standalone]

#148115

On 22/11/2019 05:41, Keith Thompson wrote:
> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> Philipp Klaus Krause <pkk@spth.de> writes:
>>> I guess the main problem is that I am not really familiar with lex /
>>> yacc. The situation with the attribute is a bit different from string
>>> literals though:
>>>
>>> 1) Inside a string literal, everything is just text. The inside of the
>>> attribute still should be tokenized.
>>
>> ... and parsed.
>>
>>> 2) [ is atoken that has many uses in C. We don't really know we're in an
>>> attribute while in lex, that knowledge is only there yacc.
>>
>> It would have been simpler to handle if [[ had been made a new
>> punctuator.  That would not be odd; both + and ++ are distinct tokens.
>> However, it seems an attribute specification starts with two separate [
>> tokens, so if you need to use lex start conditions to alter the lexing
>> inside [[ and ]], you will need to use a yacc action to trigger that.
>> Ask in a yacc/bison group to get the details.
> 
> Two consecutive [ tokens can't appear in a valid C program (prior to the
> presumed introduction of attributes in C20), but two consecutive ]
> tokens can.  Making [[ but not ]] a single token would have been odd.
> Making both [[ and ]] single token without any other changes would break
> expressions like a[b[i]].
> 
> There is precedent in C++ for dealing with nested <> template
> delimiters, but that was needed because >> is already a token.  Any
> problems with [[ and ]] didn't require such a solution.
> 

For a long time (until C++11), you needed to use "> >" to close the
nested template.  In C++11, ">>" was allowed for closing the nested
templates.  That meant that any C++ compiler supporting C++11 had to
find a solution to this problem, which may have meant changing their
parser or tokeniser significantly.  C++ front ends do not use lex/yacc
parsers any more - this is an example of why they don't.

So when attributes were included in C++, the reason there was no
problems in using "]]" to end them despite "]]" being interpreted as "]
]" in other situations, is that C++ compilers already had the facilities
in place to deal with that.


>> But I am not 100% sure you have to do this.  What is your source for
>> keywords being regarded as identifiers?  In the usual C grammar, the
>> rule for "identifier" is very general (the usual letters and digits
>> definitions), but keywords are considered to be a separate category.
>> Thus the grammar item "identifier" really means "things that have the
>> right form but which aren't keywords".
> 
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2455.pdf (C20 draft)
> 6.7.11.1p3:
> 
>     An attribute specifier that contains no attributes has no
>     effect. The order in which attribute tokens appear in an
>     attribute list is not significant. If a keyword (6.4.1) that
>     satisfies the syntactic requirements of an identifier (6.4.2) is
>     contained in an attribute token, it is considered an identifier.
>     A strictly conforming program using a standard attribute remains
>     strictly conforming in the
> 
> N2454 is the newest working draft, released November 18, 2019.
> N2455 is the same thing with markings for changes relative to C17.
> 
> I'm not sure why they didn't just forbid keywords in attribute tokens.
> Perhaps there are attributes in existing implementations (as extensions)
> that use keywords.
> 

[toc] | [prev] | [next] | [standalone]

#148116

Am Freitag, 22. November 2019 09:48:12 UTC+1 schrieb David Brown:
> 
> For a long time (until C++11), you needed to use "> >" to close the
> nested template.  In C++11, ">>" was allowed for closing the nested
> templates.  That meant that any C++ compiler supporting C++11 had to
> find a solution to this problem, which may have meant changing their
> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
> parsers any more - this is an example of why they don't.
> 

What a mess for such a minor thing. Simplicity is a virtue.
I am glad that I didn't have to use C++ in my job.

[toc] | [prev] | [next] | [standalone]

#148117

On 22/11/2019 11:00, A. K. wrote:
> Am Freitag, 22. November 2019 09:48:12 UTC+1 schrieb David Brown:
>>
>> For a long time (until C++11), you needed to use "> >" to close the
>> nested template.  In C++11, ">>" was allowed for closing the nested
>> templates.  That meant that any C++ compiler supporting C++11 had to
>> find a solution to this problem, which may have meant changing their
>> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
>> parsers any more - this is an example of why they don't.
>>
> 
> What a mess for such a minor thing. Simplicity is a virtue.
> I am glad that I didn't have to use C++ in my job.
> 

It /is/ simple, now.  It /was/ a mess, that you had to remember to write
"> >" to close nested templates instead of ">>".

That is to say, is simpler for /users/ - people programming in C++.  A
guiding principle for C and C++ standards is that it is okay to make
life harder for compiler writers if it makes life easier for the
language users.

[toc] | [prev] | [next] | [standalone]

#148150

On Fri, 2019-11-22, A. K. wrote:
> Am Freitag, 22. November 2019 09:48:12 UTC+1 schrieb David Brown:
>> 
>> For a long time (until C++11), you needed to use "> >" to close the
>> nested template.  In C++11, ">>" was allowed for closing the nested
>> templates.  That meant that any C++ compiler supporting C++11 had to
>> find a solution to this problem, which may have meant changing their
>> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
>> parsers any more - this is an example of why they don't.
>> 
>
> What a mess for such a minor thing. Simplicity is a virtue.
> I am glad that I didn't have to use C++ in my job.

You mean "I am glad that I didn't have to write a C++ compiler
in my job."

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#148160

Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:

> On Fri, 2019-11-22, A. K. wrote:

> > What a mess for such a minor thing. Simplicity is a virtue.
> > I am glad that I didn't have to use C++ in my job.
> 
> You mean "I am glad that I didn't have to write a C++ compiler
> in my job."

I am glad I don't have to read other people's convoluted C++ code in my
job. That's the biggest problem - not my code, but everybody else's idea
of "natural" and "legible". There are languages in which this is a
bigger problem than in others. It isn't in halfway decent C, or Prolog;
it very much is in C++ or Perl.

Richard

[toc] | [prev] | [next] | [standalone]

#148120

David Brown <david.brown@hesbynett.no> writes:

> On 22/11/2019 05:41, Keith Thompson wrote:
<cut>
>> There is precedent in C++ for dealing with nested <> template
>> delimiters, but that was needed because >> is already a token.  Any
>> problems with [[ and ]] didn't require such a solution.
>
> For a long time (until C++11), you needed to use "> >" to close the
> nested template.  In C++11, ">>" was allowed for closing the nested
> templates.  That meant that any C++ compiler supporting C++11 had to
> find a solution to this problem, which may have meant changing their
> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
> parsers any more - this is an example of why they don't.
>
> So when attributes were included in C++, the reason there was no
> problems in using "]]" to end them despite "]]" being interpreted as "]
> ]" in other situations, is that C++ compilers already had the facilities
> in place to deal with that.

Oh!  Are [[ and/or ]] single tokens in C++'s attributes?  If so, what a
shame there is now another difference with C.  If not, I don't see the
connection with >> since neither [[ nor ]] are single tokens. There is,
in C, no need for any special mechanism at all.

Attributes in C just have the grammar (very roughly):

 '[' '[' <stuff with balanced parens (of all sorts) ']' ']'

C++ could have done it that way too, without invoking any special
treatment for ]].

The only oddity is that (in C) you'd need to either switch the lexer
into a special mode, or introduce a grammar rule for an "attribute
identifier" that can be either a keyword or an ordinary identifier.

>>> But I am not 100% sure you have to do this.  What is your source for
>>> keywords being regarded as identifiers?  In the usual C grammar, the
>>> rule for "identifier" is very general (the usual letters and digits
>>> definitions), but keywords are considered to be a separate category.
>>> Thus the grammar item "identifier" really means "things that have the
>>> right form but which aren't keywords".
>> 
>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2455.pdf (C20 draft)
>> 6.7.11.1p3:
>> 
>>     An attribute specifier that contains no attributes has no
>>     effect. The order in which attribute tokens appear in an
>>     attribute list is not significant. If a keyword (6.4.1) that
>>     satisfies the syntactic requirements of an identifier (6.4.2) is
>>     contained in an attribute token, it is considered an identifier.
>>     A strictly conforming program using a standard attribute remains
>>     strictly conforming in the

Keith:
Thanks for the quote.  I'd not read the standard but I'd hoped they were
just using "identifier" as it was used elsewhere in the C grammar.
Obviously not.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#148121

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> David Brown <david.brown@hesbynett.no> writes:
>> On 22/11/2019 05:41, Keith Thompson wrote:
> <cut>
>>> There is precedent in C++ for dealing with nested <> template
>>> delimiters, but that was needed because >> is already a token.  Any
>>> problems with [[ and ]] didn't require such a solution.
>>
>> For a long time (until C++11), you needed to use "> >" to close the
>> nested template.  In C++11, ">>" was allowed for closing the nested
>> templates.  That meant that any C++ compiler supporting C++11 had to
>> find a solution to this problem, which may have meant changing their
>> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
>> parsers any more - this is an example of why they don't.
>>
>> So when attributes were included in C++, the reason there was no
>> problems in using "]]" to end them despite "]]" being interpreted as "]
>> ]" in other situations, is that C++ compilers already had the facilities
>> in place to deal with that.
>
> Oh!  Are [[ and/or ]] single tokens in C++'s attributes?  If so, what a
> shame there is now another difference with C.  If not, I don't see the
> connection with >> since neither [[ nor ]] are single tokens. There is,
> in C, no need for any special mechanism at all.

No, the C++ attribute syntax for an attribute-specifier is:

    attribute-specifier:
        [ [ attribute-using-prefix opt attribute-list ] ]
        alignment-specifier

C++ has a special case for treating >> as two > tokens in the context of
templates, so that you can write
    std::vector<std::vector<int>>
rather than
    std::vector<std::vector<int> >
(which was necessary in older versions of C++, before C++11 I think).

C and C++ attribute syntaxes appear to be reasonably consistent.

You suggested that treating [[ as a single token would make
parsing attributes easier, but I don't think it's necessary (or
even helpful as far as I can tell) -- and there's no particular
need to treat ]] specially (as >> is treated specially in C++).
If there's some kind of feedback from the parser to tell the lexer
to start treating keywords as ordinary identifiers, it should be
as easy to do that after seeing two consecutive [ tokens as after
seeing a hypothetical single [[ token.

Or the parser could use a syntax with an added production like:
    identifier-or-keyword:
        identifier  
        auto
        break
        case
        ...
        _Thread_local
and use it in the production for attribute-token.

> Attributes in C just have the grammar (very roughly):
>
>  '[' '[' <stuff with balanced parens (of all sorts) ']' ']'
>
> C++ could have done it that way too, without invoking any special
> treatment for ]].

It does.

> The only oddity is that (in C) you'd need to either switch the lexer
> into a special mode, or introduce a grammar rule for an "attribute
> identifier" that can be either a keyword or an ordinary identifier.

Note that a parser for C-with-attributes (C20) has to accept *any*
keywords within attribute tokens, not just the ones it uses, since "Any
attribute token that is not supported by the implementation is ignored".
A compiler can't reject
    [[ auto::register ]]
as a syntax error.  It has to recognize it as a syntactically valid
attribute specifier so it can ignore it.  (Presumably it can issue a
warning.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#148129

Keith Thompson <kst-u@mib.org> writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> David Brown <david.brown@hesbynett.no> writes:
>>> On 22/11/2019 05:41, Keith Thompson wrote:
>> <cut>
>>>> There is precedent in C++ for dealing with nested <> template
>>>> delimiters, but that was needed because >> is already a token.  Any
>>>> problems with [[ and ]] didn't require such a solution.
>>>
>>> For a long time (until C++11), you needed to use "> >" to close the
>>> nested template.  In C++11, ">>" was allowed for closing the nested
>>> templates.  That meant that any C++ compiler supporting C++11 had to
>>> find a solution to this problem, which may have meant changing their
>>> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
>>> parsers any more - this is an example of why they don't.
>>>
>>> So when attributes were included in C++, the reason there was no
>>> problems in using "]]" to end them despite "]]" being interpreted as "]
>>> ]" in other situations, is that C++ compilers already had the facilities
>>> in place to deal with that.
>>
>> Oh!  Are [[ and/or ]] single tokens in C++'s attributes?  If so, what a
>> shame there is now another difference with C.  If not, I don't see the
>> connection with >> since neither [[ nor ]] are single tokens. There is,
>> in C, no need for any special mechanism at all.
>
> No, the C++ attribute syntax for an attribute-specifier is:
>
>     attribute-specifier:
>         [ [ attribute-using-prefix opt attribute-list ] ]
>         alignment-specifier
>
> C++ has a special case for treating >> as two > tokens in the context of
> templates, so that you can write
>     std::vector<std::vector<int>>
> rather than
>     std::vector<std::vector<int> >
> (which was necessary in older versions of C++, before C++11 I think).
>
> C and C++ attribute syntaxes appear to be reasonably consistent.

Yes, my question was somewhat rhetorical.  I thought it vanishing
unlikely that C and C++ diverged on this issue.  Hence my being puzzled
by DB's suggestion that ]] would not be a problem (in C++) because of
"the facilities in place" to deal with >>.

> You suggested that treating [[ as a single token would make
> parsing attributes easier,

I did not intend to.  I wanted to say that it would simplify using a
tool like lex (so that any special lexical considerations inside
attributes could be self-contained) but I think the remark came out as
too general.

<cut>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.c

csiph-web