Path: csiph.com!eternal-september.org!feeder.eternal-september.org!reader01.eternal-september.org!.POSTED!not-for-mail
From: Keith Thompson <kst-u@mib.org>
Newsgroups: comp.lang.c
Subject: Re: How to parse attributes?
Date: Fri, 22 Nov 2019 10:53:51 -0800
Organization: None to speak of
Lines: 85
Message-ID: <lnftifyc74.fsf@kst-u.example.com>
References: <qr2u7d$lo5$1@solani.org> <cded955b-2b9f-456b-84bc-4f2fb97aad9f@googlegroups.com> <qr5m0u$f6g$1@solani.org> <87blt47is6.fsf@bsb.me.uk> <lnwobsy12n.fsf@kst-u.example.com> <qr87c4$tah$1@dont-email.me> <87tv6v6by3.fsf@bsb.me.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Injection-Info: reader02.eternal-september.org; posting-host="673dee156045fd04bdb03a9da416f64d"; logging-data="17663"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/4082uN8YESryfirnzfDis"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux)
Cancel-Lock: sha1:B/nREMI9Vq1D+71Ej3GOLdJkUqg= sha1:Ub1XbkXfjWYLwzTjFr+yl784qkg=
Xref: csiph.com comp.lang.c:148121

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> David Brown <david.brown@hesbynett.no> writes:
>> On 22/11/2019 05:41, Keith Thompson wrote:
> <cut>
>>> There is precedent in C++ for dealing with nested <> template
>>> delimiters, but that was needed because >> is already a token.  Any
>>> problems with [[ and ]] didn't require such a solution.
>>
>> For a long time (until C++11), you needed to use "> >" to close the
>> nested template.  In C++11, ">>" was allowed for closing the nested
>> templates.  That meant that any C++ compiler supporting C++11 had to
>> find a solution to this problem, which may have meant changing their
>> parser or tokeniser significantly.  C++ front ends do not use lex/yacc
>> parsers any more - this is an example of why they don't.
>>
>> So when attributes were included in C++, the reason there was no
>> problems in using "]]" to end them despite "]]" being interpreted as "]
>> ]" in other situations, is that C++ compilers already had the facilities
>> in place to deal with that.
>
> Oh!  Are [[ and/or ]] single tokens in C++'s attributes?  If so, what a
> shame there is now another difference with C.  If not, I don't see the
> connection with >> since neither [[ nor ]] are single tokens. There is,
> in C, no need for any special mechanism at all.

No, the C++ attribute syntax for an attribute-specifier is:

    attribute-specifier:
        [ [ attribute-using-prefix opt attribute-list ] ]
        alignment-specifier

C++ has a special case for treating >> as two > tokens in the context of
templates, so that you can write
    std::vector<std::vector<int>>
rather than
    std::vector<std::vector<int> >
(which was necessary in older versions of C++, before C++11 I think).

C and C++ attribute syntaxes appear to be reasonably consistent.

You suggested that treating [[ as a single token would make
parsing attributes easier, but I don't think it's necessary (or
even helpful as far as I can tell) -- and there's no particular
need to treat ]] specially (as >> is treated specially in C++).
If there's some kind of feedback from the parser to tell the lexer
to start treating keywords as ordinary identifiers, it should be
as easy to do that after seeing two consecutive [ tokens as after
seeing a hypothetical single [[ token.

Or the parser could use a syntax with an added production like:
    identifier-or-keyword:
        identifier  
        auto
        break
        case
        ...
        _Thread_local
and use it in the production for attribute-token.

> Attributes in C just have the grammar (very roughly):
>
>  '[' '[' <stuff with balanced parens (of all sorts) ']' ']'
>
> C++ could have done it that way too, without invoking any special
> treatment for ]].

It does.

> The only oddity is that (in C) you'd need to either switch the lexer
> into a special mode, or introduce a grammar rule for an "attribute
> identifier" that can be either a keyword or an ordinary identifier.

Note that a parser for C-with-attributes (C20) has to accept *any*
keywords within attribute tokens, not just the ones it uses, since "Any
attribute token that is not supported by the implementation is ignored".
A compiler can't reject
    [[ auto::register ]]
as a syntax error.  It has to recognize it as a syntactically valid
attribute specifier so it can ignore it.  (Presumably it can issue a
warning.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */