Path: csiph.com!goblin3!goblin.stu.neva.ru!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: anton@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.compilers
Subject: Re: re2c-1.0 released!
Date: Sat, 02 Sep 2017 17:02:40 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 40
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <17-09-003@comp.compilers>
References: <17-08-007@comp.compilers> <17-09-001@comp.compilers>
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="33930"; mail-complaints-to="abuse@iecc.com"
Keywords: lex
Posted-Date: 02 Sep 2017 23:50:13 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
Xref: csiph.com comp.compilers:1908

Kaz Kylheku <398-816-0742@kylheku.com> writes:
>Briefly, why would you do some hacky regex thing in lex with \1, \2,
>\3, when in the level immediately above yylex() you have proper phrase
>recognition, with $1, $2, $3.

These don't do the same thing.  Taking your example from the other
posting, do you want to recognize

2017
-08
/* bla bla */ -
28

as date?  If so, do it at the parser level, if not, the scanner.

I dimly remember a case of a lexeme type with internal structure that
was reflected in the regexp, where I then had to write C code manually
repeating some of the work that the scanner had already done, and
wishing for a feature like that provided by the OP.  Unfortunately,
and I do not remember details, that has been long ago.

>> It may seem that
>> only a lack of effort prevents developers of lexer generators like Flex
>> from implementing it (as well as fixing the ever-broken trailing
>> contexts [3]).
>
>Lack of justification due to, I suspect, lack of demand from
>sophisticated users of the tools, who use them for their intended
>purpose.

Or maybe just accepting the limitations of the tool and working around
them, without sending a feature request to the maintainer (I didn't,
at the time).

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/