Path: csiph.com!weretis.net!feeder6.news.weretis.net!feeder.usenetexpress.com!feeder-in1.iad1.usenetexpress.com!border1.nntp.dca1.giganews.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: Ulya Trofimovich <skvadrik@gmail.com>
Newsgroups: comp.compilers
Subject: Re: re2c-1.0 released!
Date: Sun, 3 Sep 2017 17:58:33 +0100
Organization: Compilers Central
Lines: 54
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <17-09-009@comp.compilers>
References: <17-08-007@comp.compilers> <17-09-001@comp.compilers> <17-09-003@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="54656"; mail-complaints-to="abuse@iecc.com"
Keywords: lex
Posted-Date: 03 Sep 2017 13:43:46 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
Content-Language: en-US
Xref: csiph.com comp.compilers:1914

> In fact we can do it all in standard lex.
> [...]
> Pseudo-code, including a mechanism for returning to the INITIAL
> state without parser involvement:
> [...]

The thing is, the new algorithm is both more *efficient* and *elegant*.
Consider how parsing date in format 'YYYY-MM-DD' would look like in re2c:


@y [0-9]{4} [-] @m [0-9]{2} [-] @d [0-9]{2} {
    unsigned year
        = (y[0] - '0') * 1000
        + (y[1] - '0') * 100
        + (y[2] - '0') * 10
        + (y[3] - '0');
    unsigned month
        = (m[0] - '0') * 10
        + (m[1] - '0');
    unsigned day
        = (m[0] - '0') * 10
        + (m[1] - '0');
}


Here re2c will optimize out all dynamic variables, because @d is always
2 characters behind the end of input, and @m is 3 characters behind @d,
and @y is 5 characters behind @m. So all three variables will be
calculated using static offsets from cursor when it points to the end of
input.


If comments and whitespace are allowed between date components, we could
change the above example like this (now re2c will use variables to track
submatch positions, since components have variable length):


com = "/*" ([^*] | ("*" [^/]))* "*""/";
wsp = [ \t\n\r]* | com;
hyp = wsp [-] wsp;

@y [0-9]{4} hyp @m [0-9]{2} hyp @d [0-9]{2} {
    unsigned year
        = (y[0] - '0') * 1000
        + (y[1] - '0') * 100
        + (y[2] - '0') * 10
        + (y[3] - '0');
    unsigned month
        = (m[0] - '0') * 10
        + (m[1] - '0');
    unsigned day
        = (m[0] - '0') * 10
        + (m[1] - '0');
}