Path: csiph.com!4.us.feeder.erje.net!feeder.erje.net!feeder.usenetexpress.com!feeder-in1.iad1.usenetexpress.com!border1.nntp.dca1.giganews.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: Hans-Peter Diettrich Newsgroups: comp.compilers Subject: Re: Regular expressions in lexing and parsing Date: Tue, 21 May 2019 07:14:14 +0200 Organization: Compilers Central Lines: 26 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <19-05-095@comp.compilers> References: <19-05-092@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="92946"; mail-complaints-to="abuse@iecc.com" Keywords: lex Posted-Date: 21 May 2019 22:46:33 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Xref: csiph.com comp.compilers:2326 Am 17.05.2019 um 15:18 schrieb Ed Davis: > So don't write lexers and parsers with regular expressions as the starting > point. Your code will be faster, cleaner, and much easier to understand and to > maintain. I think that regular expressions are primarily usable in *formal* token grammars. In contrast to parsers I'm missing *complete* formal grammars for the tokens, including whitespace, comments, strings and other special language elements, and how these fit together. Here regular expressions are easier to understand and maintain than DFA or NFA code for all those token types. The equivalent lexer *implementation* finally is up to the compiler writer. OTOH I wonder how a formal (regex?) FORTRAN lexer grammar could look like? DoDi [I've written Fortran 77 parsers. It was quite context sensitive, particularly deciding whether a statement was an assignment or something else. e.g. FORMAT(I4) = 42 DO 10 I = 1.23 (oops) Modern Fortran no longer ignores spaces which makes lexing a lot easier. -John]