Path: csiph.com!xmission!news.snarked.org!border2.nntp.dca1.giganews.com!border1.nntp.dca1.giganews.com!nntp.giganews.com!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: Hans-Peter Diettrich Newsgroups: comp.compilers Subject: Re: Languages with optional spaces and tools Date: Fri, 28 Feb 2020 20:16:56 +0100 Organization: Compilers Central Lines: 43 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <20-02-032@comp.compilers> References: <20-02-015@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="90702"; mail-complaints-to="abuse@iecc.com" Keywords: Basic, lex Posted-Date: 28 Feb 2020 21:03:08 EST X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Xref: csiph.com comp.compilers:2469 Am 19.02.2020 um 16:35 schrieb Maury Markowitz: After reading the entire tree, I feel a need for adding my $0.02: > I'm trying to write a lex/yacc (flex/bison) interpreter for classic BASICs > like the original DEC/MS, HP/DG etc. If all you have is a hammer, everything looks like a nail :-( > 100 FORI = 1 TO 10 > > The problem is that FORI part. Some BASICs allow variable names with more than > two characters, so in theory, FORI could be a variable. These BASICs outlaw > that in their parsers; any string that starts with a keyword exits then, so > this would always parse as FOR. In lex, FORI is longer than FOR, so it returns > a variable token called FORI. Your problem may be the use of lex/yacc! Some older languages are not so regular and context free and have their own rules for disambiguation. Such old lexers also can be non-greedy or follow other non-lex rules. So put aside your hammer and try other tools or your mere fingers... > [Having written Fortran parsers, not that I've ever found. I did a prepass > over each statement to figure out whether it was an assignment or something > else, then the lexing was straightforward if not pretty. -John] Right, the FORI problem was observed first with FORTRAN, and also was solved at that time. When I learned computer science, in the early 70's, the use of parser generators was almost restricted to the scientific area, for their own continued development, not for use in compiler construction. At least the development of a correct formal grammar for those old languages often; if possible at all; took more time than handcrafting a compiler front-end. Also most languages came with a rather informal description of their terminals, with a more or less complete formal grammar restricted to their parser. DoDi