Path: csiph.com!xmission!news.snarked.org!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: Christopher F Clark Newsgroups: comp.compilers Subject: Re: Applesoft tokenization phases? Date: Fri, 20 Mar 2020 07:06:55 -0400 Organization: Compilers Central Lines: 31 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <20-03-022@comp.compilers> References: <20-03-013@comp.compilers> <20-03-016@comp.compilers> <20-03-017@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="86364"; mail-complaints-to="abuse@iecc.com" Keywords: Basic, history, lex Posted-Date: 20 Mar 2020 11:46:44 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Xref: csiph.com comp.compilers:2493 Jerry Awanderin wrote: > > Also, an online AppleSoft simulator at calormen.com [3] accepts ie both > > DIFF and FEND as valid variable names. > > It is doing things differently than actual Applesoft. > > > As it seems, this issue can affect a design choice for the tokenization > > phases of an Applesoft front-end. Is the manual just informative or the > > online simulator does not accept (precisely) the particular dialect? > > The latter; the simulator accepts a different dialect. For situations like this, we added classes (and inheritance) to Yacc++ lexers and parsers, so that one could define a common subset and then extend it to cover the cases that are unique or different dialects. However, one can do similar things with lexer states, flag/switch variables, or a host of other techniques. You just have to decide how much mess you are willing to handle and how important supporting the variations are. There is a reason, BASIC compilers that remove spaces often restrict variables (like the original Basic version did) to a letter optionally followed by a digit (and a dollar sign to indicate strings). It makes the lexing much simpler, even without spaces. -- ****************************************************************************** Chris Clark email: christopher.f.clark@compiler-resources.com Compiler Resources, Inc. Web Site: http://world.std.com/~compres 23 Bailey Rd voice: (508) 435-5016 Berlin, MA 01503 USA twitter: @intel_chris