Path: csiph.com!xmission!news.snarked.org!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: "Ev. Drikos" Newsgroups: comp.compilers Subject: Applesoft tokenization phases? Date: Thu, 12 Mar 2020 17:46:00 +0200 Organization: Aioe.org NNTP Server Lines: 39 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <20-03-013@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="54884"; mail-complaints-to="abuse@iecc.com" Keywords: lex, Basic Posted-Date: 13 Mar 2020 08:26:23 EDT X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Content-Language: en-US Xref: csiph.com comp.compilers:2484 Hello, This question relates to thread "Languages with Optional Spaces". In an Applesoft II manual I've found at "classiccmp.org" [1], page 7, we read that in a variable name any alphanumeric characters after the first two are ignored, unless they contain a reserved world. FEND ie would be illegal as it contains END. To implement such a rule one could first recognize keywords and then recognize any names. We see in p123 that stmt I is tokenized as II: I. stmt: 100 FOR A = LOFT OR CAT To 15 II. tokens: 100 FOR A = LOF TO RC AT To 15 Yet, I've found ie a program at "hoist-point.com" [2] that contains: 110 DIFF = ABS(A(I)-N) Also, an online AppleSoft simulator at calormen.com [3] accepts ie both DIFF and FEND as valid variable names. As it seems, this issue can affect a design choice for the tokenization phases of an Applesoft front-end. Is the manual just informative or the online simulator does not accept (precisely) the particular dialect? Ev. Drikos PS: The eight (8) Basic examples in the Compukit UK101 Simulator project seem to be compatible to Applesoft, although I haven't seen this term in the site [4] or the computer manual that contains a Basic Reference; not sure also if the Superboard II accepts a dialect compatible to Applesoft. ------------------------------------------------------------------------ [1] http://www.classiccmp.org/cini/pdf/Apple/ [2] http://www.hoist-point.com/applesoft_basic_tutorial.htm [3] https://www.calormen.com/jsbasic/ [4] http://uk101.sourceforge.net/docs/index.html