Path: csiph.com!xmission!news.snarked.org!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: "Ev. Drikos" Newsgroups: comp.compilers Subject: Re: Languages with optional spaces Date: Sat, 29 Feb 2020 21:38:14 +0200 Organization: Aioe.org NNTP Server Lines: 28 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <20-02-034@comp.compilers> References: <20-02-015@comp.compilers> <20-02-017@comp.compilers> <20-02-033@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="86678"; mail-complaints-to="abuse@iecc.com" Keywords: lex, Basic Posted-Date: 29 Feb 2020 15:11:30 EST X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Content-Language: en-US Xref: csiph.com comp.compilers:2471 On 29/02/2020 11:48, Christopher F Clark wrote: > "Ev. Drikos" posted an interesting albeit partial solution > to the problem of keywords being part of identifiers in languages with > optional spaces. It's only an example for the beginning of statements, based on the description I read in other postings. It couldn't be the solution of an unspecified problem. The terms "usually" & "in theory" aren't specs. In contrast, one who wants to code a parser for the particular dialect of BASIC should know ie what the 6502 or the Compukit UK101 code did. > The problem is that some keywords can appear at places other than the > beginning of an identifier. > In fact, in the worst case scenario, the language can be ambiguous. >... > > With a GLR parser (or something equivalent in power, e.g. an Earley > parser or CYK) and a lexer that returns all possible sets of > tokenizations one can find all the relevant parse trees and then see > if only 1 makes semantic sense. Obviously, those who coded such BASIC parsers had some simpler rules, ie the position of the first 'TO' might be used for the statement 50. Such ruling may comply with the description given by Dr Martin Ward. Ev. Drikos