Path: csiph.com!tncsrv06.tnetconsulting.net!news.snarked.org!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end From: "Ev. Drikos" Newsgroups: comp.compilers Subject: Re: Languages with optional spaces Date: Sun, 23 Feb 2020 12:33:36 +0200 Organization: Aioe.org NNTP Server Lines: 18 Sender: news@iecc.com Approved: comp.compilers@iecc.com Message-ID: <20-02-017@comp.compilers> References: <20-02-015@comp.compilers> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="60628"; mail-complaints-to="abuse@iecc.com" Keywords: lex, comment Posted-Date: 23 Feb 2020 21:15:38 EST X-submission-address: compilers@iecc.com X-moderator-address: compilers-request@iecc.com X-FAQ-and-archives: http://compilers.iecc.com Content-Language: en-US Xref: csiph.com comp.compilers:2455 On 19/02/2020 17:35, Maury Markowitz wrote: > ... Likewise, one might modify the variable name > pattern, but I'm not sure how one says "everything that doesn't start with one > of these other 110 patterns". > This should be relatively simple with a scanner generator that supports intersection and negation operators (or difference operators), but IMHO such a rule would work ie if FORI wasn't a valid variable name. Still I can't propose to you a specific tool though. With a tool like flex, I'd try to see if different start states do work: https://www.cs.virginia.edu/~cr4bd/flex-manual/Start-Conditions.html Ev. Drikos [You can try start states but in my experience if the tokenizing rules are very context sensitive, it's easier to give up and hand-code the lexer. The lexical syntax of Basic isn't that big. -John]