Re: Applesoft tokenization phases?

Path	csiph.com!xmission!news.snarked.org!border2.nntp.dca1.giganews.com!nntp.giganews.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From	Martin Ward <martin@gkc.org.uk>
Newsgroups	comp.compilers
Subject	Re: Applesoft tokenization phases?
Date	Sat, 21 Mar 2020 19:42:17 +0000
Organization	Compilers Central
Lines	32
Sender	news@iecc.com
Approved	comp.compilers@iecc.com
Message-ID	<20-03-024@comp.compilers> (permalink)
References	<20-03-013@comp.compilers> <20-03-016@comp.compilers> <20-03-017@comp.compilers> <20-03-022@comp.compilers>
Mime-Version	1.0
Content-Type	text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding	8bit
Injection-Info	gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="66018"; mail-complaints-to="abuse@iecc.com"
Keywords	Basic, lex, history
Posted-Date	21 Mar 2020 16:50:24 EDT
X-submission-address	compilers@iecc.com
X-moderator-address	compilers-request@iecc.com
X-FAQ-and-archives	http://compilers.iecc.com
Xref	csiph.com comp.compilers:2495

Show key headers only | View raw

On 17/03/20 22:14, Ev. Drikos wrote:
> Another vague point or simply a point where I'm not really sure that I
> translate properly the manual are the reserved keywords before a certain
> delimiter. Likely an Applesoft parser must reject this valid UK101 code:
>
> 10 X=SHIMEM:
> 20 END

For those less familiar with UK101 BASIC: this code is only
valid because "HIMEM" is not a keyword in UK101 BASIC.
UK101 does not allow keywords as part of a variable.

> There is a reason, BASIC compilers that remove spaces often restrict
> variables (like the original Basic version did) to a letter optionally
> followed by a digit (and a dollar sign to indicate strings).  It makes
> the lexing much simpler, even without spaces.

Lexing is not the issue: the real reason for the restriction
is to make the symbol table much simpler. Each variable in the table
takes up a fixed amount of space. For the UK101, only the first two
characters of a variable are significant, so only two characters
are needed in the symbol table. "Lexing" a statement just consists
of replacing all keywords outside strings by special characters.
When the interpreted parses it, each statement either starts
with a keyword or is an assignment and starts with a variable.

--
			Martin

Dr Martin Ward | Email: martin@gkc.org.uk | http://www.gkc.org.uk
G.K.Chesterton site: http://www.gkc.org.uk/gkc | Erdos number: 4

Back to comp.compilers | Previous | Next — Previous in thread | Find similar

Thread

Applesoft tokenization phases? "Ev. Drikos" <drikosev@gmail.com> - 2020-03-12 17:46 +0200
  Re: Applesoft tokenization phases? George Neuner <gneuner2@comcast.net> - 2020-03-13 17:55 -0400
  Re: Applesoft tokenization phases? awanderin <awanderin@gmail.com> - 2020-03-16 00:07 -0600
    Re: Applesoft tokenization phases? "Ev. Drikos" <drikosev@gmail.com> - 2020-03-18 00:14 +0200
      Re: Applesoft tokenization phases? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2020-03-20 07:06 -0400
        Re: Applesoft tokenization phases? Martin Ward <martin@gkc.org.uk> - 2020-03-21 19:42 +0000

csiph-web