Re: Please provide a learning path for mastering lexical analysis languages

Path	csiph.com!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From	Paul B Mann <parser.generator.guy@gmail.com>
Newsgroups	comp.compilers
Subject	Re: Please provide a learning path for mastering lexical analysis languages
Date	Sun, 8 May 2022 22:27:55 -0700 (PDT)
Organization	Compilers Central
Lines	102
Sender	news@iecc.com
Approved	comp.compilers@iecc.com
Message-ID	<22-05-027@comp.compilers> (permalink)
References	<22-05-010@comp.compilers> <22-05-023@comp.compilers>
Mime-Version	1.0
Content-Type	text/plain; charset="UTF-8"
Injection-Info	gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970"; logging-data="66705"; mail-complaints-to="abuse@iecc.com"
Keywords	lex
Posted-Date	13 May 2022 13:03:24 EDT
X-submission-address	compilers@iecc.com
X-moderator-address	compilers-request@iecc.com
X-FAQ-and-archives	http://compilers.iecc.com
In-Reply-To	<22-05-023@comp.compilers>
Xref	csiph.com comp.compilers:3004

Show key headers only | View raw

/* Token Rules */

	<eof>			-> \z

	<constant>     	-> literal
					-> integer
					-> decimal
					-> real

	<identifier>   		-> letter (letter|digit)*

	integer        		-> digit+

	real	         		-> integer exp
					-> decimal exp

	decimal        		-> digit+ '.'
					->        '.' digit+
					-> digit+ '.' digit+

	exp	         		-> 'e' digit+
					-> 'E' digit+
					-> 'e' '-' digit+
					-> 'E' '-' digit+
					-> 'e' '+' digit+
					-> 'E' '+' digit+

	literal        		-> ''' lchar '''

	lchar          		-> lany
					-> '\' '\'
					-> '\' '''
					-> '\' '"'
					-> '\' 'n'
					-> '\' 't'
					-> '\' 'a'
					-> '\' 'b'
					-> '\' 'f'
					-> '\' 'r'
					-> '\' 'v'
					-> '\' '0'

	<string>       		-> '"' schar* '"'

	schar          		-> sany
					-> '\' '\'
					-> '\' '''
					-> '\' '"'
					-> '\' 'n'
					-> '\' 't'
					-> '\' 'a'
					-> '\' 'b'
					-> '\' 'f'
					-> '\' 'r'
					-> '\' 'v'
					-> '\' '0'

	{whitespace}       	-> whitechar+

	{commentline}  	-> '/' '/' neol*

	{commentblock} 	-> '/' '*' na* '*'+ (nans na* '*'+)* '/'

/* Character Sets */

	any				= 0..255 - \z
	lany           		= any - ''' - '\' - \n
	sany	         	= any - '"' - '\' - \n

	letter	      		= 'a'..'z' | 'A'..'Z' | '_'
	digit	         	= '0'..'9'

	whitechar     		= \t | \n | \r | \f | \v | ' '

	na             		= any - '*' 			// not asterisk
	nans           		= any - '*' - '/'	                // not asterisk not slash
	neol           		= any - \n 			// not end of line

	\t				=  9					// tab
	\n				= 10				// newline
	\v				= 11				// vertical feed?
	\f				= 12				// form feed
	\r				= 13				// return
	\z				= 26				// end of file
	\b				= 32				// blank/space

/* End */

The above lexical rules define C-language symbols.
It's just a lexical grammar, not too hard to figure out.
This is input to the DFA lexer generator, which is provided
with the LRSTAR parser generator on SourceForge.net.

DFA creates lexers that run 80% faster than "flex" lexers
and are about the same size.

If you need more language power to define a lexer ...
that's what parser are for.

BTW, LRSTAR creates parsers in C++ than were running
140 times faster than those created by ANTLR, using the
C++ target, the last time I did a comparison, 2 years ago.

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Please provide a learning path for mastering lexical analysis languages Roger L Costello <costello@mitre.org> - 2022-05-06 11:59 +0000
  Re: Please provide a learning path for mastering lexical analysis languages George Neuner <gneuner2@comcast.net> - 2022-05-08 12:52 -0400
    Re: Please provide a learning path for mastering lexical analysis languages Paul B Mann <parser.generator.guy@gmail.com> - 2022-05-08 22:27 -0700
    Re: Please provide a learning path for mastering lexical analysis languages Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-05-22 12:12 +0300
  Re: Please provide a learning path for mastering lexical analysis languages luser droog <luser.droog@gmail.com> - 2022-05-08 18:08 -0700
  Re: Please provide a learning path for mastering lexical analysis languages gah4 <gah4@u.washington.edu> - 2022-05-13 13:42 -0700

csiph-web