Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #3072

Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them?

Show all headers | View raw

George Neuner <gneuner2@comcast.net> writes:
>Note that Yacc and Bison also recognize textual constants in the parser
>grammar and generate a token id for the (seperately specified) lexer
>to return.

If you have a rule in yacc/bison:

E: T '+' T

the "token id" for '+' is the ASCII-code of +.  Bison generates token
ids only for tokens defined with %token.  So if you instead write

E: T PLUS T

you have to define

%token PLUS

and the value of PLUS is communicated to the scanner through the
.tab.h file.

Also note that for the last version of yacc that I have seen
documentation for, if you have a rule

S: L ":=" E

there is no token for ":=", but instead what you get is equivalent to

S: L ':' '=' E

Bison is more capable, you can, e.g., define

%token BECOMES ":="

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

Back to comp.compilers | Previous | Next — Previous in thread | Find similar

Thread

The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Roger L Costello <costello@mitre.org> - 2022-06-09 14:52 +0000
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? "Alexei A. Frounze" <alexfrunews@gmail.com> - 2022-06-09 18:07 -0700
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? gah4 <gah4@u.washington.edu> - 2022-06-09 23:01 -0700
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2022-06-10 12:26 +0200
  The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-06-11 23:45 +0300
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? George Neuner <gneuner2@comcast.net> - 2022-06-11 18:15 -0400
    Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2022-06-12 14:10 +0000

csiph-web