Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #3060

The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them?

Show all headers | View raw

Hi Folks,

Page 84-85 of the dragon book [1] says:

There are several reasons for separating the analysis phase of compiling into
lexical analysis and parsing.

1. Simpler design is perhaps the most important consideration. The separation
of lexical analysis from syntax analysis often allows us to simplify one or
the other of these phases. For example, a parser embodying the conventions for
comments and white space is significantly more complex than one that can
assume comments and white space have already been removed by a lexical
analyzer. It we are designing a new language, separating the lexical and
syntactic conventions can lead to a cleaner overall language design.

2. Compiler efficiency is improved. A separate lexical analyzer allows us to
construct a specialized and potentially more efficient processor for the task.
A large amount of time is spent reading the source program and partitioning it
into tokens. Specialized buffering techniques for reading input characters and
processing tokens can significantly speed up the performance of a compiler.

3. Compiler portability is enhanced. Input alphabet peculiarities and other
device-specific anomalies can be restricted to the lexical analyzer. The
representation of special or non-standard symbols, such as the up-arrow in
Pascal, can be isolated in the lexical analyzer.

Specialized tools have been designed to help automate the construction of
lexical analyzers and parsers when they are separated.
----------------------------
Those seem like compelling reasons for separating the lexical analysis from
parsing, so why does ANTLR not do so; i.e., why does ANTLR combine them?

/Roger

[1] Compilers: Principles, Techniques, and Tools by Aho, Sethi, and Ullman.

Back to comp.compilers | Previous | Next — Next in thread | Find similar

Thread

The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Roger L Costello <costello@mitre.org> - 2022-06-09 14:52 +0000
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? "Alexei A. Frounze" <alexfrunews@gmail.com> - 2022-06-09 18:07 -0700
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? gah4 <gah4@u.washington.edu> - 2022-06-09 23:01 -0700
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2022-06-10 12:26 +0200
  The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-06-11 23:45 +0300
  Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? George Neuner <gneuner2@comcast.net> - 2022-06-11 18:15 -0400
    Re: The dragon book says separating lexical analysis and parsing is beneficial, so why doesn't ANTLR separate them? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2022-06-12 14:10 +0000

csiph-web