Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #609

Re: Recursive descent parsing and optimization, was Good practical language and OS agnostic text?

Show key headers only | View raw

"BGB" <cr88192@hotmail.com> wrote in message
> On 4/21/2012 2:22 AM, Uli Kusterer wrote:

>> - Recursive descent parsers: It's the obvious way to write a parser.

> although I use recursive descent, the above sounds different from what I
> usually do.

> ReadStatement:
>   checks for and handles vaious statement types
>   calls ReadExpression
>
> ReadExpression:
>   (actually, this is a tower of functions, one for each precedence
> level, working from lowest to highest precedence)

Sounds like it's influenced by the C grammar, which defines expressions
using something like 13 or 17 layers of precedence.

Beyond about 3-4 levels, I found that unmanageable. For expression syntax, I
don't use any precedence in the grammar at all; I have precedence as an
attribute of an operator, and an expression can be parsed with a single
function.

Or rather two: readfactor(priority), and readterm(). Readfactor() deals with
the  binary operators linking successive terms, while readterm() does all
the real work (since my syntax doesn't distinguish between expressions and
statements, that's quite a big workload).

>> - Tokenizing: Essentially grab all the words in your source text and
>> build an array with an entry for each so you can more quickly walk
>> forward and backward without having to scan individual characters.

> partly agreed, except that it isn't really strictly necessary to build
> an array up-front.

> most of my parsers don't bother using an array, but instead just call
> the tokenizer function directly to peek and parse tokens based on the
> current "stream position" (generally a raw char pointer in my language
> parsers).

I've tried reading all the tokens in a separate pass, but didn't really like
it. And it takes a lot more storage as well, especially with macro
expansions.

Instead I read them as I go along, but with provision for a one-symbol
look-ahead.

>> - Syntax tree: Build a tree structure that represents the parsed program.
>> You

> typically, things like simplifying expressions, propagating constants,
> ... is done in a stage I have typically called "reduction", which
> happens between parsing and prior to producing (bytecode) output.

I use the following passes (which seem to be fairly typical):

Syntax analysis (lexing and parsing)
Name resolution (a recent introduction for me)
Type analysis (static type checks and coercions, constant folding)
Code generation (to intermediate code or to byte-code)
Final pass (from intermediate code to the target code)

Usually invoked one after the other for the entire module, where a
compile-time expressions is needed, then the first three passes have to be
done immediately (and the result had better be a constant value..)

I use the same structure now when generating byte-code (originally such a
compiler was just single-pass). Because such code is usually dynamically
typed, the type analysis pass only needs a nominal amount of work, but still
takes care of a few things (l-values for example).

> some past experiments (special purpose tests), have implied that
> potentially threaded code can pull off "reasonably solid" interpreter
> performance (potentially within 3-5x of native).

Assuming you're talking about dynamic typing, I found it difficult to get
within 3-5x, unless some sort of type hinting is used, or perhaps you're
comparing with not too highly optimised native code. Or it's code that is
memory-intensive, then memory access will dominate.

--
Bartc

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-17 21:28 +0000
  Re: Good practical language and OS agnostic text? Philip Herron <redbrain@gcc.gnu.org> - 2012-04-18 14:25 +0100
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 16:32 +0000
      Re: Good practical language and OS agnostic text? arnold@skeeve.com (Aharon Robbins) - 2012-04-20 03:58 +0000
        Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-22 10:10 +0000
      Re: Good practical language and OS agnostic text? "BartC" <bc@freeuk.com> - 2012-04-20 09:45 +0100
        Re: Good practical language and OS agnostic text? "Jonathan Thornburg" <jthorn@astro.indiana.edu> - 2012-04-21 15:04 +0000
  Re: Good practical language and OS agnostic text? BGB <cr88192@hotmail.com> - 2012-04-18 08:39 -0700
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 17:32 +0000
  Re: Good practical language and OS agnostic text? Alain Ketterlin <alain@dpt-info.u-strasbg.fr> - 2012-04-18 18:24 +0200
    Re: Good practical language and OS agnostic text? Hans-Peter Diettrich <DrDiettrich1@aol.com> - 2012-04-19 13:53 +0200
      Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-21 03:07 +0000
      Re: Good practical language and OS agnostic text? "BartC" <bc@freeuk.com> - 2012-04-21 12:01 +0100
        Re: code quality, was Good practical language and OS agnostic text? Hans-Peter Diettrich <DrDiettrich1@aol.com> - 2012-04-22 12:41 +0200
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 11:31 +0000
      Re: Good practical language and OS agnostic text? "Jonathan Thornburg" <jthorn@astro.indiana.edu> - 2012-04-20 16:19 +0000
  Re: Good practical language and OS agnostic text? "Derek M. Jones" <derek@knosof.co.uk> - 2012-04-18 18:16 +0100
    Re: Good practical language and OS agnostic text? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-04-18 22:43 +0000
      Re: Good practical language and OS agnostic text? BGB <cr88192@hotmail.com> - 2012-04-19 00:05 -0700
      Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 11:31 +0000
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 16:32 +0000
  Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-18 19:30 +0000
    Re: Good practical language and OS agnostic text? "BartC" <bc@freeuk.com> - 2012-04-19 18:43 +0100
  Re: Good practical language and OS agnostic text? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-04-18 20:29 +0000
    Re: Good practical language and OS agnostic text? Hans-Peter Diettrich <DrDiettrich1@aol.com> - 2012-04-19 14:20 +0200
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 19:05 +0000
      Re: Good practical language and OS agnostic text? Uli Kusterer <ulimakesacompiler@googlemail.com> - 2012-04-21 11:30 +0200
  Re: Good practical language and OS agnostic text? Roberto Waltman <usenet@rwaltman.com> - 2012-04-18 22:00 -0400
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-19 11:31 +0000
      Re: Good practical language and OS agnostic text? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-04-20 07:02 +0000
        Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-22 11:10 +0000
          Re: Good practical language and OS agnostic text? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-04-22 23:56 +0000
        Re: PL/360, was Good practical language and OS agnostic text? ArarghMail204@Arargh.com - 2012-04-24 19:13 -0500
  Re: Good practical language and OS agnostic text? Bakul Shah <usenet@bitblocks.com> - 2012-04-18 21:15 -0700
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-20 16:06 +0000
  Re: Good practical language and OS agnostic text? torbenm@diku.dk (Torben Ægidius Mogensen) - 2012-04-19 14:58 +0200
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-20 16:06 +0000
  Re: Good practical language and OS agnostic text? "Joe Schmo" <askmeforit@myisp.com> - 2012-04-21 02:53 -0600
    Re: Writing parsers, was Good practical language and OS agnostic text? Uli Kusterer <ulimakesacompiler@googlemail.com> - 2012-04-22 16:18 +0200
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-23 19:12 +0000
  Re: Good practical language and OS agnostic text? Uli Kusterer <ulimakesacompiler@googlemail.com> - 2012-04-21 11:22 +0200
    Re: Good practical language and OS agnostic text? BGB <cr88192@hotmail.com> - 2012-04-21 18:58 -0700
      Re: writing interpreters, was Good practical language and OS agnostic text? Uli Kusterer <ulimakesacompiler@googlemail.com> - 2012-04-22 12:53 +0200
        Re: writing interpreters, was Good practical language and OS agnostic text? BGB <cr88192@hotmail.com> - 2012-04-22 12:29 -0700
      Re: generating bytecode, was Good practical language and OS agnostic text? Uli Kusterer <ulimakesacompiler@googlemail.com> - 2012-04-22 13:12 +0200
      Re: Recursive descent parsing and optimization, was Good practical language and OS agnostic text? "BartC" <bc@freeuk.com> - 2012-04-22 12:51 +0100
        Re: Recursive descent parsing and optimization, was Good practical language and OS agnostic text? "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> - 2012-04-22 18:18 +0200
          Re: Recursive descent parsing and optimization, was Good practical language and OS agnostic text? "Bartc" <bartc@freeuk.com> - 2012-04-23 10:59 +0100
        Re: Recursive descent parsing and optimization, was Good practical language and OS agnostic text? BGB <cr88192@hotmail.com> - 2012-04-22 13:45 -0700
    Re: Good practical language and OS agnostic text? compilers@is-not-my.name - 2012-04-22 22:11 +0000
      Re: Good practical language and OS agnostic text? "BartC" <bc@freeuk.com> - 2012-04-23 18:41 +0100
  Re: Good practical language and OS agnostic text? basile@starynkevitch.net - 2012-05-02 22:16 -0700
    Re: Good practical language and OS agnostic text? Johann 'Myrkraverk' Oskarsson <johann@2ndquadrant.com> - 2012-06-06 16:52 +0000

csiph-web