Re: Best language for implementing compilers?

From	Christopher F Clark <christopher.f.clark@compiler-resources.com>
Newsgroups	comp.compilers
Subject	Re: Best language for implementing compilers?
Date	2019-03-11 10:06 -0700
Organization	Compilers Central
Message-ID	<19-03-014@comp.compilers> (permalink)
References	<19-02-002@comp.compilers> <19-02-004@comp.compilers> <19-02-006@comp.compilers> <19-03-009@comp.compilers> <19-03-011@comp.compilers>

Show all headers | View raw

On Sunday, March 10, 2019 at 9:08:12 PM UTC-4, Hans-Peter Diettrich wrote:
> Am 10.03.2019 um 12:13 schrieb Christopher F Clark:
>
> > All that said, the output of any decent
> > C/C++ lexer and parser generator is often more than fast enough.  That's
> > despite lexing and parsing often taking upto a third of the compilation
time.
> > BTW, lexing (because it looks at every character) is the dominant factor
in
> > that.
>
> If so, then lexing were the dominant factor for *all* parsers with a
> lexer, not only for C/C++.
>
> My experience and the existence of specific workarounds identify two C
> specific properties as most time consuming, the preprocessor and the
> lack of a multi-module (project) compilation. None of these is related
> to a *lexer in the strict sense* (tokenizer), because it all happens in
> between the tokenizer and parser.
>
> DoDi

Sorry for the confusion.  By C/C++ lexer and parser, I meant one written in
those languages, not compiling those languages.  To be more precise, I mean
ones where a program has generated the parsing and lexing tables and the code
that interprets those tables is in C/C++.  You can get pretty close to the
optimal code sequence using C code.  I suspect with jit'ed Java code or
similar you can probably get similar performance and code sequences.

Moreover, I was talking raw lexing and parsing speed of reading source code.
If you don't "read" the source code at all, e.g. precompiled headers, you can
go much faster.  However, in that case, you probably have to "read" but not
lex or parse the predigested code from a file.

Your point about the C preprocessor is also valid. Depending upon the
implementation, you might have to lex the code twice.  Once to get
preprocessor tokens and again to get the kind of tokens the parser consumes.
Of course, a clever design can probably find ways to use mostly the same
tokens for both purposes and skip the 2nd tokenizing except when features like
token pasting are used.  On the other hand, a naive implementation might
actually write out the pre-processed file and re-read it.  As you can imagine
the performance difference between the two implementations will likely be
noticable.
[I've seen systems that cache tokenized header files which should help.  It's a
little tricky due to token pasting, but it's not that hard. -John]

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Best language for implementing compilers? "Costello, Roger L." <costello@mitre.org> - 2019-02-08 12:20 +0000
  Re: Best language for implementing compilers? Nala Ginrut <nalaginrut@gmail.com> - 2019-02-09 00:27 +0800
  Re: Best language for implementing compilers? George Neuner <gneuner2@comcast.net> - 2019-02-08 18:36 -0500
    Re: Best language for implementing compilers? Bart <bc@freeuk.com> - 2019-02-11 12:59 +0000
      Re: Best language for implementing compilers? drb@ihatespam.msu.edu (Dennis Boone) - 2019-02-12 09:38 -0600
        Re: Best language for implementing compilers? drb@ihatespam.msu.edu (Dennis Boone) - 2019-02-19 11:22 -0500
          Re: Best language for implementing compilers? arnold@skeeve.com (Aharon Robbins) - 2019-02-20 11:48 +0000
      Re: Best language for implementing compilers? Kaz Kylheku <157-073-9834@kylheku.com> - 2019-02-12 16:45 +0000
      Re: Best language for implementing compilers? mertesthomas@gmail.com - 2019-03-09 01:47 -0500
        Re: Best language for implementing compilers? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2019-03-09 10:14 +0100
          Re: Best language for implementing compilers? George Neuner <gneuner2@comcast.net> - 2019-03-09 22:47 -0500
          Re: Best language for implementing compilers? Kaz Kylheku <157-073-9834@kylheku.com> - 2019-03-10 05:40 +0000
        Re: Best language for implementing compilers? Bart <bc@freeuk.com> - 2019-03-09 12:34 +0000
        Re: Best language for implementing compilers? George Neuner <gneuner2@comcast.net> - 2019-03-09 22:57 -0500
          Re: Best language for implementing compilers? Kaz Kylheku <157-073-9834@kylheku.com> - 2019-03-10 05:48 +0000
      Re: Best language for implementing compilers? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2019-03-10 04:13 -0700
        Re: Best language for implementing compilers? Bart <bc@freeuk.com> - 2019-03-10 15:33 +0000
          Re: Best language for implementing compilers? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2019-03-11 10:49 -0700
            Re: Best language for implementing compilers? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2019-03-12 06:54 +0100
              Re: Best language for implementing compilers? Bart <bc@freeuk.com> - 2019-03-13 01:50 +0000
        Re: Best language for implementing compilers? Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2019-03-10 16:13 +0100
          Re: Best language for implementing compilers? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2019-03-11 10:06 -0700
        Re: Best language for implementing compilers? George Neuner <gneuner2@comcast.net> - 2019-03-10 18:23 -0400
        Re: Best language for implementing compilers? George Neuner <gneuner2@comcast.net> - 2019-03-10 18:58 -0400
  Re: Best language for implementing compilers? "Robin Vowels" <robin51@dodo.com.au> - 2019-02-09 19:58 +1100
  Best language for implementing compilers? David Lovemore <davidlovemore@gmail.com> - 2019-02-12 03:28 -0800
  Re: Best language for implementing compilers? mertesthomas@gmail.com - 2019-03-12 10:40 -0700

csiph-web