What does it mean to "move characters" in the lexer?

From	Roger L Costello <costello@mitre.org>
Newsgroups	comp.compilers
Subject	What does it mean to "move characters" in the lexer?
Date	2022-06-21 10:27 +0000
Organization	Compilers Central
Message-ID	<22-06-057@comp.compilers> (permalink)

Show all headers | View raw

Hi Folks,

Page 89 of the dragon book says:

Because a large amount of time can be consumed moving characters, specialized
buffering techniques have been developed to reduce the amount of overhead to
process an input character.

Page 102 of "A Retargetable C Compiler: Design and Implementation" says:

The lexical analyzer's main activity is moving characters, so minimizing the
amount of character movement helps increase speed.

And on page 103 it says:

An important consequence of this design is that most of the input characters
are accessed by *cp and many characters are never moved. Only identifiers
(excluding keywords) and string literals that appear in executable code are
copied out of the buffer into permanent storage.

I don't understand what they mean by "moving characters". Do they mean copying
characters? Do they mean reading characters from a file into memory? Would you
explain what this "character movement" thing is all about, please?

/Roger
[Keeping in mind that this was written in the 1970s, they mean copying strings
of characters from one place to another.  On a PDP-11.  With 64K bytes of memory.
It is still true that character processing in a lexer consumes a large fraction
of the time in compilers.  -John]

Back to comp.compilers | Previous | Next — Next in thread | Find similar

Thread

What does it mean to "move characters" in the lexer? Roger L Costello <costello@mitre.org> - 2022-06-21 10:27 +0000
  Re: What does it mean to "move characters" in the lexer? gah4 <gah4@u.washington.edu> - 2022-06-21 10:30 -0700
    Re: What does it mean to "move characters" in the lexer? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-06-22 00:44 +0300
      Re: What does it mean to "move characters" in the lexer? Kaz Kylheku <480-992-1380@kylheku.com> - 2022-06-22 01:13 +0000
        Re: What does it mean to "move characters" in the lexer? Thomas Koenig <tkoenig@netcologne.de> - 2022-06-22 11:45 +0000
  Re: What does it mean to "move characters" in the lexer? Kaz Kylheku <480-992-1380@kylheku.com> - 2022-06-22 01:05 +0000

csiph-web