Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #3025

An ingenious scanning problem!

Show all headers | View raw

Hi Folks,

Page 115-116 of the Flex User's Manual [1] has an ingenious problem posed by
Jan Kort on 05 September 1998.

Here's a description of the problem:

The input contains this: TEST1\n

That is, the input contains the string TEST1 followed by a newline.

The Flex scanner contains a rule that matches on the input:

TEST1\n     { action }

Here's a description of its action:

	Of the text that was matched, I only
	want the first 5 characters (i.e., TEST1).

That action is expressed in Flex this way:

TEST1\n    { yyless(5); }

In the scanner there are also two newline rules.

One newline rule matches on a newline at the beginning of a line (i.e., empty
line):

^\n     { action }

The other newline rule matches on a newline at the end of a line:

\n     { action }

Question: After the action for TEST1\n is executed, which newline rule is
executed?

The answer depends on the meaning of yyless(). Here are two possible meanings
for yyless():

(a) The scanner is backing up in the input stream.
(b) The scanner is pushing new characters onto the input stream.

If the meaning of yyless() is (a), then the second newline rule will be
executed.

If the meaning of yyless() is (b), then the first newline rule will be
executed.

So which does Flex do?

............................

Answer: (b)

Flex treats the newline from TEST1\n as a newline at the beginning of a line
(i.e., empty line). Flex executes this rule:

^\n     { action }

Such an ingenious problem!

Comments?

/Roger

[1] https://epaperpress.com/lexandyacc/download/flex.pdf
[This is a good example of the reasons you don't want your lexer to be too clever. -John]

Back to comp.compilers | Previous | Next — Next in thread | Find similar

Thread

An ingenious scanning problem! Roger L Costello <costello@mitre.org> - 2022-05-28 19:03 +0000
  RE: An ingenious scanning problem! Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-06-01 13:34 +0300

csiph-web