Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #2971 > unrolled thread

Why is flex pattern-matching of NULs slow?

Back to article view | Back to comp.compilers

Contents

  Why is flex pattern-matching of NULs slow? Roger L Costello <costello@mitre.org> - 2022-04-08 11:06 +0000
    Why is flex pattern-matching of NULs slow? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-04-09 21:40 +0300

#2971 — Why is flex pattern-matching of NULs slow?

Hi Folks,

The Flex manual says this:

    Pattern-matching of NULs is substantially slower
    than matching other characters.

Why is that?

/Roger
[My recollection is that zero is used as a flag value in internal
tables and there is some slow kludge to say that this is a nul not the
flag, but perhaps someone who has looked at the code more recently
will remember the details. -John]

[toc] | [next] | [standalone]

#2976

I haven't looked at Flex in a while either, but what I remember is
that 0 is used as end of buffer and EOF indication and that you had to
validate against that.  I don't recall whether that required an
attempt at reading or not.  It wouldn't surprise me if it were used as
a flag also, and for a "null pointer".  Depending upon how you look at
it, C either hates 0 or loves it, but it is very often "special".

But if you are parsing human readable ASCII text, having 0 (NUL) be an
EOF mark is actually not a bad solution.  If I recall correctly, that
isn't even a bad choice for human readable UTF-8 (including
non-latin-1 texts, because 2 and 3 byte sequences don't have NULs in
them).  It only becomes a pain if you want to parse binary data.

By the way, in our lexer, we used -1, i.e. what getc used to return
for EOF for the same condition and I don't recall how we put it in the
buffer (or whether we even did).  Being ex-PL/I and Pascal
programmers, we used strings with lengths in many places instead of C
strings.  I don't remember whether we used Paul Abrahams clever hack
to put the length at the end of the string which if done right also
serves as a null byte for use as C strings.

--
******************************************************************************
Chris Clark                  email: christopher.f.clark@compiler-resources.com
Compiler Resources, Inc.  Web Site: http://world.std.com/~compres
23 Bailey Rd                 voice: (508) 435-5016
Berlin, MA  01503 USA      twitter: @intel_chris
------------------------------------------------------------------------------
[You're right about UTF-8, where NUL is also a reasonable string terminator.
UTF-8 is self-synchonizing -- the bytes of no UTF-8 code point are a prefix
or suffix of any other code point. -John]

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compilers

csiph-web