Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #3132

Scannerless parsing was: Why does the lexer convert text integer lexemes ...?

Show all headers | View raw

On 18/07/2022 06:39, gah4 wrote:
> [In my experience separating the lexer from the parser makes it a lot easier
> to deal with common lexical situations like skipping white space and comments.
> You could certainly do that in a combined scheme but I'm not sure it would end
> up any simpler. -John]

Maybe not simpler, but it won't be necessarily more complex. I've just
transferred an example for SQL from my old desktop. The FSA/GLR parser
built can parse ie this command, without a scanner, unambiguously in the
simulator:

SELECT ALL FIRST,LAST FROM USERS;

Below, there are four BNF rules. The first has in the right part the
required lookahead operator '-:' as the grammar allows consecutive IDs.

The second rule may be seen as a meta-rule that will be expanded. That
is, the Builder will make the 'dirty' job for the programmer and attach
the separator to each element listed in the 4th BNF rule below, which in
turn must have only one symbol in each alternative (in the right side).

IMHO, this grammar doesn't look very complex, but others may see it so.

Regards,
Ev. Drikos

------------------------------------------------------------------------

<identifier body> ::=
        <identifier start> [ <identifier part> ... ] -: <identifier part>

(<can be followed by separator>) ::=
            ( <can be followed by separator> ) <separator>
      |     ( <can be followed by separator> )

<separator> ::=
            { <comment> | <white spaces> }...

<can be followed by separator> ::=
            <regular identifier>
      |     <unsigned numeric literal>
..
...
      |     <left brace>
      |     <right brace>

[I don't see how this grammar will allow a comment before the statement. -John]

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Roger L Costello <costello@mitre.org> - 2022-07-14 10:25 +0000
  Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? gah4 <gah4@u.washington.edu> - 2022-07-14 10:03 -0700
  Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? George Neuner <gneuner2@comcast.net> - 2022-07-14 16:38 -0400
    Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Spiros Bousbouras <spibou@gmail.com> - 2022-07-15 07:08 +0000
    Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Jan Ziak <0xe2.0x9a.0x9b@gmail.com> - 2022-07-15 03:02 -0700
      Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Jan Ziak <0xe2.0x9a.0x9b@gmail.com> - 2022-07-15 10:50 -0700
      Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? George Neuner <gneuner2@comcast.net> - 2022-07-17 16:52 -0400
        Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Thomas Koenig <tkoenig@netcologne.de> - 2022-07-18 05:44 +0000
      Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? George Neuner <gneuner2@comcast.net> - 2022-07-17 18:01 -0400
  Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Kaz Kylheku <480-992-1380@kylheku.com> - 2022-07-15 14:41 +0000
  Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? "matt.ti...@gmail.com" <matt.timmermans@gmail.com> - 2022-07-16 05:32 -0700
  Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? Christopher F Clark <christopher.f.clark@compiler-resources.com> - 2022-07-17 13:10 -0400
    Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? gah4 <gah4@u.washington.edu> - 2022-07-17 20:39 -0700
      Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? gah4 <gah4@u.washington.edu> - 2022-07-19 16:39 -0700
        Re: Why does the lexer convert text integer lexemes to binary integers? I thought that lexers should be simple? luser droog <luser.droog@gmail.com> - 2022-07-21 14:16 -0700
      Scannerless parsing was: Why does the lexer convert text integer lexemes ...? "Ev. Drikos" <drikosev@gmail.com> - 2022-07-21 13:41 +0300
        Re: Scannerless parsing was: Why does the lexer convert text integer lexemes ...? "Ev. Drikos" <drikosev@gmail.com> - 2022-07-22 12:29 +0300

csiph-web