Re: coupling LALR with a scanner?

Path	csiph.com!x330-a1.tempe.blueboxinc.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!border3.nntp.dca.giganews.com!border1.nntp.dca.giganews.com!border4.nntp.dca.giganews.com!border2.nntp.dca.giganews.com!nntp.giganews.com!news.iecc.com!lnews.iecc.com!nerds-end
From	Chris F Clark <cfc@shell01.TheWorld.com>
Newsgroups	comp.compilers
Subject	Re: coupling LALR with a scanner?
Date	Mon, 03 Oct 2011 11:59:42 -0400
Organization	The World Public Access UNIX, Brookline, MA
Lines	24
Sender	news@iecc.com
Approved	comp.compilers@iecc.com
Message-ID	<11-10-006@comp.compilers> (permalink)
References	<11-07-013@comp.compilers> <11-07-015@comp.compilers> <11-07-018@comp.compilers> <11-08-004@comp.compilers> <11-09-016@comp.compilers> <11-09-017@comp.compilers> <11-09-022@comp.compilers> <11-09-023@comp.compilers> <11-10-003@comp.compilers> <11-10-005@comp.compilers>
NNTP-Posting-Host	lnews.iecc.com
X-Trace	gal.iecc.com 1317672328 78740 64.57.183.34 (3 Oct 2011 20:05:28 GMT)
X-Complaints-To	abuse@iecc.com
NNTP-Posting-Date	Mon, 3 Oct 2011 20:05:28 +0000 (UTC)
Keywords	parse
Posted-Date	03 Oct 2011 16:05:28 EDT
X-submission-address	compilers@iecc.com
X-moderator-address	compilers-request@iecc.com
X-FAQ-and-archives	http://compilers.iecc.com
Xref	x330-a1.tempe.blueboxinc.net comp.compilers:288

Show key headers only | View raw

"Armel" <armelasselin@hotmail.com> writes:

>> there is an argument for introducing such a gap to "segment"
>> tokens into smaller chunks for both performance and expressibility
>> reasons.
>
> could you elaborate on this segmentation mechanism?

I would like to, but the idea is going through the patent process, as
it has hardware design implications.  Once I know that the patent has
been filed, or get other ok from Intel legal, I will write the idea
up.

> In my lexer generator, the developer can introduce start states by himself
> and 'cut' complex expressions into smaller expressions which still respect
> AFD capabilities and introduce dynamic regular expressions where absolutely
> necessary, for languages allowing dynamic string delimiters for example
> (e.g. doc-strings like << END_OF_STR_MARKER, some lines then a line with
> END_OF_STR_MARKER only on the line), languages such as ruby are very funny
> from that point of view if I remember well.

Those sound like useful extensions.  In the hands of an experienced
parser writer, that could make the process much easier.  Let the
system figure out what it can, then adjust where you must.

Back to comp.compilers | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-07-05 01:02 +0200
  Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-07-07 10:28 +0200
    coupling LALR with a scanner? "Karsten Nyblad" <uu3kw29sb7@snkmail.com> - 2011-07-07 10:46 +0200
    Re: coupling LALR with a scanner? "Karsten Nyblad" <uu3kw29sb7@snkmail.com> - 2011-07-08 14:39 +0200
      Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-08-04 11:17 +0200
        Re: coupling LALR with a scanner? Paul B Mann <paul@paulbmann.com> - 2011-09-13 13:38 -0700
          Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-09-16 10:47 +0200
            Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-09-19 13:52 +0200
            Re: coupling LALR with a scanner? Paul B Mann <paul@paulbmann.com> - 2011-09-19 12:12 -0700
              Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-09-20 09:40 +0200
                Re: coupling LALR with a scanner? Chris Dodd <cdodd@acm.org> - 2011-09-23 23:59 +0100
                Re: coupling LALR with a scanner? Chris F Clark <cfc@shell01.TheWorld.com> - 2011-09-29 00:00 -0400
                Re: coupling LALR with a scanner? "Armel" <armelasselin@hotmail.com> - 2011-10-02 16:41 +0200
                Re: coupling LALR with a scanner? Chris F Clark <cfc@shell01.TheWorld.com> - 2011-10-03 11:59 -0400
                Re: coupling LALR with a scanner? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-10-03 21:08 +0000
          Re: coupling LALR with a scanner? Paul B Mann <paul@paulbmann.com> - 2011-09-17 10:38 -0700

csiph-web