Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #35414 > unrolled thread

RFD Recognizer

Back to article view | Back to comp.lang.forth

Contents

  RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-02 11:56 -0800
    Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-03 05:37 -0600
      Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-03 12:56 +0000
        Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-04 04:23 -0600
          Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-04 11:25 +0000
            Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-05 04:43 -0600
              Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-05 12:35 +0000
                Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-06 10:33 -0600
                  Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-07 15:29 +0000
                    Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-07 12:57 -0600
                      Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-07 12:27 -0800
                        Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-09 00:42 +0100
                      Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-07 20:55 +0000
      Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-03 06:43 -0800
        Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-03 07:54 -0800
          Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-03 08:37 -0800
            Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-03 10:08 -0800
          Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-04 12:15 +0000
            Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-09 10:15 -0800
              Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-09 12:11 -0800
                Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-09 22:46 +0100
        Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-04 04:37 -0600
          Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-04 06:32 -0800
            Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-05 04:55 -0600
              Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-05 10:06 -0800
        Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-04 16:02 +0000
          Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-04 11:06 -0800
            Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-05 08:18 -0800
              Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-05 08:59 -0800
              Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-05 10:23 -0800
                Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-08 15:36 +0000
                  Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-08 10:54 -0800
                  Re: RFD Recognizer Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2015-01-08 20:50 +0000
            Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-08 14:24 +0000
              Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-08 16:06 +0000
              Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-09 01:11 +0100
          Re: RFD Recognizer Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-01-05 17:19 +0100
      Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-03 16:08 +0100
      Re: RFD Recognizer hughaguilar96@gmail.com - 2015-01-03 16:05 -0800
        Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-04 04:42 -0600
        Re: RFD Recognizer stephen@mpeforth.com (Stephen Pelc) - 2015-01-06 18:01 +0000
      Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-04 15:11 +0000
        Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-05 05:00 -0600
          Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-05 13:39 +0000
            Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-06 04:57 -0600
              Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-06 18:00 +0000
                Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-07 05:13 -0600
                  Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-07 17:18 +0000
                    Re: RFD Recognizer Andrew Haley <andrew29@littlepinkcloud.invalid> - 2015-01-07 13:04 -0600
    Re: RFD Recognizer Julian Fondren <julian.fondren@gmail.com> - 2015-01-04 07:48 -0800
      Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-05 06:01 -0800
        Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-05 14:31 +0000
          Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-05 08:43 -0800
            Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-08 15:22 +0000
          Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-06 02:48 +0100
            Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-06 03:13 +0000
      Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-08 13:45 +0000
    Re: RFD Recognizer Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2015-01-08 20:44 +0000
      Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-17 09:26 -0800
        Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-17 18:54 +0100
          Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-17 12:35 -0800
        Re: RFD Recognizer Matthias Trute <matthias.trute@gmail.com> - 2015-01-18 04:16 -0800
          Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-18 13:38 +0000
            Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-18 13:56 +0000
              Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-18 07:10 -0800
                Re: RFD Recognizer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-18 16:14 +0000
        Re: RFD Recognizer Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2015-01-20 18:26 +0000
          Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-20 23:54 +0100
            Re: RFD Recognizer JennyB <jennybrien@googlemail.com> - 2015-01-21 09:05 -0800
              Re: RFD Recognizer Bernd Paysan <bernd.paysan@gmx.de> - 2015-01-21 20:50 +0100
              Re: RFD Recognizer albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-22 10:55 +0000

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#35674

Matthias Trute wrote:
> A surprising (for me) outcome is that the SEARCH ORDER wordlist may
> be dropped from the standard. Recognizers can take over all of their
> functionality. I'd suggest another RFD for that however.

Things can not easily go out of the standard.  It might be that the search 
order is superfluous if you have a recognizer stack where vocabularies or 
wordlists are just recognizers.

There is some caveat: The locals are supposed to be found before any search 
in the standard wordlists are performed, and ALSO/PREVIOUS are specified to 
not affect that locals search.  It is pretty natural to implement locals as 
recognizer which gets pushed on top of the recognizer stack, but you can't 
push it on the top of the search order.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/
net2o ID: kQusJzA;7*?t=uy@X}1GWr!+0qqp_Cn176t4(dQ*

[toc] | [prev] | [next] | [standalone]

#35436

Matthias Trute <matthias.trute@gmail.com> wrote:
> Andrew,
> 
>> My initial reactions:
>> 
>> 
>> 3.  I'm not convinced that it makes any sense to invoke the recognizer
>>     mechanism for simple dictionary lookups.  It's a big chunk of
>>     overhead for such a critical inner loop.
> 
> The other way around: The interpreter (the recognizer loop) calls
> the dictionary lookup.

Yes, I know.

> And it does it once, just like INTERPRET calls FIND once as of
> today. The dictionary lookup works as before, without recognizers.

I know how it works: I disagree that it should work that way.

>> 4.  How often does it make sense to have a POSTPONE action for a
>>     recognizer?  Right now I can think of no use for it at all.
> 
> POSTPONE a word from the dictionary is well established and can cause
> trouble wrt the immediate flag handling.

I don't know what this refers to.

> POSTPONing something different is new.
> 
> btw: this is an area, where I think that the name tokens introduced with
> NAME>COMPILE and friends will become really interesting.

But not necessarily useful.
 
>> 5.  SET-RECOGNIZERS and GET-RECOGNIZERS push a lot of stuff onto the
>>     data stack for no obvious purpose.
> 
> They are modelled after GET-ORDER/SET-ORDER. Same question, same answer.

It's not at all clear that GET-ORDER/SET-ORDER is a model that should
be followed.  This porposal has to stand on its own.

>> So, while the basic idea is OK, I think this is an over-engineered
>> proposal which could be cut down to something smaller without loss of
>> power.
> 
> I welcome feedback. IMHO it is already pretty small. The R:FAIL vs 0 flag
> is noticed. What else?

I reckon it could be cut down to about half the size: the POSTPONE
action is AFAICS a solution looking for a problem, so it could go.
Recognizers could be a simple stack.  They could be a simple fallback
when dictionary lookup failed.

>> I am rather concerned that most of the more exotic uses for this won't
>> make Forth programs any easier to read or write, but are more of the
>> unnecessarily "cute" syntactical tricks loved by a subset of Forth
>> programmers, especially newbies.
> 
> I don't consider myself a newbie in Forth, despite being quiet at
> c.l.f. until now.

This seems like a bit of a non-reply.  Note the "especially
newbies". not exclusively.

>> (I have this nagging feeling that we may be about to repeat the
>> mistakes of Smalltalk-72, where programs parsed their own
>> arguments.  The system was made much cleaner in Smalltalk-80 which
>> has a fixed simple syntax.)
> 
> There are recognizers available to do exactly that. Words like S"
> and TO could become obsolete.

S" and TO could be made obsolete if we decided.

We have to be discuss this proposal seriously.  It has great and
far-reaching consequences, and not all of these consequences are
necessarily good.  It is potentially the most significant change to
the Forth language for decades.

Andrew.

[toc] | [prev] | [next] | [standalone]

#35451

> >> 3.  I'm not convinced that it makes any sense to invoke the recognizer
> >>     mechanism for simple dictionary lookups.

> I know how it works: I disagree that it should work that way.

Let me be clear: Recognizers call the dictionary lookup. A dictionary
lookup does not call or use a recognizer.

*If* (again: IF) the dictionary lookup is changed into a list of recognizers
things would be different, but *that* proposal may be written by someone 
else, not me.

> It's not at all clear that GET-ORDER/SET-ORDER is a model that should
> be followed.  This porposal has to stand on its own.

Sorry for the mis-understanding. I use these words to change the recognizer
stack by using the standard data stack words. For me is DUP much nicer
to use than an ALSO. By defining the get/set-recognizer words I can use
DUP (or whatever else fits the problem at hand) to change the stack and
I don't have to invent words like ALSO.

One alternative way to deal with stacks (esp those stacks which I call 
configuration stacks like recognizers or search order stacks) would 
be words like >stack and stack>, similar to the return stack 
words >r and r>. IMHO a matter of taste as long as the stack
depth for the configuration stacks is limited.

> >> So, while the basic idea is OK, I think this is an over-engineered
> >> proposal which could be cut down to something smaller without loss of
> >> power.
> > 
> > I welcome feedback. IMHO it is already pretty small. The R:FAIL vs 0 flag
> > is noticed. What else?
> 
> I reckon it could be cut down to about half the size: the POSTPONE
> action is AFAICS a solution looking for a problem, so it could go.

I wouln't let it go, but it may be made optional by placing it into
a (new) RECOGNIZER-EXT section.

> Recognizers could be a simple stack.  They could be a simple fallback
> when dictionary lookup failed.

For that the introduction section which describes the changes to to
text interpreter needs some clarification, right? That would make it
clear that recognizers are really optional and are not necessarily
part of the core system, even if supported.

> S" and TO could be made obsolete if we decided.

Sure, those suggestions are not part of the recognizer
proposal, they illustrate only what you wrote here:

> We have to be discuss this proposal seriously.  It has great and
> far-reaching consequences, and not all of these consequences are
> necessarily good.

Matthias

[toc] | [prev] | [next] | [standalone]

#35467

Matthias Trute <matthias.trute@gmail.com> wrote:
>> >> 3.  I'm not convinced that it makes any sense to invoke the recognizer
>> >>     mechanism for simple dictionary lookups.
> 
>> I know how it works: I disagree that it should work that way.
> 
> Let me be clear: Recognizers call the dictionary lookup. A dictionary
> lookup does not call or use a recognizer.

Yes, I know.

>> >> So, while the basic idea is OK, I think this is an over-engineered
>> >> proposal which could be cut down to something smaller without loss of
>> >> power.
>> > 
>> > I welcome feedback. IMHO it is already pretty small. The R:FAIL vs 0 flag
>> > is noticed. What else?
>> 
>> I reckon it could be cut down to about half the size: the POSTPONE
>> action is AFAICS a solution looking for a problem, so it could go.
> 
> I wouln't let it go, but it may be made optional by placing it into
> a (new) RECOGNIZER-EXT section.

It shouldn't be in any section at all unless it becomes useful in
Forth applications.

>> Recognizers could be a simple stack.  They could be a simple fallback
>> when dictionary lookup failed.
> 
> For that the introduction section which describes the changes to to
> text interpreter needs some clarification, right? That would make it
> clear that recognizers are really optional and are not necessarily
> part of the core system, even if supported.

That's right.  My proposal would be to replace Section c in 3.4 The
Forth text interpreter with

c) If a definition name matching the string is not found, [description
of calling the recogizer stack]

I repeat:

>> We have to be discuss this proposal seriously.  It has great and
>> far-reaching consequences, and not all of these consequences are
>> necessarily good.

Andrew.

[toc] | [prev] | [next] | [standalone]

#35483

> > I wouln't let it go, but it may be made optional by placing it into
> > a (new) RECOGNIZER-EXT section.
> 
> It shouldn't be in any section at all unless it becomes useful in
> Forth applications.

Fine with me. I use it in applications and I know of projects using it.
So I'll keep it.
 
> That's right.  My proposal would be to replace Section c in 3.4 The
> Forth text interpreter with
> 
> c) If a definition name matching the string is not found, [description
> of calling the recogizer stack]

I'll think about a description that makes clear that recognizers can
be the core of the text interpreter or (alternativly) an extension of the
standard interpreter. A major problem for that is, that the standard
text interpreter has no hooks at all. Exception -13 would fit, but exceptions
are optional themselves.

Matthias

[toc] | [prev] | [next] | [standalone]

#35460

In article <f6f5bea4-5b36-4e27-9866-3837595dc982@googlegroups.com>,
Matthias Trute  <matthias.trute@gmail.com> wrote:
>Andrew,
>
>> My initial reactions:
>>
>> 1.  Vectored NUMBER or some more general equivalent has been around
>>     for a long time.  It's a good thing to have.
>
>I played with such strategies too. The big BUT is that they
>solve the problems in a end-of-pipe manner like catching not-found exceptions
>or make the system *really* complex with many such hooks for every
>theoretical/academic situation.

I'd rather not. I came up with denotations to get rid of the special
status numbers had in a Forth system. Instead if look-up-and-if-failed
I got look-up. Then I got string denotations and dictionary-entry-denotations
practically free.

>
>>
>> 2.  This proposal looks rather complicated.  Can it not be made simpler
>>     without loss of functionality?  For example, R:FAIL does not seem
>>     to do anything useful.  It's very cute to make it the failure
>>     action but there's an awful lot of DUP R:FAIL <> IF executed in
>>     the inner loop of DO-RECOGNIZER.  Should the token used for
>>     notifying a failure not simply be zero?
>
>That would work too. R:FAIL has the optical benefit that you don't need
>to define a "speaking" name for the 0.

Contrasting to denotations. They always result in a (double) constant that
was the same for compilation or interpretation.

>
>>
>> 3.  I'm not convinced that it makes any sense to invoke the recognizer
>>     mechanism for simple dictionary lookups.  It's a big chunk of
>>     overhead for such a critical inner loop.
>
>The other way around: The interpreter (the recognizer loop) calls the
>dictionary lookup. And it does it once, just like INTERPRET calls FIND
>once as of today. The dictionary lookup works as before, without recognizers.

The original denotations (@2002) made for a simpler system than
dictionary lookup plus numbers-as-an-exception.

>
>>
>> 4.  How often does it make sense to have a POSTPONE action for a
>>     recognizer?  Right now I can think of no use for it at all.
>
>POSTPONE a word from the dictionary is well established and can cause
>trouble wrt the immediate flag handling. POSTPONing something different
>is new.

Anton Ertl likes postponing numbers. My design of denotations was more
like "numbers are smart anyway" (handled differently during compilation)
"nobody ever postpones numbers" "a dea (xt) is an address (number) that
is the same while compiling or interpreting, so it is a number"
I really based the denotation on the premise that they could not, should
not and would not be postponed, ever.

>
>btw: this is an area, where I think that the name tokens introduced with
>NAME>COMPILE and friends will become really interesting.

I applaud your insight that name tokens are important in this context.

I'd have it this way: There is nothing but dea (dictionary entry addresses
aka name tokens.) That is all we need. name tokens can and should be used for
xt's. If this doesn't result in sufficient speed, use an optimiser.
The optimiser has always the information to work with, because the dea/nt
has pointers to everything.

>
>>
>> 5.  SET-RECOGNIZERS and GET-RECOGNIZERS push a lot of stuff onto the
>>     data stack for no obvious purpose.
>
>They are modelled after GET-ORDER/SET-ORDER. Same question, same answer.
>
>>     They make it possible to
>>     prepend a recognizer before dictionary lookup, but that is a Bad
>>     Idea.  IMO...
>
>It depends. Generally speaking I think that's a great new feature. Not
>only for the "literate programming" Julian Fondren wrote about a few
>days ago.

I'd rather not have a separate mechanism for this. wordlist's fit the
bill. To this purpose I've given a name to the minimum
search order vocabulary such that I can add definitions to it by
making it current. (That name is ONLY, so ONLY is the name of vocabulary.)

>
>>
>> So, while the basic idea is OK, I think this is an over-engineered
>> proposal which could be cut down to something smaller without loss of
>> power.
>
>I welcome feedback. IMHO it is already pretty small. The R:FAIL vs 0 flag
>is noticed. What else?
>
>> I am rather concerned that most of the more exotic uses for this won't
>> make Forth programs any easier to read or write, but are more of the
>> unnecessarily "cute" syntactical tricks loved by a subset of Forth
>> programmers, especially newbies.
>
>I don't consider myself a newbie in Forth, despite being quiet at
>c.l.f. until now.
>
>> (I have this nagging feeling that
>> we may be about to repeat the mistakes of Smalltalk-72, where programs
>> parsed their own arguments.  The system was made much cleaner in
>> Smalltalk-80 which has a fixed simple syntax.)
>
>There are recognizers available to do exactly that. Words like S" and
>TO could become obsolete.

Yourforth is an attempt to make Forth simpler and more consistent
along this road. Unfortunately for now it means sacrificing some ANSI
compatibility.

>
>Matthias


Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#35461

> >
> >btw: this is an area, where I think that the name tokens introduced with
> >NAME>COMPILE and friends will become really interesting.
> 
> I applaud your insight that name tokens are important in this context.

We'll see. As of now name tokens are a rather new feature. With the 
definition of POSTPONE as NAME>COMPILE SWAP POSTPONE LITERAL COMPILE, 
it may be an option to drop the postpone action from the recognizers
indeed. 

The price to pay is to dŕop the possibility of postponing any literals.
Replacing execution tokens with name tokens looks like a huge task however.
IMHO that's another story to tell.

Matthias

[toc] | [prev] | [next] | [standalone]

#35478

On Sunday, 4 January 2015 19:06:57 UTC, Matthias Trute  wrote:
> > >
> > >btw: this is an area, where I think that the name tokens introduced with
> > >NAME>COMPILE and friends will become really interesting.
> > 
> > I applaud your insight that name tokens are important in this context.
> 
> We'll see. As of now name tokens are a rather new feature. With the 
> definition of POSTPONE as NAME>COMPILE SWAP POSTPONE LITERAL COMPILE, 
> it may be an option to drop the postpone action from the recognizers
> indeed. 
> 
> The price to pay is to dŕop the possibility of postponing any literals.

Please don't do that. Recognizers finally make sense of what previously had to be done with immediate parsing words. It is now obvious how to POSTPONE them, because the parsed content is available at compile time.

In fact, the pattern is almost the same:
  
  /compiling action/ DUP >R EXECUTE R> COMPILE,

so with a bit of cunning, it's possible to use the same declaration for all literals without having to explicitly declare a POSTPONE method. It doesn't hold for other cases like mini-oof methods, but in any case to postpone /anything/ you only need one extra cell per recognized type, not per instance.  I think that's good value.

> Replacing execution tokens with name tokens looks like a huge task however.

Perhaps not.  As I understand it, the three actions for a recognizer that returned a name token would be: 

    NAME>INTERPRET EXECUTE
    NAME>COMPILE EXECUTE
    NAME>COMPILE SWAP POSTPONE LITERAL COMPILE,

For a straightforward user-defined word NAME>INTERPRET returns the only xt and NAME>COMPILE returns the xt plus COMPILE, (if default) or EXECUTE (if immediate)

With Standard Forth the headers are opaque, but now we are using REC:WORD instead  of FIND, so each header is a data structure whose address is the name token.  It  holds the xt, a pointer to its recognizer, and possibly other information.  

Define all the words you need in this manner, define REC:WORD to find them, define ' ['] FIND etc. to use REC:WORD, then recompile.

That's all!  ;)

[toc] | [prev] | [next] | [standalone]

#35481

On Monday, 5 January 2015 16:18:41 UTC, JennyB  wrote:

> With Standard Forth the headers are opaque, but now we are using REC:WORD instead  of FIND, so each header is a data structure whose address is the name token.  It  holds the xt, a pointer to its recognizer, and possibly other information.  
> 
> Define all the words you need in this manner, define REC:WORD to find them, define ' ['] FIND etc. to use REC:WORD, then recompile.
> 
> That's all!  ;)

This is a recipe for writing a Forth on top of a parent Forth, which I did many years ago when the Standard was young (9 screens of F83). I think is is now possible even more simply, without any knowledge of the parent's inner workings.

[toc] | [prev] | [next] | [standalone]

#35484

> > The price to pay is to dŕop the possibility of postponing any literals.
> 
> Please don't do that.

Andrew sees them as a burden, not as a possibility for something new. I
agree with him, that the usefulness of having them is somewhat speculative
and more an opinion than a fact.

> > Replacing execution tokens with name tokens looks like a huge task however.
> 
> Perhaps not.  As I understand it, the three actions for a recognizer that 
> returned a name token would be: 
> 
>     NAME>INTERPRET EXECUTE
>     NAME>COMPILE EXECUTE
>     NAME>COMPILE SWAP POSTPONE LITERAL COMPILE,

I prefer to have both the REC:WORD which wraps FIND and in addition to
it a REC:NAME that operates with the name tokens. FIND is well
established and unlikely to be thrown out real soon, and name tokens are too 
new. I remember many discussions around the various semantics, so I doubt that
the final word is already spoken.

Recognizers can help to "play" with them and prove their usefulness (or not)
in a very flexible manner.

Mathias
PS: that REC:WORD is tied to FIND may make a rename to REC:FIND feasible.

[toc] | [prev] | [next] | [standalone]

#35590

Matthias Trute <matthias.trute@gmail.com> writes:
>> > The price to pay is to d=C5=95op the possibility of postponing any lite=
>rals.
>>=20
>> Please don't do that.
>
>Andrew sees them as a burden, not as a possibility for something new.

That seems to be his general outlook, although there are exceptions
like N>R and NR>.  The question is if there is any proposal for
recognizers that he would not object to.

The other question is: If you remove features from your proposal that
supporters of recognizers like, you may lose the support of the
supporters; and you may still not get the support of the naysayers.

That's what happened with the directories proposal
<http://www.forth200x.org/directories.html>; even when I had extracted
an absolute minimum that everybody should be able to agree to
<http://www.forth200x.org/directories1.html>, the same person still
objected, and of course there was little support from anyone else,
because that proposal no longer presented a real advance.

> FIND is well
>established and unlikely to be thrown out real soon,

FIND has been defined very fuzzily since Forth-94, I have seen few
uses in standard programs, avoid it myself, and I hope that a clean
replacement will be proposed soon, making it possible to retire FIND
to obsolescent status soon.  Throwing it out completely will take a
long time, because we do not remove words without due warning.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]

#35603

Anton,
 
> That seems to be his general outlook, although there are exceptions
> like N>R and NR>.  The question is if there is any proposal for
> recognizers that he would not object to.

Every idea should have its advocatus diaboli. And I am thankful for
everyone seriously discussing my idea (for which Andrew is not a
real a.d. he seems to like most of it).

> The other question is: If you remove features from your proposal that
> supporters of recognizers like, you may lose the support of the
> supporters; and you may still not get the support of the naysayers.

I take two things from the discussion: recognizers should not be a
replacement of the original text interpreter and POSTPONE is something
that requires more arguments/use cases to prove its usefulness. I don't
want to remove features from the proposal but am thinking about making
the entry level less high. I never expected that the 1st RFD will be the
last one. It took me a few month to prepare it, the next iteration is
hopefully faster ready for discussion. If I come to the conclusion that
the suggested changes are too troublesome I'll decline them (and give
the reasons). If they can be used with little effort, I may include them.

*I* haven't had to postpone literals yet, for me that's a by-product 
from the design, to be explored by users. I'm sure they will find
crazy uses for it ;)

As of now I have 5 new words, 4 of them are rather trivial management
words, only one is slightly more complex and two CORE words have to change
their implementations (not the semantics). That is pretty small.

> > FIND is well
> >established and unlikely to be thrown out real soon,
> 
> FIND has been defined very fuzzily since Forth-94, I have seen few
> uses in standard programs, avoid it myself, and I hope that a clean
> replacement will be proposed soon, making it possible to retire FIND
> to obsolescent status soon.  Throwing it out completely will take a
> long time, because we do not remove words without due warning.

I've only recently implemented the full name token stuff and started
to play with a REC:NAME instead of the REC:WORD. IMO nice and clean
so far. It will be interesting to follow the discussion. With my 
recognizers I'm certain that I can easily play with anything that 
may come with little to no effort in my programs.

Matthias

[toc] | [prev] | [next] | [standalone]

#35607

On 08/01/2015 15:36, Anton Ertl wrote:
> Matthias Trute<matthias.trute@gmail.com>  writes:
>>>> >> >The price to pay is to d=C5=95op the possibility of postponing any lite=
>> >rals.
>>> >>=20
>>> >>Please don't do that.
>> >
>> >Andrew sees them as a burden, not as a possibility for something new.
> That seems to be his general outlook, although there are exceptions
> like N>R and NR>.  The question is if there is any proposal for
> recognizers that he would not object to.
>
> The other question is: If you remove features from your proposal that
> supporters of recognizers like, you may lose the support of the
> supporters; and you may still not get the support of the naysayers.
>

The proposal shouldn't be watered down for one gainsayer. If put to the 
vote and voting means anything acceptance by the majority should decide.

-- 
Gerry

[toc] | [prev] | [next] | [standalone]

#35587

Matthias Trute <matthias.trute@gmail.com> writes:
>> >
>> >btw: this is an area, where I think that the name tokens introduced with
>> >NAME>COMPILE and friends will become really interesting.
>>=20
>> I applaud your insight that name tokens are important in this context.
>
>We'll see. As of now name tokens are a rather new feature. With the=20
>definition of POSTPONE as NAME>COMPILE SWAP POSTPONE LITERAL COMPILE,=20
>it may be an option to drop the postpone action from the recognizers
>indeed.=20
>
>The price to pay is to d=C5=95op the possibility of postponing any literals=
>.

When you posted your proposal in the mailing list, I pointed out that
you can compose all the three behaviours (interpretation, compilation,
postponing) from three parts:

1) a parse-time action
2) a LITERAL-like action
3) a run-time action

The composition is as follows:

interpretation: parse-time run-time
compilation: parse-time POSTPONE literal-like POSTPONE run-time
postponing:
   parse-time POSTPONE literal-like DOUBLE-POSTPONE literal-like
   DOUBLE-POSTPONE run-time

If you define recognizers through these components, you don't need to
specify the three components, in particular not a POSTPONE action; and
yet POSTPONEing literals works as does any other POSTPONEing of
recognizers.  With that, one might leave it up to systems whether they
support POSTPONEing recognizers or not.

Disadvantage: Does not combine with doing the dictionary lookup as a
recognizer for immediate words:

If you make the immediate word a parse-time action with a noop for
literal-like and noop for run-time, it works correctly for
interpretation and compilation, but not for POSTPONE.  And anything
else is even further from the desired behaviour.  One could enhance
this scheme to support immediate words correctly, but I don't see a
clean way to do that.

So there seems to be a choice:

1) Compose the behaviour of recognizers of these components, but do
not treat the dictionary as a recognizer.

2) Treat the dictionary as a recognizer, but build recognizers from
interpretation, compilation, and postponeing behaviour.

BTW, for ( and \, the standard POSTPONE ( etc. behaviour is not so
great inside ]] ... [[, and they would be better treated als totally
immediate:

]] ( n1 ) 1 + ( n2 ) [[

The standard POSTPONE S" behaviour is also not so nice inside 
]] ... [[, but that can easily be worked around by using a recognizer, 
so it's not a problem.

But for words like IF, we do want the standard POSTPONE IF behaviour
inside ]] ... [[.

[IF] [ELSE] [THEN] is an interesting case, but not using that inside
]] ... [[ is probably little hardship.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]

#35592

In article <2015Jan8.152418@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>Matthias Trute <matthias.trute@gmail.com> writes:
>>> >
>>> >btw: this is an area, where I think that the
>name tokens introduced with
>>> >NAME>COMPILE and friends will become really
>interesting.
>>>=20
>>> I applaud your insight that name tokens are
>important in this context.
>>
>>We'll see. As of now name tokens are a rather
>new feature. With the=20
>>definition of POSTPONE as NAME>COMPILE SWAP
>POSTPONE LITERAL COMPILE,=20
>>it may be an option to drop the postpone action
>from the recognizers
>>indeed.=20
>>
>>The price to pay is to d=C5=95op the
>possibility of postponing any literals=
>>.
>
>When you posted your proposal in the mailing
>list, I pointed out that
>you can compose all the three behaviours
>(interpretation, compilation,
>postponing) from three parts:
>
>1) a parse-time action
>2) a LITERAL-like action
>3) a run-time action
>
>The composition is as follows:
>
>interpretation: parse-time run-time
>compilation: parse-time POSTPONE literal-like
>POSTPONE run-time
>postponing:
>   parse-time POSTPONE literal-like
>DOUBLE-POSTPONE literal-like
>   DOUBLE-POSTPONE run-time
>
>If you define recognizers through these
>components, you don't need to
>specify the three components, in particular not
>a POSTPONE action; and
>yet POSTPONEing literals works as does any other
>POSTPONEing of
>recognizers.  With that, one might leave it up
>to systems whether they
>support POSTPONEing recognizers or not.
>
>Disadvantage: Does not combine with doing the
>dictionary lookup as a
>recognizer for immediate words:
>
>If you make the immediate word a parse-time
>action with a noop for
>literal-like and noop for run-time, it works
>correctly for
>interpretation and compilation, but not for
>POSTPONE.  And anything
>else is even further from the desired behaviour.
>One could enhance
>this scheme to support immediate words
>correctly, but I don't see a
>clean way to do that.
>
>So there seems to be a choice:
>
>1) Compose the behaviour of recognizers of these
>components, but do
>not treat the dictionary as a recognizer.
>
>2) Treat the dictionary as a recognizer, but
>build recognizers from
>interpretation, compilation, and postponeing behaviour.
>
>BTW, for ( and \, the standard POSTPONE ( etc.
>behaviour is not so
>great inside ]] ... [[, and they would be better
>treated als totally
>immediate:
>
>]] ( n1 ) 1 + ( n2 ) [[
>
>The standard POSTPONE S" behaviour is also not
>so nice inside
>]] ... [[, but that can easily be worked around
>by using a recognizer,
>so it's not a problem.
>
>But for words like IF, we do want the standard
>POSTPONE IF behaviour
>inside ]] ... [[.
>
>[IF] [ELSE] [THEN] is an interesting case, but
>not using that inside
>]] ... [[ is probably little hardship.

This kind of discussion begins to become hard from me to follow.

In my original idea's I had
"  : a prefix word or denotation word
"aap" : a denotation recognized by the prefix.
Numbers are recognized by a prefix, being their first digit
or by + and -, and numbers were another example of denotations
and not much more.

Is there a similar distinction in recognizers?
There should be coined a term equivalent to denotation to
combat confusion.

Or to put it differently.
How would a word that recognize 1234H as hex be called?
(A recognizer, I guess ).
Now what would be contained in the name field of that word?
Can that be arbitrary?
How would 1234H in a text be called? A number is not the answer
I'm after. It is a number by accident.

Personally I would say the rule:
You can't postpone a number or other denotation, like a constant
string.
is easy to remember, because we never could do that anyway.
(This wouldn't prevent to use this in non-portable extensions.)

The rule: you can't use [IF] in some very specific
circumstance is much harder on me. Especially where the background
of this evades me.

>
>- anton

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#35616

Anton Ertl wrote:
> When you posted your proposal in the mailing list, I pointed out that
> you can compose all the three behaviours (interpretation, compilation,
> postponing) from three parts:
> 
> 1) a parse-time action
> 2) a LITERAL-like action
> 3) a run-time action
> 
> The composition is as follows:
> 
> interpretation: parse-time run-time
> compilation: parse-time POSTPONE literal-like POSTPONE run-time
> postponing:
>    parse-time POSTPONE literal-like DOUBLE-POSTPONE literal-like
>    DOUBLE-POSTPONE run-time

That's true, but the dot-parser example will produce much worse code than 
with a direct compilation and postpoining method.  My proposal would be to 
have a defining word for recognizers which takes the lit-like and the run-
time action, and produces the three (interpretation+compilation+postpoining) 
xts, and then is easier to use.

Forthermore, and that's important: This approach does not work on the 
dictionary lookup, and only works with the dot parser if the parsed method 
has default compilation semantics (which is not warranted at all).

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/
net2o ID: kQusJzA;7*?t=uy@X}1GWr!+0qqp_Cn176t4(dQ*

[toc] | [prev] | [next] | [standalone]

#35479

> Anton Ertl likes postponing numbers. My design of denotations was more
> like "numbers are smart anyway" (handled differently during compilation)
> "nobody ever postpones numbers" "a dea (xt) is an address (number) that
> is the same while compiling or interpreting, so it is a number"
> I really based the denotation on the premise that they could not, should
> not and would not be postponed, ever.

On smart numbers: If you do constant folding, e.g. replacing : 15+ ( n -- n ) 3 5 * + ; internally with the result, then numbers have no state check at all, they are simply left on stack both in compilation and interpret mode.

[toc] | [prev] | [next] | [standalone]

#35428

Andrew Haley wrote:
> 2.  This proposal looks rather complicated.  Can it not be made simpler
>     without loss of functionality?  For example, R:FAIL does not seem
>     to do anything useful.  It's very cute to make it the failure
>     action but there's an awful lot of DUP R:FAIL <> IF executed in
>     the inner loop of DO-RECOGNIZER.  Should the token used for
>     notifying a failure not simply be zero?

If I look at my implementation, there is one DUP R:FAIL <> IF check in the 
inner loop of DO-RECOGNIZER.  There are three users of DO-RECOGNIZER, the 
interpreter, the compiler, and the postpone loop.  Having a 0 would simplify 
the one check in the inner loop to DUP IF and complicate the three users of 
DO-RECOGNIZER, which then all need special-casing the fail case.

The reference implementation takes the same approach.

I'm fairly convinced that the tradeoff is better the way it is now.  You 
actully can implement R:FAIL as 0 if you like, as it is an opaque type; all 
consumers of recognizers are interal stuff in INTERPRET and POSTPONE, and 
you are, as implementer, free to choose the tradeoffs as you like.

The recognizer stack works like the search order stack, the SET-RECOGNIZERS 
and GET-RECOGNIZERS give you access to the stack in the same way as SET-
ORDER and GET-ORDER does.

The proposal asks one question at the end, how to POSTPONE literals, and the 
obvious answer is that you just append the compilation semantics of those 
literals to the current definition when you execute that postponed literal.

The question whether you allow to put some other recognizer in front of the 
dictionary lookup is up to the user - I would rather not put constraints 
here.  This approach is not limited to literals.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#35434

On Saturday, January 3, 2015 4:37:44 AM UTC-7, Andrew Haley wrote:
> Matthias Trute <matthias.trute@gmail.com> wrote:
> > 
> > some time ago I posted the following text to the Forth200x mailing list. 
> ...
> 1.  Vectored NUMBER or some more general equivalent has been around
>     for a long time.  It's a good thing to have.

There is no word called NUMBER in the ANS-Forth standard, so Andrew is not making any sense as usual. 

I consider the lack of vectored words to be one of the major failings of the ANS-Forth standard. If >NUMBER and various other low-level words had been defined in the standard as being vectored, quite a lot could be done to extend ANS-Forth without breaking ANS-Forth compliance. The concept of vectoring was discussed in "Starting Forth," the book that Elizabeth Rather derives her entire knowledge of Forth from, so she should have known about the concept when she wrote the ANS-Forth standard --- yet >NUMBER is not vectored --- apparently she didn't understand the "hairy" concepts covered in "Starting Forth."

The fact that >NUMBER is not vectored, is a huge problem for me when writing a cross-compiler in ANS-Forth! I work around it, but awkwardly. I have communicated with Ching privately, and he says that this bug in the design of ANS-Forth also prevents him from writing ABC-Forth in pure ANS-Forth. He has asked me to ask Stephen Pelc if >NUMBER can be made a vectored function in VFX, and I said that I would ask --- Pelc routinely and casually introduces words that can't be written in ANS-Forth into VFX to work around the bugs in ANS-Forth design, so I would expect that this would be easy for him to do.

Note for Matthias Trute: I am writing a cross-compiler for the 8-bit AVR at this time (I picked the AVR because it is the smallest processor that my Forth can realistically be compiled for, so if the AVR version works, any other processor should be an easy port). It generates an intermediate file for the Atmel assembler, so it doesn't provide interactive development such as your amForth does. It has lambda functions. Most likely, after I finish the cross-compiler, I will write an on-chip version of my Forth that is compatible --- people can use the on-chip version to develop their program interactively, then use the cross-compiler to generate an optimized production version. If you have any interest in this (writing the on-chip version, for example!), contact me by email. I haven't tried your amForth yet, but I likely will one of these days --- if nothing else, just to pick up ideas that can be incorporated into my own AVR Forth.

[toc] | [prev] | [next] | [standalone]

#35437

hughaguilar96@gmail.com wrote:
> On Saturday, January 3, 2015 4:37:44 AM UTC-7, Andrew Haley wrote:
>> Matthias Trute <matthias.trute@gmail.com> wrote:
>> > 
>> > some time ago I posted the following text to the Forth200x mailing list. 
>> ...
>> 1.  Vectored NUMBER or some more general equivalent has been around
>>     for a long time.  It's a good thing to have.
> 
> There is no word called NUMBER in the ANS-Forth standard, so Andrew
> is not making any sense as usual.

Which part of "vectored NUMBER or some more general equivalent" is
causing you difficulty?  I can explain the words in more detail if you
like.

Andrew.

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.lang.forth

csiph-web