Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #25006 > unrolled thread

Adding NTs or making XTs the only opaque Forth type

Back to article view | Back to comp.lang.forth

Contents

  Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-05 21:11 +0100
    Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-06 07:01 +0000
      Re: Adding NTs or making XTs the only opaque Forth type stephenXXX@mpeforth.com (Stephen Pelc) - 2013-08-06 16:10 +0000
        Re: Adding NTs or making XTs the only opaque Forth type Bernd Paysan <bernd.paysan@gmx.de> - 2013-08-06 22:22 +0200
    Re: Adding NTs or making XTs the only opaque Forth type stephenXXX@mpeforth.com (Stephen Pelc) - 2013-08-06 15:54 +0000
      Re: Adding NTs or making XTs the only opaque Forth type albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-08-06 17:43 +0000
        Re: Adding NTs or making XTs the only opaque Forth type "Elizabeth D. Rather" <erather@forth.com> - 2013-08-06 13:16 -1000
          Re: Adding NTs or making XTs the only opaque Forth type "Elizabeth D. Rather" <erather@forth.com> - 2013-08-06 13:55 -1000
          Re: Adding NTs or making XTs the only opaque Forth type Bernd Paysan <bernd.paysan@gmx.de> - 2013-08-07 02:33 +0200
            Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-07 16:29 +0100
              Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-07 10:54 -0500
                Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-07 17:10 +0100
                  Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-07 12:06 -0500
                    Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-07 18:43 +0100
                      Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-07 12:47 -0500
                        Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-07 19:14 +0100
                          Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-07 15:13 -0500
                            Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-07 23:09 +0100
                        Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-08 13:45 +0000
                          Re: Adding NTs or making XTs the only opaque Forth type "Alex McDonald" <blog@rivadpm.com> - 2013-08-08 15:41 +0100
                          Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-08 10:29 -0500
                            Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-08 16:15 +0000
                              Re: Adding NTs or making XTs the only opaque Forth type Bernd Paysan <bernd.paysan@gmx.de> - 2013-08-08 19:25 +0200
                              Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-08 14:05 -0500
                                Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-12 16:28 +0000
                                  Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-12 12:44 -0500
                                    Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-13 07:08 +0000
                                      Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-13 03:40 -0500
                                        Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-13 09:47 +0000
                                          Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-13 05:55 -0500
                                            Re: Adding NTs or making XTs the only opaque Forth type anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-08-13 12:18 +0000
                                              Re: Adding NTs or making XTs the only opaque Forth type Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-08-13 10:32 -0500

Page 1 of 2  [1] 2  Next page →

#25006 — Adding NTs or making XTs the only opaque Forth type

I've taken the liberty of moving this to a new thread.

on 05/08/2013 18:21:44, Anton Ertl wrote:
> "Alex McDonald" <blog@rivadpm.com> writes:
>>I'll sort out my replies shortly. What appears to be required is a
>>pre-requisite document and refining the spec to make an XT as a the only
>>point of reference.
> 
> There is no consensus on that.
> 

Well, before saying such, it's at least worth summarising the arguments.
This thread has become a little furball-like.

Stephen Pelc and Albert vd Horst directly support the TC's unwritten
intention, expressed by Elizabeth Rather, that an xt as a single opaque
type. Andrew Haley, although having no implementation to support, is also
for the concept.

I'm not clear on Bernd Paysan's or Marcel Hendrix's position, although
both have put forward code that supports some form of name token support.
Albert has support for a name token. Bernd points out that Gforth beyond
0.7.0 now supports finding a header from the xt alone.

Anton Ertl and Peter Fälth state that finding a name anything from an xt
is either impossible (for some systems), and would exclude extant systems
that can't be changed to support the concept; or that an opaque name
token is a better concept than having an xt alone. 
 
Systems that would be adversely affected are Gforth 0.7.0 and lxf/ntf.
All other mentioned systems in this thread seem to support the ability to
get from xt to header fields. I have support for a name token in my
system even though the code space and header space are distinct, and it
can get from an xt to it through a back pointer. A classic Win32Forth has
the header and xt as contiguous areas in the dictionary; the name token
is a fixed offset from the xt and vice-versa.

Stephen points out that, even though it might potentially be an expensive
operation to find the header, the cost is immaterial given that this is a
tools feature where operation cost is not as important.

Unresolved issues include: systems that support dual xts from FIND based
on STATE; :NONAME may lack a header; a required clarification for ALIAS
and SYNONYM; and other systems where the xt is not identifiable with a
named token.

I would welcome clarification on any that I got wrong above.

> Given that this stuff is already used in practice according to the
> responses, maybe the way to go forward is to get a CfV version out at
> some point (so that the spec does not change anymore), and then let
> user demand do its magic. If there are several systems that implement
> your proposal, and programs that use it, maybe even the most
> fundamentalist "nts are evil" advocate will see the value of the
> proposal eventually. After all, they can console themselves with "my
> system does it better".

I suspect that Stephen would take the opposite position and the lack of
consensus would simply move to the CfV with his as a major voice of
dissent. Stephen makes a living out of his product, and changes cost him
real money. I'd appreciate his input on continuing with the proposal for
TRAVERSE-WORDLIST and its associated name token friends.

> 
> However, sometimes this kind of approach works, and often it doesn't.
> 

True. The effort has been worth it though. It has thrown up quite a few
intersting issues and wrinkles I hadn't anticipated. I've learned from
it.

> - anton

[toc] | [next] | [standalone]

#25031

"Alex McDonald" <blog@rivadpm.com> writes:
> the TC's unwritten
>intention, expressed by Elizabeth Rather, that an xt as a single opaque
>type.

It's unclear what that means, however, if there was such an intent at
all.  A few interpretations that come to mind:

1) They did not like the words for dealing with headers in systems
like fig-Forth: There you have one header with name field, link field
code field, and parameter field, and words for going from one field to
the next.  To go from the name field to the parameter field, you had
to do something like "1 TRAVERSE LFA> >PFA".  I think it's very likely
that this is what Elizabeth Rather is remembering, and I think we can
find consensus that we do not want this kind of design.

2) They did not standardize stuff like NT>STRING, because that would
have required putting an NT in the standard, and they (or at least
some of them) did not want to go there.

3) Each named word has one xt and that represents everything about it.
If that was the unwritten intent, it directly contradicts the written
intent of supporting different implementation approaches, including
ones where interpretation semantics and compilation semantics are
associated with different xts (which is reflected in the normative
specification of FIND and maybe other places).  And it is incompatible
with the specification of COMPILE, (one can destandardize the xts of
S" and TO to work around that, but that's incompatible with this
version of the claimed intent).

4) A named word can have several parts, and each one of them is
represented by an xt (but there can be several xts associated with a
named word).

>Systems that would be adversely affected are Gforth 0.7.0 and lxf/ntf.

The fact that there are several systems that cannot satisfy Stephen
Pelc's nt=xt requirement without significant changes deep in their
bowels makes me suspect that there are more.

>Stephen points out that, even though it might potentially be an expensive
>operation to find the header, the cost is immaterial given that this is a
>tools feature where operation cost is not as important.

I have used access to the name of the word in application contexts.  I
would expect that NT>STRING would be used that way, too.  So
perfomance of NT>STRING is relevant.

>> Given that this stuff is already used in practice according to the
>> responses, maybe the way to go forward is to get a CfV version out at
>> some point (so that the spec does not change anymore), and then let
>> user demand do its magic. If there are several systems that implement
>> your proposal, and programs that use it, maybe even the most
>> fundamentalist "nts are evil" advocate will see the value of the
>> proposal eventually. After all, they can console themselves with "my
>> system does it better".
>
>I suspect that Stephen would take the opposite position and the lack of
>consensus would simply move to the CfV with his as a major voice of
>dissent. Stephen makes a living out of his product, and changes cost him
>real money.

But allowing nt!=xt would not require a change in his product, as
nt=xt would be allowed.  By contrast, requiring nt=xt would require a
changes in other products, and he has said so in this discussion. I
guess he is so nonchalant about that because it's not his product that
would be affected.  If it was his product he would use
"disenfrenchise" in every posting.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2013: http://www.euroforth.org/ef13/

[toc] | [prev] | [next] | [standalone]

#25035

On Tue, 06 Aug 2013 07:01:59 GMT, anton@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>"Alex McDonald" <blog@rivadpm.com> writes:
>> the TC's unwritten
>>intention, expressed by Elizabeth Rather, that an xt as a single opaque
>>type.
>
>It's unclear what that means, however, if there was such an intent at
>all.  A few interpretations that come to mind:
>
>1) They did not like the words for dealing with headers in systems
>like fig-Forth: There you have one header with name field, link field
>code field, and parameter field, and words for going from one field to
>the next.  To go from the name field to the parameter field, you had
>to do something like "1 TRAVERSE LFA> >PFA".  I think it's very likely
>that this is what Elizabeth Rather is remembering, and I think we can
>find consensus that we do not want this kind of design.

That situation was nasty and tended to impose implementation styles.

>2) They did not standardize stuff like NT>STRING, because that would
>have required putting an NT in the standard, and they (or at least
>some of them) did not want to go there.

I do not recollect anyone proposing such a thing. At the time, I
believe that the TC members were content for such words to be system
dependent.

>3) Each named word has one xt and that represents everything about it.
>If that was the unwritten intent, it directly contradicts the written
>intent of supporting different implementation approaches,

Please justify such an assertion. At present, you repeat this as if
it was obvious.

> including
>ones where interpretation semantics and compilation semantics are
>associated with different xts (which is reflected in the normative
>specification of FIND and maybe other places).  And it is incompatible
>with the specification of COMPILE, (one can destandardize the xts of
>S" and TO to work around that, but that's incompatible with this
>version of the claimed intent).

We could have a "heated debate" about gforth's COMPILE, and S" but
this is not the place. Classifying the separate interpretation and
compilation behaviours as xts is what leads to the confusion. If
we call them ixts and cxts and treat them as internal factors, then
life becomes clearer.

>I have used access to the name of the word in application contexts.  I
>would expect that NT>STRING would be used that way, too.  So
>perfomance of NT>STRING is relevant.

Please could you give examples where the performance is important.

>But allowing nt!=xt would not require a change in his product, as
>nt=xt would be allowed.  By contrast, requiring nt=xt would require a
>changes in other products, and he has said so in this discussion. I
>guess he is so nonchalant about that because it's not his product that
>would be affected.

MPE has changed VFX as a result of your statements about standards
compliance. That cost real money. Please have the grace to at least
consider my position and realise that there may be faults in gforth.

Stephen


-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#25039

Stephen Pelc wrote:

>>But allowing nt!=xt would not require a change in his product, as
>>nt=xt would be allowed.  By contrast, requiring nt=xt would require a
>>changes in other products, and he has said so in this discussion. I
>>guess he is so nonchalant about that because it's not his product that
>>would be affected.
> 
> MPE has changed VFX as a result of your statements about standards
> compliance. That cost real money. Please have the grace to at least
> consider my position and realise that there may be faults in gforth.

Actually, there are two people behind Gforth, and Anton is currently not so 
much involved, as his job is demanding too much time.  The other major 
contributor behind Gforth (that's me) did change a lot of things here, and 
has about the same troubles with arguments towards Anton as you have.

In Gforth git-head, nt=xt, name>string is a cheap O(1) operation, and 
COMPILE, is doing the same stuff as in VFX: execute the compile,-field in 
the header, which has been set by SET-COMPILER.

So actually there is no stringent requirement for Anton to admit things, 
because these changes went into Gforth anyways ;-).  I went even further 
than you, because in VFX, there still is a distinct name field, in Gforth 
git-head (to become 0.8) there isn't.

The only thing I added due to pressure from Anton was to allow dual-xt 
words; this also is backward compatible to Gforth 0.7, and therefore a good 
idea.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25034

On Mon, 5 Aug 2013 21:11:28 +0100, "Alex McDonald" <blog@rivadpm.com>
wrote:

>Systems that would be adversely affected are Gforth 0.7.0 and lxf/ntf.
>All other mentioned systems in this thread seem to support the ability to
>get from xt to header fields. I have support for a name token in my
>system even though the code space and header space are distinct, and it
>can get from an xt to it through a back pointer. A classic Win32Forth has
>the header and xt as contiguous areas in the dictionary; the name token
>is a fixed offset from the xt and vice-versa.

Ignoring :NONAME words, all systems keep names somewhere. Many systems
use similar linked list structures for names and header information.
At present, the navigation words are all implementation dependent.
I really do not want to standardise the raft of dictionary navigation
words that was in FIG-Forth.

Without careful consideration of what is required, the portability 
promised by TRAVERSE-WORDLIST is illusory. That portability requires
the xt fed to TRAVERSE-WORDLIST to come from a portable word. It's
that set of portable words that bothers me.

>> maybe even the most
>> fundamentalist "nts are evil" advocate will see the value of the
>> proposal eventually. After all, they can console themselves with "my
>> system does it better".
>
>I suspect that Stephen would take the opposite position and the lack of
>consensus would simply move to the CfV with his as a major voice of
>dissent. Stephen makes a living out of his product, and changes cost him
>real money. I'd appreciate his input on continuing with the proposal for
>TRAVERSE-WORDLIST and its associated name token friends.

Adding NTs to VFX involves a little bit of renaming and changing some
documentation. I also believe that with a bit of hacking, both words
  XT>NT  ( xt -- nt )
  NT>XT  ( nt -- xt )
can be implemented on almost any system providing that "brute force
and ignorance" solutions are acceptable.

I don't believe that "nts are evil", I just do not believe that they
have been justified yet. VFX has name fields. I realise that one *can*
write tools with TRAVERSE-WORDLIST, but there have been no compelling
examples of *what* these tools might be. Without examples, an nt
remains what MPE calls a WIBNI - "Wouldn't It Be Nice If". WIBNIs
are not enough to justify standardisation.

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#25036

In article <5201151b.2293282027@news.demon.co.uk>,
Stephen Pelc <stephenXXX@mpeforth.com> wrote:
>
>I don't believe that "nts are evil", I just do not believe that they
>have been justified yet. VFX has name fields. I realise that one *can*
>write tools with TRAVERSE-WORDLIST, but there have been no compelling
>examples of *what* these tools might be. Without examples, an nt
>remains what MPE calls a WIBNI - "Wouldn't It Be Nice If". WIBNIs
>are not enough to justify standardisation.

Case in point. ciforth has TRAVERSE-WORDLIST which is called FOR-WORDS.
It was never used in a library. It appeared not useful in ciasdis
where the collection of assembler words is a wordlist that is traversed.

I used it in
: WORDS  'ID. CONTEXT @ FOR-WORDS ; ( or some such).

Then I decided that I want the words in the order most recent last.
Now WORDS did not become large, for a useful modification.
Gone was the last place where TRAVERSE-WORDLIST is used.

I join you in doubting that this kind of words is useful.
If it opens a can of worms around nt's and xt's, it is probably
not worth it.

>
>Stephen
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#25042

On 8/6/13 7:43 AM, Albert van der Horst wrote:
> In article <5201151b.2293282027@news.demon.co.uk>,
> Stephen Pelc <stephenXXX@mpeforth.com> wrote:
>>
>> I don't believe that "nts are evil", I just do not believe that they
>> have been justified yet. VFX has name fields. I realise that one *can*
>> write tools with TRAVERSE-WORDLIST, but there have been no compelling
>> examples of *what* these tools might be. Without examples, an nt
>> remains what MPE calls a WIBNI - "Wouldn't It Be Nice If". WIBNIs
>> are not enough to justify standardisation.
>
> Case in point. ciforth has TRAVERSE-WORDLIST which is called FOR-WORDS.
> It was never used in a library. It appeared not useful in ciasdis
> where the collection of assembler words is a wordlist that is traversed.
>
> I used it in
> : WORDS  'ID. CONTEXT @ FOR-WORDS ; ( or some such).
>
> Then I decided that I want the words in the order most recent last.
> Now WORDS did not become large, for a useful modification.
> Gone was the last place where TRAVERSE-WORDLIST is used.
>
> I join you in doubting that this kind of words is useful.
> If it opens a can of worms around nt's and xt's, it is probably
> not worth it.

The problem is, as soon as you start encouraging folks to do things with 
names and other header info, you find yourself either constraining 
header organization or adding a plethora of specialized tools that are 
rarely used.

The real intent of the ANS Forth TC, as I recall, was to provide user 
control of data space, but keep code/dictionary space opaque, hence the 
xt. Implementers can and should provide extensive programming and/or 
introspection tools; it's an important "quality of implementation" 
issue. And users of well-documented implementations can add tools to 
their favorite systems, but there is no guarantee they'll be portable.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#25044

On 8/6/13 1:16 PM, Elizabeth D. Rather wrote:
> On 8/6/13 7:43 AM, Albert van der Horst wrote:
...
>>
>> I join you in doubting that this kind of words is useful.
>> If it opens a can of worms around nt's and xt's, it is probably
>> not worth it.
>
> The problem is, as soon as you start encouraging folks to do things with
> names and other header info, you find yourself either constraining
> header organization or adding a plethora of specialized tools that are
> rarely used.
>
> The real intent of the ANS Forth TC, as I recall, was to provide user
> control of data space, but keep code/dictionary space opaque, hence the
> xt. Implementers can and should provide extensive programming and/or
> introspection tools; it's an important "quality of implementation"
> issue. And users of well-documented implementations can add tools to
> their favorite systems, but there is no guarantee they'll be portable.

Further in this vein: Consider the definition of "data space" from Forth94:

"data space: The logical area of the dictionary that *can be accessed*.
  data field: The data space associated with a word defined via CREATE."
(emphasis mine)

Other spaces are defined (dictionary, name space, code space, etc.), so 
the document can can talk about them, but only data space (and, more 
specifically, a "data field") can be *accessed* portably. To quote A.2.1:

"data field
In earlier standards, data fields were known as “parameter fields”.
On subroutine threaded Forth systems, everything is object code. There 
are no traditional code or data fields. Only a word defined by CREATE or 
by a word that calls CREATE has a data field. Only a data field
defined via CREATE can be manipulated portably."

Further, note that Section 3.1.3.3 Addresses, starts this way:

"3.1.3.3 Addresses
An address identifies a location in *data space* with a size of one 
address unit, which a program may fetch from or store into except for 
the restrictions established in this Standard."

That is, you can neither fetch from or store into an address that is not 
in *data space* portably.

Section 3.3.1 includes the warning that "The relationship between [name 
space/code space] and data space is implementation dependent."

Now, traditional (and some modern) Forths integrate name space, code 
space, and data space. They're entitled to do that, which is why "data 
space" is defined distinctly from "data field".

And you're welcome to write code that accesses any of these spaces 
*providing* you accept the fact that this code has an implementation 
dependency on a system that permits such access and documents what you 
will find at those addresses that are not in standard "data fields". 
Such code is not guaranteed to be portable across all Standard Systems.

So, the problem with an nt is that you don't know what it points to or 
what you can do with whatever it points ton, unless you start writing a 
whole bunch of restrictions and things which the Forth94 TC very 
emphatically did not want to do.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700  n
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#25046

Elizabeth D. Rather wrote:
> The real intent of the ANS Forth TC, as I recall, was to provide user
> control of data space, but keep code/dictionary space opaque, hence the
> xt. Implementers can and should provide extensive programming and/or
> introspection tools; it's an important "quality of implementation"
> issue. And users of well-documented implementations can add tools to
> their favorite systems, but there is no guarantee they'll be portable.

Well, the point here is: If a number of implementers provide such 
introspection tools (and yes, they do!), we can look at common practice, and 
then standardize them if we find some.

Those tools are tools, they quite likely will not be used at all in an 
application.  How often do you use .S in an application?  WORDS or SEE?  SEE 
is pretty expensive to implement, compared to adding NAME>INT, NAME>COMP, 
and NAME>STRING.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25054

on 07/08/2013 01:33:01, Bernd Paysan wrote:
> Elizabeth D. Rather wrote:
>> The real intent of the ANS Forth TC, as I recall, was to provide user
>> control of data space, but keep code/dictionary space opaque, hence the
>> xt. Implementers can and should provide extensive programming and/or
>> introspection tools; it's an important "quality of implementation"
>> issue. And users of well-documented implementations can add tools to
>> their favorite systems, but there is no guarantee they'll be portable.
> 
> Well, the point here is: If a number of implementers provide such
> introspection tools (and yes, they do!), we can look at common
> practice, and then standardize them if we find some.
> 
> Those tools are tools, they quite likely will not be used at all in an
> application. How often do you use .S in an application? WORDS or SEE?
> SEE is pretty expensive to implement, compared to adding NAME>INT,
> NAME>COMP, and NAME>STRING.
> 

Some use cases for TRAVERSE-WORDLIST would appear to be appropriate (and
to address Stephen's WIBNI observation).

1. Obvious: WORDS. Just about every Forth I've come across has such a
word.

2. Some implementations of WORDS are a little less useful than others. A
filter is useful; so the ability to display words that (for instance)
that start with "STR*" or contain "*STR*" or end in "*STR".

3. Finding duplicate names across wordlists.

4. Cross referencing. Given a word, show all the words that use it. This
is very much an internal operation and implementation specific. ITC and
DTC are relatively simple; with STC and native code it may be impossible.
But the specification doesn't go that far.

5. Wordlist merging, extraction and copying. 

6. Assisting in the generation of documentation. My Forth carries enough
information in the header to tell you:

a. the name
b. the source file and line number where it's defined
c. the type of the word (VALUE, FVALUE, CONSTANT, : and so on)
d. the stack effects (for some, not all, words)
e. the execution and compilation xts and their size; immediacy
f. an ITC equivalent stream for SEE (it's an STC/NCC)

Now, these are not arguments for an extended set of words to manage
entries, but examples of what can be done. The intention of
TRAVERSE-WORDLIST is not to make such introspection tools portable, but
to simplify the creation of such tools by providing a *portable traversal
of a wordlist*.

As a user of Forth system X, I may be interested in creating a set of
dictionary tools. I can CREATE and FIND in a standards-compliant way. But
without carnal knowledge of the dictionary structures, I can't implement
even the simplest introspection such as counting the number of words on
such a Forth.

It's also important to note that some Forths may have, and mine certainly
does, several different wordlist implementations[1]. The management
mechanisms are very different for each style of dictionary. To get around
the internal differences, each provides 3 uniform and simple interface
mechanisms. (a) a word that implements TRAVERSE-WORDLIST for this
dictionary type (b) a word that implements CREATE and : that defines a
named header and (c) a word that implements SEARCH-WORLDIST that returns
an xt.

(Of course, once I have an entry though TRAVERSE-WORDLIST, the rest is up
to the documented parts of such an entry for this specific
implementation. The same is true of an xt from FIND; I can only portably
EXECUTE or COMPILE, it. Anything else is system specific.) 

What's missing in the current standard is that standardised mechanism for
traversal.

[1] Hash chained on collision (the only one supported by the bare
kernel): hash with rehash: ternary tree: SQLLite database.  

[toc] | [prev] | [next] | [standalone]

#25055

Alex McDonald <blog@rivadpm.com> wrote:
> on 07/08/2013 01:33:01, Bernd Paysan wrote:
>> Elizabeth D. Rather wrote:
>>> The real intent of the ANS Forth TC, as I recall, was to provide user
>>> control of data space, but keep code/dictionary space opaque, hence the
>>> xt. Implementers can and should provide extensive programming and/or
>>> introspection tools; it's an important "quality of implementation"
>>> issue. And users of well-documented implementations can add tools to
>>> their favorite systems, but there is no guarantee they'll be portable.
>> 
>> Well, the point here is: If a number of implementers provide such
>> introspection tools (and yes, they do!), we can look at common
>> practice, and then standardize them if we find some.
>> 
>> Those tools are tools, they quite likely will not be used at all in an
>> application. How often do you use .S in an application? WORDS or SEE?
>> SEE is pretty expensive to implement, compared to adding NAME>INT,
>> NAME>COMP, and NAME>STRING.
>> 
> 
> Some use cases for TRAVERSE-WORDLIST would appear to be appropriate (and
> to address Stephen's WIBNI observation).
> 
> 1. Obvious: WORDS. Just about every Forth I've come across has such a
> word.

You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
gives no guarantee as to any specific order of node visits to each
word through the invoked xt with one exception."  A good WORDS has to
do better than that.

> 5. Wordlist merging, extraction and copying. 

Is it possible to do these things in a portable way?

Andrew.

[toc] | [prev] | [next] | [standalone]

#25056

on 07/08/2013 16:54:54, Andrew Haley wrote:
> Alex McDonald <blog@rivadpm.com> wrote:
>> on 07/08/2013 01:33:01, Bernd Paysan wrote:
>>> Elizabeth D. Rather wrote:
>>>> The real intent of the ANS Forth TC, as I recall, was to provide user
>>>> control of data space, but keep code/dictionary space opaque, hence the
>>>> xt. Implementers can and should provide extensive programming and/or
>>>> introspection tools; it's an important "quality of implementation"
>>>> issue. And users of well-documented implementations can add tools to
>>>> their favorite systems, but there is no guarantee they'll be portable.
>>>
>>> Well, the point here is: If a number of implementers provide such
>>> introspection tools (and yes, they do!), we can look at common
>>> practice, and then standardize them if we find some.
>>>
>>> Those tools are tools, they quite likely will not be used at all in an
>>> application. How often do you use .S in an application? WORDS or SEE?
>>> SEE is pretty expensive to implement, compared to adding NAME>INT,
>>> NAME>COMP, and NAME>STRING.
>>>
>>
>> Some use cases for TRAVERSE-WORDLIST would appear to be appropriate (and
>> to address Stephen's WIBNI observation).
>>
>> 1. Obvious: WORDS. Just about every Forth I've come across has such a
>> word.
> 
> You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
> gives no guarantee as to any specific order of node visits to each
> word through the invoked xt with one exception."  A good WORDS has to
> do better than that.

That's up to the implementation, and how TRAVERSE-WORDLIST returns its
results. For instance, my WORDS uses TRAVERSE-WORDLIST which returns the
nodes in definition order newest to oldest. Your TRAVERSE-WORDLIST may
return in alphabetic sequence. You might want to sort the results of
TRAVERSE-WORDLIST before displaying them. Or you may not use
TRAVERSE-WORDLIST for WORDS at all, since you might want WORDS to do
something quite different. 

> 
>> 5. Wordlist merging, extraction and copying.
> 
> Is it possible to do these things in a portable way?

Probably not, but that's not the point. They're capabilities made easier
by having wordlist traversal. 


> 
> Andrew.

[toc] | [prev] | [next] | [standalone]

#25057

Alex McDonald <blog@rivadpm.com> wrote:
> on 07/08/2013 16:54:54, Andrew Haley wrote:
>> Alex McDonald <blog@rivadpm.com> wrote:
>>> 1. Obvious: WORDS. Just about every Forth I've come across has such a
>>> word.
>> 
>> You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
>> gives no guarantee as to any specific order of node visits to each
>> word through the invoked xt with one exception."  A good WORDS has to
>> do better than that.
> 
> That's up to the implementation, and how TRAVERSE-WORDLIST returns its
> results.

Right, so you know what I'm going to say: you might as well not use
TRAVERSE-WORDLIST, then, just do whataver you used to do.  Which is
probably just following links.

> For instance, my WORDS uses TRAVERSE-WORDLIST which returns the
> nodes in definition order newest to oldest. Your TRAVERSE-WORDLIST may
> return in alphabetic sequence. You might want to sort the results of
> TRAVERSE-WORDLIST before displaying them. Or you may not use
> TRAVERSE-WORDLIST for WORDS at all, since you might want WORDS to do
> something quite different. 

Indeed.

Andrew.

[toc] | [prev] | [next] | [standalone]

#25058

on 07/08/2013 18:06:28, Andrew Haley wrote:
> Alex McDonald <blog@rivadpm.com> wrote:
>> on 07/08/2013 16:54:54, Andrew Haley wrote:
>>> Alex McDonald <blog@rivadpm.com> wrote:
>>>> 1. Obvious: WORDS. Just about every Forth I've come across has such a
>>>> word.
>>>
>>> You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
>>> gives no guarantee as to any specific order of node visits to each
>>> word through the invoked xt with one exception."  A good WORDS has to
>>> do better than that.
>>
>> That's up to the implementation, and how TRAVERSE-WORDLIST returns its
>> results.
> 
> Right, so you know what I'm going to say: you might as well not use
> TRAVERSE-WORDLIST, then, just do whataver you used to do.  Which is
> probably just following links.

Right, and you know what I'm going to say; there are many Forths that
have a similar word to TRAVERSE-WORDLIST as a factor of WORDS that
(probably, but not always) just follows links. 

> 
>> For instance, my WORDS uses TRAVERSE-WORDLIST which returns the
>> nodes in definition order newest to oldest. Your TRAVERSE-WORDLIST may
>> return in alphabetic sequence. You might want to sort the results of
>> TRAVERSE-WORDLIST before displaying them. Or you may not use
>> TRAVERSE-WORDLIST for WORDS at all, since you might want WORDS to do
>> something quite different.
> 
> Indeed.

Such as? I should have asked first time round.

> 
> Andrew.

[toc] | [prev] | [next] | [standalone]

#25059

Alex McDonald <blog@rivadpm.com> wrote:
> on 07/08/2013 18:06:28, Andrew Haley wrote:
>> Alex McDonald <blog@rivadpm.com> wrote:
>>> on 07/08/2013 16:54:54, Andrew Haley wrote:
>>>> Alex McDonald <blog@rivadpm.com> wrote:
>>>>> 1. Obvious: WORDS. Just about every Forth I've come across has such a
>>>>> word.
>>>>
>>>> You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
>>>> gives no guarantee as to any specific order of node visits to each
>>>> word through the invoked xt with one exception."  A good WORDS has to
>>>> do better than that.
>>>
>>> That's up to the implementation, and how TRAVERSE-WORDLIST returns its
>>> results.
>> 
>> Right, so you know what I'm going to say: you might as well not use
>> TRAVERSE-WORDLIST, then, just do whatever you used to do.  Which is
>> probably just following links.
> 
> Right, and you know what I'm going to say; there are many Forths
> that have a similar word to TRAVERSE-WORDLIST as a factor of WORDS
> that (probably, but not always) just follows links.

You reckon?  Similar to TRAVERSE-WORDLIST, not something that just
follows links?  Why mess about with XTs if you know it's just a linked
list?  Surely you'd just have a word that takes the address of a
header and returns the previous header.

>>> For instance, my WORDS uses TRAVERSE-WORDLIST which returns the
>>> nodes in definition order newest to oldest. Your TRAVERSE-WORDLIST may
>>> return in alphabetic sequence. You might want to sort the results of
>>> TRAVERSE-WORDLIST before displaying them. Or you may not use
>>> TRAVERSE-WORDLIST for WORDS at all, since you might want WORDS to do
>>> something quite different.
>> 
>> Indeed.
> 
> Such as? I should have asked first time round.

For example, you might have a dictionary with multiple hash chains,
but you really want WORDS to display in create order.  One of my
favourite Forths does exactly this.  Not that I ever use WORDS, mind
you.

Andrew.

[toc] | [prev] | [next] | [standalone]

#25060

on 07/08/2013 18:47:57, Andrew Haley wrote:
> Alex McDonald <blog@rivadpm.com> wrote:
>> on 07/08/2013 18:06:28, Andrew Haley wrote:
>>> Alex McDonald <blog@rivadpm.com> wrote:
>>>> on 07/08/2013 16:54:54, Andrew Haley wrote:
>>>>> Alex McDonald <blog@rivadpm.com> wrote:
>>>>>> 1. Obvious: WORDS. Just about every Forth I've come across has such a
>>>>>> word.
>>>>>
>>>>> You can't use TRAVERSE-WORDLIST for WORDS because "TRAVERSE-WORDLIST
>>>>> gives no guarantee as to any specific order of node visits to each
>>>>> word through the invoked xt with one exception."  A good WORDS has to
>>>>> do better than that.
>>>>
>>>> That's up to the implementation, and how TRAVERSE-WORDLIST returns its
>>>> results.
>>>
>>> Right, so you know what I'm going to say: you might as well not use
>>> TRAVERSE-WORDLIST, then, just do whatever you used to do.  Which is
>>> probably just following links.
>>
>> Right, and you know what I'm going to say; there are many Forths
>> that have a similar word to TRAVERSE-WORDLIST as a factor of WORDS
>> that (probably, but not always) just follows links.
> 
> You reckon?  Similar to TRAVERSE-WORDLIST, not something that just
> follows links? 

I noted some in the RfD.

Wordlist traversal functionality is available through very similar
words in Win32Forth (VOC-ITERATE), VFX (WalkWordList), iForth
(doWORDS), lxf/ntf (WALK-WORDLIST), and ciforth (FOR-WORDS). 

(Albert did note that he's removed it from ciforth and replaced it with
some other construct; he didn't say what.)

TRAVERSE-WORDLIST follows links -- if that's the structure of the
wordlist.

> Why mess about with XTs if you know it's just a linked
> list? Surely you'd just have a word that takes the address of a header
> and returns the previous header.

Sorry, you lost me here.

> 
>>>> For instance, my WORDS uses TRAVERSE-WORDLIST which returns the
>>>> nodes in definition order newest to oldest. Your TRAVERSE-WORDLIST may
>>>> return in alphabetic sequence. You might want to sort the results of
>>>> TRAVERSE-WORDLIST before displaying them. Or you may not use
>>>> TRAVERSE-WORDLIST for WORDS at all, since you might want WORDS to do
>>>> something quite different.
>>>
>>> Indeed.
>>
>> Such as? I should have asked first time round.
> 
> For example, you might have a dictionary with multiple hash chains,
> but you really want WORDS to display in create order.  One of my
> favourite Forths does exactly this.  

Which is exactly the structure and exactly what my Forth
(TRAVERSE-WORDLIST) and Win32Forth v6 (VOC-ITERATE) provide. (Not
surprising that both use the same technique, really, as I authored the
changes to that part of Win32Forth.)

On the other hand, someone might want a sorted list by name. They'll use
TRAVERSE- WORDLIST (or VOC-ITERATE) since thre's no way they can tell how
the wordlist is implemented without inspecting the source implementation
of that wordlist.

> Not that I ever use WORDS, mind
> you.

You won't need to use TRAVERSE-WORDLIST either ;-) 

> Andrew.

[toc] | [prev] | [next] | [standalone]

#25064

Alex McDonald <blog@rivadpm.com> wrote:
> on 07/08/2013 18:47:57, Andrew Haley wrote:
> 
>> Not that I ever use WORDS, mind you.
> 
> You won't need to use TRAVERSE-WORDLIST either ;-) 

Probably not, and that is a true dilemma for a Forth minimalist like
me.  On the one hand I don't want God-only-knows what kind of ill-
thought-out crud in the standard, but on the other hand I really don't
want to be the bad guy who stands in the way of things that other
people want.

I guess I'll just have to talk to Stephen at EuroForth and he'll let
me know what the right thing to think is.  ;-)

Andrew.

[toc] | [prev] | [next] | [standalone]

#25066

on 07/08/2013 21:13:49, Andrew Haley wrote:
> Alex McDonald <blog@rivadpm.com> wrote:
>> on 07/08/2013 18:47:57, Andrew Haley wrote:
>>
>>> Not that I ever use WORDS, mind you.
>>
>> You won't need to use TRAVERSE-WORDLIST either ;-)
> 
> Probably not, and that is a true dilemma for a Forth minimalist like
> me.  On the one hand I don't want God-only-knows what kind of ill-
> thought-out crud in the standard, 

Too late. Much of the crud imho was in place before this proposal. 

> but on the other hand I really don't
> want to be the bad guy who stands in the way of things that other
> people want.

Your observations are appreciated. 

> 
> I guess I'll just have to talk to Stephen at EuroForth and he'll let
> me know what the right thing to think is.  ;-)

I'll address my comments to the organ grinder in that case. Winky.

> 
> Andrew.

[toc] | [prev] | [next] | [standalone]

#25070

Andrew Haley <andrew29@littlepinkcloud.invalid> writes:
> Why mess about with XTs if you know it's just a linked
>list?

Does every Forth have a plain linked list in addition to whatever
efficient data structure it has for name lookup?  Even if that is the
case (which I doubt), do we want to put such a requirement in the
standard?

>Surely you'd just have a word that takes the address of a
>header and returns the previous header.

"Previous"?  I assume you mean "next".

That may not be implementable in some systems without deep internal
changes.

E.g., from what I read many Forth systems use, for each wordlist, a
hash table with external chaining.  Proceeding to the next entry
within a chain may be viable (in Gforth it's not, but then Gforth has
the plain linked list in addition), but once you are at the end of a
chain, how do you find the next header?

An interface like you suggest "impinges on the implementation", as
Stephen Pelc would say.  TRAVERSE-WORDLIST is designed to be abstract
enough that it does not impinge any more than WORDS does.

>For example, you might have a dictionary with multiple hash chains,
>but you really want WORDS to display in create order.  One of my
>favourite Forths does exactly this.

Gforth does that by having a plain linked list in addition to the hash
table.  But I don't think that's a universal approach.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2013: http://www.euroforth.org/ef13/

[toc] | [prev] | [next] | [standalone]

#25071

on 08/08/2013 14:45:36,  wrote:
> Andrew Haley <andrew29@littlepinkcloud.invalid> writes:
>> Why mess about with XTs if you know it's just a linked
>>list?
> 
> Does every Forth have a plain linked list in addition to whatever
> efficient data structure it has for name lookup?  Even if that is the
> case (which I doubt), do we want to put such a requirement in the
> standard?

It's already there, if you read the spec with language lawyer goggles.
See below.

> 
>>Surely you'd just have a word that takes the address of a
>>header and returns the previous header.
> 
> "Previous"?  I assume you mean "next".

Ah, now that clarifies it.

> 
> That may not be implementable in some systems without deep internal
> changes.
> 
> E.g., from what I read many Forth systems use, for each wordlist, a
> hash table with external chaining.  Proceeding to the next entry
> within a chain may be viable (in Gforth it's not, but then Gforth has
> the plain linked list in addition), but once you are at the end of a
> chain, how do you find the next header?
> 
> An interface like you suggest "impinges on the implementation", as
> Stephen Pelc would say.  TRAVERSE-WORDLIST is designed to be abstract
> enough that it does not impinge any more than WORDS does.

In particular, it frees the implementor to have any number of different
types of wordlist to suit particular circumstances.  

> 
>>For example, you might have a dictionary with multiple hash chains,
>>but you really want WORDS to display in create order.  One of my
>>favourite Forths does exactly this.
> 
> Gforth does that by having a plain linked list in addition to the hash
> table. But I don't think that's a universal approach.

I've experimented with ternary trees and SQL databases for wordlists; and
support for TRAVERSE-WORDLIST, CREATE : and SEARCH-WORDLIST (3
interfaces, of which one is missing) makes it easy to plug them in as
full wordlists.

The only requirement is that TRAVERSE-WORDLIST visits nodes once in no
specified order. The default in my Forth returns nodes by definition time
newest to oldest, and the ternary tree returns an alpha ascending. It's
not hard to generate and sort a list using TRAVERSE-WORDLIST should some
other order be required. What is hard is generating the list in the first
place given the diversity of potential wordlist implementations.

The definition for a wordlist is also contradictory; it refers to it as 
"implementation-dependant" and as "a list". That might be Andrew's
justification for some form of link from header to header, though it
takes a pretty tight readng of it to assert this. I suspect "collection"
is what the TC meant, and list was a convenient (if sloppy) shorthand. 

(If it's truly implementation-dependant it also makes a wordlist value
opaque; that's one more to add to the opaque type list along with xt and
a potential nt.)

Glossary:
word list: A list of associated Forth definition names that may be
examined during a dictionary search.

16.3.1 Data types
Word list identifiers are implementation-dependent single-cell values
that identify word lists.

Append table 16.1 to table 3.1.
Table 16.1: Data types
Symbol  Data type                Size on stack
wid     word list identifiers    1 cell

> 
> - anton

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.forth

csiph-web