Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #1007 > unrolled thread

Implementation of execute-parsing-file

Back to article view | Back to comp.lang.forth

Contents

  Implementation of execute-parsing-file Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-04-05 10:50 +0100
    Re: Implementation of execute-parsing-file David Kuehling <dvdkhlng@gmx.de> - 2011-04-05 11:58 +0200
      Re: Implementation of execute-parsing-file Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-04-05 12:26 +0100
        Re: Implementation of execute-parsing-file Josh Grams <josh@qualdan.com> - 2011-04-05 22:35 +0000
          Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-05 18:31 -0700
            Re: Implementation of execute-parsing-file Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-04-06 10:15 +0100
              Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-06 09:04 -0700
                Re: Implementation of execute-parsing-file Josh Grams <josh@qualdan.com> - 2011-04-07 10:19 +0000
                  Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-07 07:54 -0700
                    Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-07 16:32 +0000
                      Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-08 18:22 -0700
                        Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-09 13:10 +0000
                          Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-09 12:43 -0700
                            Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-10 13:22 +0000
                              Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-10 09:44 -0700
                                Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-11 14:22 +0000
                                  Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-11 08:37 -0700
                              Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-11 09:05 -0700
                                Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-13 16:50 +0000
                                  Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-13 12:37 -0700
                          Re: Implementation of execute-parsing-file Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-04-10 02:55 -0500
                            Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-10 13:11 +0000
                              Re: Implementation of execute-parsing-file Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-04-10 09:27 -0500
                                Re: Implementation of execute-parsing-file anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-04-11 14:11 +0000
                  Re: Implementation of execute-parsing-file Doug Hoffman <glidedog@gmail.com> - 2011-04-07 11:46 -0400
                    Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-08 18:24 -0700
                      Re: Implementation of execute-parsing-file Doug Hoffman <glidedog@gmail.com> - 2011-04-09 02:53 -0400
                        Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-09 13:01 -0700
              Re: Implementation of execute-parsing-file BruceMcF <agila61@netscape.net> - 2011-04-06 09:14 -0700
          Re: Implementation of execute-parsing-file Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-04-06 10:15 +0100

Page 1 of 2  [1] 2  Next page →

#1007 — Implementation of execute-parsing-file

There have previously been discussions on the issue of using ANS Forth 
words such as WORD, SOURCE, >IN etc on a text file that is not a Forth 
source file e.g. see

https://groups.google.com/group/comp.lang.forth/browse_frm/thread/2debffde7a48617f/fb21c08ade99ef93?hl=en&lnk=gst&q=file-source#fb21c08ade99ef93
https://groups.google.com/group/comp.lang.forth/browse_frm/thread/93aac6e9ce493606/7cf533cf0afb8c90?hl=en&lnk=gst&q=file-source#7cf533cf0afb8c90

GForth has a word EXECUTE-PARSING-FILE that can be used to parse and 
process a non-Forth text file but, AFAIK, there isn't an ANS Forth 
implementation. Anton Ertl implemented a related word called 
EXECUTE-PARSING in ANS Forth, see

https://groups.google.com/group/comp.lang.forth/browse_frm/thread/1434bda49fa6d48c/b70e1d9320747fbc?hl=en&lnk=gst&q=%22prepended+to+the+string%22#b70e1d9320747fbc

Having a need for an ANS Forth EXECUTE-PARSING-FILE I used the method in 
EXECUTE-PARSING to implement it and integrated the two to get:

\ --------------------------------------

wordlist constant execute-parsing-wordlist

get-current execute-parsing-wordlist set-current

\ X is prepended to the string, then the string is EVALUATEd
: X ( xt -- )
     previous execute
     source >in ! drop ; immediate \ skip remaining input

set-current

: >order ( wid -- )
   >r get-order 1+ r> swap set-order ;

: (exec-parsing)  ( ... xt u caddr -- ... )
    dup >r s" X " r> swap cmove
    tuck >r                          ( -- xt caddr u+2 ) ( R: -- caddr )
    execute-parsing-wordlist >order
    ['] evaluate catch               ( -- )
    r> free throw throw
;

: execute-parsing ( ... caddr u xt -- ... )
    -rot dup >r 2 chars + tuck       ( -- xt u+2 caddr u+2 ) ( R: -- u )
    allocate throw                   ( -- xt u+2 caddr caddr2 )
    tuck 2 chars + r> cmove          ( -- xt u+2 caddr2 ) ( R: -- )
    (exec-parsing)                   ( -- )
;

: execute-parsing-file  ( ... fid xt -- ... )
    swap dup file-size throw drop    ( -- xt fid u )
    tuck 2>r 2 chars +               ( -- xt u+2 )  ( R: -- fid u )
    dup allocate throw               ( -- xt u+2 caddr )
    dup 2 chars + r> r> read-file    ( -- xt u+2 caddr u2 ior ) ( R: -- )
    throw drop                       ( -- xt u+2 caddr )
    (exec-parsing)                   ( -- )
;

\ A crude test program

: type-words		\ The parsing word
    begin
       bl word count dup
    while
       cr type
    repeat
    2drop cr
;

: go  ( caddr u -- )
    r/o open-file throw dup >r       ( R: -- fid )
    ['] type-words execute-parsing-file
    r> close-file throw
;

\ s" filename" go

\ --------------------------------------

Is there any reason this is not standard ANS Forth, it works on several 
Forth systems. I can't see anything in the standard that limits the size 
of a string being EVALUATEd or that prohibits line ending control 
characters in the string. Of course CR and LF have to be handled by the 
parsing word e.g. type-words above which, in this case, simply ignores them.

-- 
Gerry

[toc] | [next] | [standalone]

#1008

>>>>> "Gerry" == Gerry Jackson <gerry@jackson9000.fsnet.co.uk> writes:

> There have previously been discussions on the issue of using ANS Forth
> words such as WORD, SOURCE, >IN etc on a text file that is not a Forth
> source file e.g. see

> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/2debffde7a48617f/fb21c08ade99ef93?hl=en&lnk=gst&q=file-source#fb21c08ade99ef93
> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/93aac6e9ce493606/7cf533cf0afb8c90?hl=en&lnk=gst&q=file-source#7cf533cf0afb8c90

> GForth has a word EXECUTE-PARSING-FILE that can be used to parse and
> process a non-Forth text file but, AFAIK, there isn't an ANS Forth
> implementation. Anton Ertl implemented a related word called
> EXECUTE-PARSING in ANS Forth, see

> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/1434bda49fa6d48c/b70e1d9320747fbc?hl=en&lnk=gst&q=%22prepended+to+the+string%22#b70e1d9320747fbc

> Having a need for an ANS Forth EXECUTE-PARSING-FILE I used the method
> in EXECUTE-PARSING to implement it and integrated the two to get:

I didn't take the time to analyze your implementation in detail, but I
think it fails at REFILL, which would advance by a single line in
EXECUTE-PARSING-FILE but which will skip to the end of the file with
your implementation.

An easy way to test that is to do POSTPONE \  with your implementation
which would skip more than just a single line.

(I was looking at this example that removes comments starting on '#'
from a file:
http://mosquito.dyndns.tv/freesvn/trunk/scriptometer/remove_comments.fs)

cheers,

David
-- 
GnuPG public key: http://user.cs.tu-berlin.de/~dvdkhlng/dk.gpg
Fingerprint: B17A DC95 D293 657B 4205  D016 7DEF 5323 C174 7D40

[toc] | [prev] | [next] | [standalone]

#1009

On 05/04/2011 10:58, David Kuehling wrote:
>>>>>> "Gerry" == Gerry Jackson<gerry@jackson9000.fsnet.co.uk>  writes:
>
>> There have previously been discussions on the issue of using ANS Forth
>> words such as WORD, SOURCE,>IN etc on a text file that is not a Forth
>> source file e.g. see
>
>> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/2debffde7a48617f/fb21c08ade99ef93?hl=en&lnk=gst&q=file-source#fb21c08ade99ef93
>> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/93aac6e9ce493606/7cf533cf0afb8c90?hl=en&lnk=gst&q=file-source#7cf533cf0afb8c90
>
>> GForth has a word EXECUTE-PARSING-FILE that can be used to parse and
>> process a non-Forth text file but, AFAIK, there isn't an ANS Forth
>> implementation. Anton Ertl implemented a related word called
>> EXECUTE-PARSING in ANS Forth, see
>
>> https://groups.google.com/group/comp.lang.forth/browse_frm/thread/1434bda49fa6d48c/b70e1d9320747fbc?hl=en&lnk=gst&q=%22prepended+to+the+string%22#b70e1d9320747fbc
>
>> Having a need for an ANS Forth EXECUTE-PARSING-FILE I used the method
>> in EXECUTE-PARSING to implement it and integrated the two to get:
>
> I didn't take the time to analyze your implementation in detail, but I
> think it fails at REFILL, which would advance by a single line in
> EXECUTE-PARSING-FILE but which will skip to the end of the file with
> your implementation.

As the string (file) is being evaluated, REFILL would return 0, 
effectively skipping the rest of the file
>
> An easy way to test that is to do POSTPONE \  with your implementation
> which would skip more than just a single line.

Your are right, in my application I wasn't thinking of using REFILL or \.

>
> (I was looking at this example that removes comments starting on '#'
> from a file:
> http://mosquito.dyndns.tv/freesvn/trunk/scriptometer/remove_comments.fs)
>

To get that, very non-standard, code to work I would need to redefine 
REFILL to a skip to the next line unless already there, returning FALSE 
at the end of the string otherwise TRUE, and redefine \ to skip to the 
next line. I guess GForth does this by default as, IIRC, it slurps a 
file into memory before interpreting it.

This is doable but beyond the scope of what I want to do, so, as it 
doesn't emulate the functionality of the GForth word exactly, I should 
probably give it another name. It's useful, to me at least, with 
restrictions on using REFILL and POSTPONE \

-- 
Gerry

[toc] | [prev] | [next] | [standalone]

#1015

Gerry Jackson wrote: <ineu9q$bdq$1@dont-email.me>
> On 05/04/2011 10:58, David Kuehling wrote:
>> Gerry Jackson<gerry@jackson9000.fsnet.co.uk>  writes:
>>>
>>> GForth has a word EXECUTE-PARSING-FILE that can be used to parse and
>>> process a non-Forth text file but, AFAIK, there isn't an ANS Forth
>>> implementation.
>>>
>>> Having a need for an ANS Forth EXECUTE-PARSING-FILE I used the method
>>> in EXECUTE-PARSING to implement it and integrated the two to get:
>>
>> I didn't take the time to analyze your implementation in detail, but I
>> think it fails at REFILL, which would advance by a single line in
>> EXECUTE-PARSING-FILE but which will skip to the end of the file with
>> your implementation.
>
> As the string (file) is being evaluated, REFILL would return 0, 
> effectively skipping the rest of the file
>>
>> An easy way to test that is to do POSTPONE \  with your implementation
>> which would skip more than just a single line.

You don't even have to POSTPONE it: using \ at all will skip the entire
rest of the file.

> Your are right, in my application I wasn't thinking of using REFILL or \.

> as it doesn't emulate the functionality of the GForth word exactly, I
> should probably give it another name. It's useful, to me at least,
> with restrictions on using REFILL and POSTPONE \

Yeah, I messed around with this a while ago.  You can get close (as you
did), but I can't see any way to do a full implementation in standard
Forth.  If you EVALUATE the file as one string, then you have problems
with words which expect the parse area to hold a single line.  If you
EVALUATE it line-by-line, then you have problems with words like [IF]
which expect REFILL to return the next line.

--Josh

[toc] | [prev] | [next] | [standalone]

#1016

On Apr 5, 6:35 pm, Josh Grams <j...@qualdan.com> wrote:

> Yeah, I messed around with this a while ago.  You can get close (as you
> did), but I can't see any way to do a full implementation in standard
> Forth.  If you EVALUATE the file as one string, then you have problems
> with words which expect the parse area to hold a single line.  If you
> EVALUATE it line-by-line, then you have problems with words like [IF]
> which expect REFILL to return the next line.

I think that's right. It may be possible for a specific application to
be written so that its parsing words can test if they are being called
with the parse-file-with or whatever name, but that was the argument
for something like INCLUDE-WITH as a standard word ~ that it cannot be
done in a standard war. INCLUDE-WITH would be ( ca u "filename" --
x*i ) where the first string is evaluated, and then parsing passes to
the included file. If the first string contains a word that does
parsing with refill, it can do the equivalent of execute-parsing-file.

[toc] | [prev] | [next] | [standalone]

#1020

On 06/04/2011 02:31, BruceMcF wrote:
> On Apr 5, 6:35 pm, Josh Grams<j...@qualdan.com>  wrote:
>
>> Yeah, I messed around with this a while ago.  You can get close (as you
>> did), but I can't see any way to do a full implementation in standard
>> Forth.  If you EVALUATE the file as one string, then you have problems
>> with words which expect the parse area to hold a single line.  If you
>> EVALUATE it line-by-line, then you have problems with words like [IF]
>> which expect REFILL to return the next line.
>
> I think that's right. It may be possible for a specific application to
> be written so that its parsing words can test if they are being called
> with the parse-file-with or whatever name, but that was the argument
> for something like INCLUDE-WITH as a standard word ~ that it cannot be
> done in a standard war.

Has there been a discussion on INCLUDE-WITH, I couldn't find one in 
Google groups. One of the references I gave was to INTERPRET-WITH which 
would be a more general solution.

INCLUDE-WITH would be ( ca u "filename" --
> x*i ) where the first string is evaluated, and then parsing passes to
> the included file. If the first string contains a word that does
> parsing with refill, it can do the equivalent of execute-parsing-file.

What's the advantage of evaluating a string over passing an xt to 
INCLUDE-WITH?

-- 
Gerry

[toc] | [prev] | [next] | [standalone]

#1021

On Apr 6, 5:15 am, Gerry Jackson <ge...@jackson9000.fsnet.co.uk>
wrote:
> What's the advantage of evaluating a string over passing an xt to
> INCLUDE-WITH?

AFAIR, flexibility and reliability ~ evaluating a string and then
passing into parsing a file can do anything passing an XT can do, but
is also the more convenient form for using macro processing, and of
course a debugging version that logs the evaluate string makes it easy
to see when the string handed for conditional compilation is not the
string you thought ought to have been handed under those conditions.

And of course, for small systems, if you can save the preferred
settings as a block of actions, standard default in another, and
special cases in others, you can save valuable dictionary real estate
with a block variable instead of defining the settings as various
words and passing the xt to the word:

user-settings 2@ INCLUDE-WTH system-restart.fs

[toc] | [prev] | [next] | [standalone]

#1034

BruceMcF wrote:
> On Apr 6, 5:15 am, Gerry Jackson <ge...@jackson9000.fsnet.co.uk>
> wrote:
>> What's the advantage of evaluating a string over passing an xt to
>> INCLUDE-WITH?
>
> AFAIR, flexibility and reliability ~ evaluating a string and then
> passing into parsing a file can do anything passing an XT can do, but
> is also the more convenient form for using macro processing,

Hrm.  I'd argue the other way on that.  It's *less* reliable, as it's
late binding, so if the evaluated string is defined in a different place
than where it's used, you have to worry about whether all the names have
the expected meaning.  It's no more flexible -- if the word takes an xt
and you really want to evaluate a string, you can pass a string and the
xt of EVALUATE.  Also it's just an unnecessary evaluation of a string,
which I try to avoid where possible.

But then I think that string-replacement macro processing is generally a
very bad fit for Forth, and I avoid it wherever another reasonable
option exists...

> and of course a debugging version that logs the evaluate string makes
> it easy to see when the string handed for conditional compilation is
> not the string you thought ought to have been handed under those
> conditions.

I can't follow that at all.  Can you give me an example?

--Josh

[toc] | [prev] | [next] | [standalone]

#1040

On Apr 7, 6:19 am, Josh Grams <j...@qualdan.com> wrote:
> Hrm.  I'd argue the other way on that.  It's *less* reliable, as it's
> late binding, so if the evaluated string is defined in a different place
> than where it's used, you have to worry about whether all the names have
> the expected meaning.

Its as late or early binding as you wish it to be, since it can of
course be:
   S" EXECUTE" INCLUDE-WITH foo.fs
or
   S" foo.fs" R/O OPEN-FILE THROW
   S" EXECUTE" INCLUDE-FILE-WITH

IOW, where the objection about it being an unecessary evaluation of a
string is valid, the objection about it being late binding is not.

Remember that there's only a capability gap *to be bridged* for Forth
source or self-executing data files where "something needs to be done"
prior to parsing that file as source.

Indeed, the entirely standard way to accomplish execute-parsing-file
is to write the word EXECUTE in its own line to a newly created
temporary text file, then copy the source file line by line, include-
file, then delete the temp file ~ so the rationale is not really that
"it can't be done", but rather that doing it in the entirely standard
way requires two passes through a file and creation of an otherwise
unnecessary temporary data file.

So its not strictly true, as claimed, that it "can't be done" in
Standard Forth, but rather that it can't be done in the more efficient
process that people are envisioning.

If the file in question is not being parsed as source, there is, of
course, no *need* for a standard word to get over the "REFILL" gap --
a direct line by line file parser with the same capabilities as the
built in file parser is not hard to put together, and then its not an
issue because the "get-next-line" function is as you specified it.

(Or as someone else specified it. If there a line by line file parser
somewhere in FLAG? I recall a csv file wordset being discussed a few
years back ~ that might contain within it all of the line by line file
parser words required by any application which can get by with just
the primitive built-in Forth parser routines).

> It's no more flexible -- if the word takes an xt and you really
> want to evaluate a string, you can pass a string and the
> xt of EVALUATE.

2VARIABLE include-with$
: parse-prelude ( x*i -- x*j ) include-with$ 2@ EVALUATE ;
: parse-file-after ( x*i fid ca1 u1 -- x*j )
   include-with$ 2! ['] parse-prelude execute-parsing-file ;

So, yes, any way to get something done with a specified file "next in
line" when you hit REFILL will be workable.

The direct way to develop and test those files is by prepending a line
to the file to be parsed and including it, and so the direct approach
to moving that from the test bench to an application involving the
files in their original format without the prepended source is to pass
the prepend string to the word that will evaluate it first before the
parser passes on to the source file.

[toc] | [prev] | [next] | [standalone]

#1046

BruceMcF <agila61@netscape.net> writes:
>On Apr 7, 6:19=A0am, Josh Grams <j...@qualdan.com> wrote:
>> Hrm. =A0I'd argue the other way on that. =A0It's *less* reliable, as it's
>> late binding, so if the evaluated string is defined in a different place
>> than where it's used, you have to worry about whether all the names have
>> the expected meaning.
>
>Its as late or early binding as you wish it to be, since it can of
>course be:
>   S" EXECUTE" INCLUDE-WITH foo.fs
>or
>   S" foo.fs" R/O OPEN-FILE THROW
>   S" EXECUTE" INCLUDE-FILE-WITH

That would late-bind EXECUTE.  There are workarounds for this kind of
misdesign (look at the standard implementation of EXECUTE-PARSING),
but it's better to avoid the misdesign than to work around it.

Also, with a string to EVALUATE the current input source would be the
string, not the FILE, and it would be very messy both in the
specification and in the implementation to let a parsing word in the
string parse the file.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2010: http://www.euroforth.org/ef10/

[toc] | [prev] | [next] | [standalone]

#1073

On Apr 7, 12:32 pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> BruceMcF <agil...@netscape.net> writes:
> >On Apr 7, 6:19=A0am, Josh Grams <j...@qualdan.com> wrote:
> >> Hrm. =A0I'd argue the other way on that. =A0It's *less* reliable, as it's
> >> late binding, so if the evaluated string is defined in a different place
> >> than where it's used, you have to worry about whether all the names have
> >> the expected meaning.
>
> >Its as late or early binding as you wish it to be, since it can of
> >course be:
> >   S" EXECUTE" INCLUDE-WITH foo.fs
> >or
> >   S" foo.fs" R/O OPEN-FILE THROW
> >   S" EXECUTE" INCLUDE-FILE-WITH
>
> That would late-bind EXECUTE.  There are workarounds for this kind of
> misdesign (look at the standard implementation of EXECUTE-PARSING),
> but it's better to avoid the misdesign than to work around it.

Yes, if EXECUTE is not executed on a standard system, it will indeed
not have standard effects. But if you do not know what the words you
call will do, then you don't know what execute-parsing-file will do
either, so that argument seems to be a bit empty.

> Also, with a string to EVALUATE the current input source would be the
> string, not the FILE, and it would be very messy both in the
> specification and in the implementation to let a parsing word in the
> string parse the file.

In what way is it messy? When either a word in the input source or the
Forth interpreter at the end of the input source does a REFILL, you
get the first line of the file and you keep getting lines of the file
until/unless you return to evaluating, you go to the end of the file.
Pasting it on top from outside the system is certainly messy, but
there's no reason it has to be messy is built into the implementation.

[toc] | [prev] | [next] | [standalone]

#1081

BruceMcF <agila61@netscape.net> writes:
>On Apr 7, 12:32=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> BruceMcF <agil...@netscape.net> writes:
>> >On Apr 7, 6:19=3DA0am, Josh Grams <j...@qualdan.com> wrote:
>> >> Hrm. =3DA0I'd argue the other way on that. =3DA0It's *less* reliable, =
>as it's
>> >> late binding, so if the evaluated string is defined in a different pla=
>ce
>> >> than where it's used, you have to worry about whether all the names ha=
>ve
>> >> the expected meaning.
>>
>> >Its as late or early binding as you wish it to be, since it can of
>> >course be:
>> > =A0 S" EXECUTE" INCLUDE-WITH foo.fs
>> >or
>> > =A0 S" foo.fs" R/O OPEN-FILE THROW
>> > =A0 S" EXECUTE" INCLUDE-FILE-WITH
>>
>> That would late-bind EXECUTE. =A0There are workarounds for this kind of
>> misdesign (look at the standard implementation of EXECUTE-PARSING),
>> but it's better to avoid the misdesign than to work around it.
>
>Yes, if EXECUTE is not executed on a standard system, it will indeed
>not have standard effects. But if you do not know what the words you
>call will do, then you don't know what execute-parsing-file will do
>either, so that argument seems to be a bit empty.

As a programmer, I generally know what the words visible during
compilation do, but I have no influence on what will be visible at
run-time.  Consider:

: foo1 ... ['] bar execute-parsing-file ... ;
: foo2 ... s" bar" include-file-with ... ;
: foo3 ... ['] bar s" execute" include-file-with ;

And consider the case that one of these words will be called (maybe
through some intermediate layer) from an application word written by
someone else in some sealed application vocabulary, like this:

: flip ... foo? ... ;
...
: flop ... flip ... ;

wordlist constant app-wl
app-wl set-current

: app-word ... flop ... ;

app-wl 1 set-context \ seal the whole thing

and now the user types APP-WORD.  If FOO1 was used, everything works
nicely, whereas with FOO2 and FOO3 the user will most likely get an
"undefined word" error message (or worse, the program might use a
different word than the programmer had in mind).

An advice frequently given here is "Don't do this!".  And I agree:
Don't use late binding for referencing words that you know at compile
time.  And don't design words such that they require late binding.

>> Also, with a string to EVALUATE the current input source would be the
>> string, not the FILE, and it would be very messy both in the
>> specification and in the implementation to let a parsing word in the
>> string parse the file.
>
>In what way is it messy?

Write a specification and compare it to a specification for
EXECUTE-PARSING-FILE.  Write tests, especially for corner cases.
Implement it on several systems, and see how much effort it is, and
how many bugs crop up.

>When either a word in the input source or the
>Forth interpreter at the end of the input source does a REFILL, you
>get the first line of the file and you keep getting lines of the file
>until/unless you return to evaluating, you go to the end of the file.

So, the parsing word that you pass is required to start off with a
REFILL in order to parse anything from the actual file?  Even more
messyness than I originally was thinking of.

Also, this approach requires additional complications if the system
just mmaps the file into memory and is designed for then processing
this memory image sequentially (I don't know if there is any such
system, though).

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2010: http://www.euroforth.org/ef10/

[toc] | [prev] | [next] | [standalone]

#1091

On Apr 9, 9:10 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> As a programmer, I generally know what the words visible during
> compilation do, but I have no influence on what will be visible at
> run-time.

If you are parsing a self-executing data file, you better be
exercising control over what is visible at runtime.

That's an application I see of this word, where using the Forth system
line at a time file parser is called for, since a wordset has been
developed for which the data file itself is source file for that
wordset. But then, to do that, you have to have control over what is
visible.

If the state of the Forth dictionary is not supposed to matter when
you parse the file, I don't see the need to parse the file as a Forth
source file. And it its not being parsed as a Forth source file,
there's no real challenge with doing it portably.

[toc] | [prev] | [next] | [standalone]

#1118

BruceMcF <agila61@netscape.net> writes:
>On Apr 9, 9:10=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> As a programmer, I generally know what the words visible during
>> compilation do, but I have no influence on what will be visible at
>> run-time.
>
>If you are parsing a self-executing data file, you better be
>exercising control over what is visible at runtime.

For parsing self-executing data files, we already have a word:
INCLUDED (and friends).

EXECUTE-PARSING-FILE is for processing files without text-interpreting
the contents, but still using words that take strings from the input
stream (what many useful system and application words do).

>If the state of the Forth dictionary is not supposed to matter when
>you parse the file, I don't see the need to parse the file as a Forth
>source file.

The need comes because many useful words take string inputs from the
input stream.  If you just wanted to process the file with READ-LINE
and friends, you would have to implement versions of these words that
take their string input as a c-addr u pair (or something like that)
from that stack.

Maybe the original authors of the parsing words misdesigned them, but
now they are there, and using them in combination with
EXECUTE-PARSING-FILE is probably superior to the reimplementation
approach outlined above.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2010: http://www.euroforth.org/ef10/

[toc] | [prev] | [next] | [standalone]

#1125

On Apr 10, 9:22 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> Maybe the original authors of the parsing words misdesigned them, but
> now they are there, and using them in combination with
> EXECUTE-PARSING-FILE is probably superior to the reimplementation
> approach outlined above.

The why the ( fid xt -- ) specification? It would seem far more
flexible with a ( x*i xt fid -- x*j ) specification.

[toc] | [prev] | [next] | [standalone]

#1151

BruceMcF <agila61@netscape.net> writes:
>On Apr 10, 9:22=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> Maybe the original authors of the parsing words misdesigned them, but
>> now they are there, and using them in combination with
>> EXECUTE-PARSING-FILE is probably superior to the reimplementation
>> approach outlined above.
>
>The why the ( fid xt -- ) specification? It would seem far more
>flexible with a ( x*i xt fid -- x*j ) specification.

Hmm?  The specification given in the Gforth 0.7.0 manual
<http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/The-Input-Stream.html>
is:

|execute-parsing-file     i*x fileid xt -- j*x   gforth  "execute-parsing-file"
|
|Make fileid the current input source, execute xt ( i*x -- j*x ), then
|restore the previous input source.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2010: http://www.euroforth.org/ef10/

[toc] | [prev] | [next] | [standalone]

#1153

On Apr 11, 10:22 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> Hmm?  The specification given in the Gforth 0.7.0 manual
> <http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/The-Input-Str...>
> is:
>
> |execute-parsing-file     i*x fileid xt -- j*x   gforth  "execute-parsing-file"
> |
> |Make fileid the current input source, execute xt ( i*x -- j*x ), then
> |restore the previous input source.

Ah, I had been looking at the specification given by the OP.

[toc] | [prev] | [next] | [standalone]

#1155

On Apr 10, 9:22 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> For parsing self-executing data files, we already have a word:
> INCLUDED (and friends).

> EXECUTE-PARSING-FILE is for processing files without text-interpreting
> the contents, but still using words that take strings from the input
> stream (what many useful system and application words do).

Yes, but in a given parsing application, only a given number of them
are required, and if each parsing word has a non-parsing factor,
applying them to an application specific data file parser is
straightforward.

Where execute-parsing-file gives me a capability that is only
otherwise available to standard source by writing new include files on
the fly is prepending an action or source into the include file
processs.

Whether the application words take strings directly from the input
stream or not depends on whether the application programmer decides to
make them do so.

> The need comes because many useful words take string inputs from the
> input stream.  If you just wanted to process the file with READ-LINE
> and friends, you would have to implement versions of these words that
> take their string input as a c-addr u pair (or something like that)
> from that stack.

All parsing words ought to have versions of the words that take their
string input from the stack in any event. There may be a hole or two
in the standard Forth words in respect to that, but they are only an
obstacle when attempting to parse Forth source in a portable way ~
when parsing a data file they can simply be replaced.

> Maybe the original authors of the parsing words misdesigned them, but
> now they are there, and using them in combination with
> EXECUTE-PARSING-FILE is probably superior to the reimplementation
> approach outlined above.

If the parsing task is so primitive that the Forth source parser
suffices as is, its a piece of cake to replace it, and if it requires
building something more complex, its more secure to build that up with
non-parsing factors.

[toc] | [prev] | [next] | [standalone]

#1194

BruceMcF <agila61@netscape.net> writes:
>On Apr 10, 9:22=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:
>> The need comes because many useful words take string inputs from the
>> input stream. =A0If you just wanted to process the file with READ-LINE
>> and friends, you would have to implement versions of these words that
>> take their string input as a c-addr u pair (or something like that)
>> from that stack.
>
>All parsing words ought to have versions of the words that take their
>string input from the stack in any event.

Yes, but that does not help the programmer who wants to use a word
(possibly written by someone else) that does what he needs, but takes
its string input from the input stream.

>There may be a hole or two
>in the standard Forth words in respect to that

So if even standard words don't have the desired property, can we
expect all user-defined words to have it?

>but they are only an
>obstacle when attempting to parse Forth source in a portable way ~
>when parsing a data file they can simply be replaced.

No, you can parse Forth source with INCLUDED or EVALUATE.  But you
might find an existing parsing word useful for parsing data.

>> Maybe the original authors of the parsing words misdesigned them, but
>> now they are there, and using them in combination with
>> EXECUTE-PARSING-FILE is probably superior to the reimplementation
>> approach outlined above.
>
>If the parsing task is so primitive that the Forth source parser
>suffices as is, its a piece of cake to replace it, and if it requires
>building something more complex, its more secure to build that up with
>non-parsing factors.

If that were so, there would be no need for EXECUTE-PARSING-FILE, yet
this thread was started because there is someone who has a need for it.

I also used to think that we should have non-parsing factors for all
parsing words, but I am no longer sure of that, for several reasons:

1) This principle increases the number of words and makes Forth more
cumbersome to use.

2) Non-parsing factors are harder to write, because they have to
explicitly manage the input string.

So maybe it's better to write words that are easier to write as
parsing words without non-parsing equivalent, and make them
non-parsing on demand by calling them through EXECUTE-PARSING.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2010: http://www.euroforth.org/ef10/

[toc] | [prev] | [next] | [standalone]

#1197

On Apr 13, 12:50 pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:
> BruceMcF <agil...@netscape.net> writes:
> >On Apr 10, 9:22=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >wrote:
> >> The need comes because many useful words take string inputs from the
> >> input stream. =A0If you just wanted to process the file with READ-LINE
> >> and friends, you would have to implement versions of these words that
> >> take their string input as a c-addr u pair (or something like that)
> >> from that stack.

> >All parsing words ought to have versions of the words that take their
> >string input from the stack in any event.

> Yes, but that does not help the programmer who wants to use a word
> (possibly written by someone else) that does what he needs, but takes
> its string input from the input stream.

No, it doesn't. It also does not encourage that programmer to write
yet more words of the same type and pass the same limitation on to
anyone who may wish to borrow from their code.

> >There may be a hole or two
> >in the standard Forth words in respect to that

> So if even standard words don't have the desired property, can we
> expect all user-defined words to have it?

The standard words had to cope with existing legacy, user defined
words are free to avoid what have turned out to be tarpits.

> >but they are only an
> >obstacle when attempting to parse Forth source in a portable way ~
> >when parsing a data file they can simply be replaced.

> No, you can parse Forth source with INCLUDED or EVALUATE.  But you
> might find an existing parsing word useful for parsing data.

While concurrently we have Ed declaring that an implementation should
be free to break some current possible uses of parsing words to parse
data so long as it does not interfere with the way that Forth source
is normally laid out.

> >> Maybe the original authors of the parsing words misdesigned them, but
> >> now they are there, and using them in combination with
> >> EXECUTE-PARSING-FILE is probably superior to the reimplementation
> >> approach outlined above.

> >If the parsing task is so primitive that the Forth source parser
> >suffices as is, its a piece of cake to replace it, and if it requires
> >building something more complex, its more secure to build that up with
> >non-parsing factors.

> If that were so, there would be no need for EXECUTE-PARSING-FILE, yet
> this thread was started because there is someone who has a need for it.

A want for it, in any event.

> I also used to think that we should have non-parsing factors for all
> parsing words,

I was, of course, paraphrasing you.


> but I am no longer sure of that, for several reasons:

> 1) This principle increases the number of words and makes Forth more
> cumbersome to use.

I don't follow how the second follows from the first.

> 2) Non-parsing factors are harder to write, because they have to
> explicitly manage the input string.

Yes, that is how the legacy words wandered into the tarpit in the
first place. Sidestepping a problem rather than solving it often has
more appealing short term consequences than long term consequences.

> So maybe it's better to write words that are easier to write as
> parsing words without non-parsing equivalent, and make them
> non-parsing on demand by calling them through EXECUTE-PARSING.

In any event, EXECUTE-PARSING-FILE seems like it might be overkill for
providing a standard word for allowing that functionality, if:
   FORTH-SOURCE ( fid -- )
... to preserve current input source and establish the file as the
current source would suffice. Then EXECUTE-PARSING-FILE with the
deliberately obscure placing of the fid between the XT and its
possible arguments would be:

: EXECUTE-PARSING-FILE ( x*i fid xt -- x*j ) >R FORTH-SOURCE R>
EXECUTE ;

and ``INCLUDE-FILE-WITH'' ( x*i xt fid -- x*j ) would be:
: INCLUDE-FILE-WITH ( x*i xt fid -- x*j ) FORTH-SOURCE EXECUTE ;

I guess it does place unnesting from a file source on the text
interpreter, which seems like it could otherwise could be a discrete
process in INCLUDE-FILE and descendants, but EXECUTE-PARSING-FILE
requires that anyway, for the case where it returns with a position in
the file indicated by 'fid' as the source for the file interpreter.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.forth

csiph-web