Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #19218 > unrolled thread

Offline compilation of Forth

Back to article view | Back to comp.lang.forth

Contents

  Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-01-28 11:54 +0000
    Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-01-28 17:19 +0000
      Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-01-28 22:08 +0100
      Re: Offline compilation of Forth "Peter Knaggs" <pjk@bcs.org.uk> - 2013-02-03 10:50 +0000
        Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 12:11 +0000
          Re: Offline compilation of Forth "A. K." <akk@nospam.org> - 2013-02-03 14:59 +0100
            Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-03 15:14 +0100
              Re: Offline compilation of Forth Hannu Vuolasaho <hannu.vuolasaho@nospam.tut.fi.invalid> - 2013-02-03 15:25 +0000
            Re: Offline compilation of Forth Gary Bergstrom <g.bergstrom@ieee.org> - 2013-02-05 08:13 -0800
        Re: Offline compilation of Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-02-04 16:23 +0000
    Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-28 12:26 -1000
    Re: Offline compilation of Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-01-29 08:55 +0000
      Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-01-29 17:49 +0100
        Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-01-30 00:42 -0800
          Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-01-30 11:58 +0000
          Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-01-30 23:16 -0800
            Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-01-31 13:29 +0000
              Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-03 12:49 +1100
                Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 20:37 -0800
        Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-03 21:15 +0200
          Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-04 02:04 +0100
        Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-04 13:22 +0000
          Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-04 20:07 +0100
            Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-04 23:07 +0200
              Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-05 18:33 +0100
                Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-06 22:53 +0200
                  Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-07 00:30 +0100
      Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-01-30 00:36 -0800
    Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-01-30 18:12 +0000
      Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-30 09:36 -1000
      Re: Offline compilation of Forth "A. K." <akk@nospam.org> - 2013-01-31 07:45 +0100
      Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-01-31 03:37 -0600
        Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-02-01 00:01 +0000
          Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 14:59 -1000
            Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-02-05 15:17 +0000
              Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-02-05 10:46 -0600
                Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-05 18:47 +0100
      Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-01-31 21:47 +1100
        Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-01-31 06:12 -0600
          Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-01-31 13:32 +0000
          Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 08:27 -1000
            Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-01 10:23 +1100
              Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 13:53 -1000
                Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-02-01 03:17 +0000
                  Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 18:43 -1000
                  Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-01 08:45 -0800
                Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-01-31 23:05 -0800
                  Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 21:25 -1000
                  Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-01 08:55 -0800
                    Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-01 09:18 -1000
                      Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-02-01 13:04 -0800
                Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-03 11:20 +1100
                  Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 12:20 +0000
                    Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 07:22 -0800
                      Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 17:54 +0000
                        Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 10:17 -0800
                        Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-02-03 19:33 +0000
                          Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 11:53 -0800
                            Re: Offline compilation of Forth Coos Haak <chforth@hccnet.nl> - 2013-02-04 00:59 +0100
                          Re: Offline compilation of Forth Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2013-02-04 11:38 +0100
                        Re: Offline compilation of Forth Brad Eckert <hwfwguy@gmail.com> - 2013-02-06 09:13 -0800
                          Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 21:02 -0800
                        Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 20:55 -0800
                      Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-03 08:26 -1000
                    Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-05 13:04 +1100
                      Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-05 10:33 +0000
                        Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-05 08:56 -1000
                          Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-08 02:07 +1100
                            Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-07 19:03 +0100
                              Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-09 22:45 +1100
                                Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-09 18:56 +0100
                                  Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-11 00:19 +0100
                                Re: Offline compilation of Forth Alex McDonald <blog@rivadpm.com> - 2013-02-09 11:59 -0800
                                  Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-13 13:13 +1100
                                    Re: Offline compilation of Forth Alex McDonald <blog@rivadpm.com> - 2013-02-12 19:19 -0800
                          Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 21:01 -0800
        Re: Offline compilation of Forth Roberto Waltman <usenet@rwaltman.com> - 2013-02-03 16:26 -0500
          Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-05 15:32 +1100
            Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-02-04 23:18 -0800

Page 1 of 4  [1] 2 3 4  Next page →

#19218 — Offline compilation of Forth

Hello.

I recently started getting acquainted with Forth. I'm quite impressed
that such a simple core language can provide Lisp-like syntax
manipulation power. However, I'm wondering about how this power
constrains the implementation options.

Forth is often used in embedded systems. I'm not very familiar with
them, but I'd expect resources to be so scarce that cross-compilation to
the target's native code would be the preferred implementation
technique: interpretation would waste cycles on an already slow
microcontroller, and there wouldn't be enough memory for a native
non-cross-compiler, JIT or otherwise.

However, standard Forth is specified as an interpreted language, where
execution semantics and compilation semantics can be interleaved, each
affecting the other. For instance:

variable counter
0 counter !
: inc counter @ dup 1+ counter ! ;
inc .
: foo inc postpone literal ; immediate
: bar foo . ;
inc .
bar
bye

With such a tight interdependency between compilation and execution,
it's hard to see how standard Forth could be efficiently compiled
statically in the general case. Of course a compiler could try to
recognize easily-compilable (and presumably common) patterns, but I
suspect such analysis to be very hard.

Lisps have had exactly the same problem. The solution has been to
manually separate the compile-time code (used by macros for syntax
extensions) from the run-time code, originally with "eval-when"
annotations, and later with module systems that automate the hurdles of
phase separation. The phases can still share _code_, but they cannot
affect each other's _state_.

So how do offline native Forth compilers tackle the problem? Do they
also require use of non-standard features in order to provide efficient
compilation, or is there some other solution?

Thanks in advance!


Lauri

[toc] | [next] | [standalone]

#19226

On Mon, 28 Jan 2013 11:54:34 +0000 (UTC), Lauri Alanko <la@iki.fi>
wrote:

>Forth is often used in embedded systems. I'm not very familiar with
>them, but I'd expect resources to be so scarce that cross-compilation to
>the target's native code would be the preferred implementation
>technique: interpretation would waste cycles on an already slow
>microcontroller, and there wouldn't be enough memory for a native
>non-cross-compiler, JIT or otherwise.
>
>However, standard Forth is specified as an interpreted language, where
>execution semantics and compilation semantics can be interleaved, each
>affecting the other. For instance:
>
>variable counter
>0 counter !
>: inc counter @ dup 1+ counter ! ;
>inc .
>: foo inc postpone literal ; immediate
>: bar foo . ;
>inc .
>bar
>bye

This all depends what you mean by "embedded system". Let's start by
assuming that we can exclude embedded Linux from this discussion.
However, even an 8051 (128/256 bytes) or an MSP430 (512 bytes) has
enough RAM to contain a simple interpreter, but not enough RAM to
contain a full native code code generator.

Nowadays, a modern ARM device, e.g. STM32F4xx, will have 1Mb Flash
and 192kb RAM on-chip. Such a device could have a full-blown code
generator if the application demanded it. However, the common
approach is to use a cross-compiled application that contains a
simple text interpreter. We use such configurations with Telnet
and HTTP servers that use the Forth interpreter to provide
server-side scripting.

The cross compiler provides good code generation. For examples
of current cross compilers see:
  http://www.mpeforth.com/xc7.htm

The other approach, usually called Umbilical Forth or tethered Forth,
is to a use a desktop cross compiler that provides the full
interpreter and compiler and a target link and target monitor 
for code development and interactive debugging. Such systems can be
surprisingly powerful.

>With such a tight interdependency between compilation and execution,
>it's hard to see how standard Forth could be efficiently compiled
>statically in the general case. Of course a compiler could try to
>recognize easily-compilable (and presumably common) patterns, but I
>suspect such analysis to be very hard.
>
>Lisps have had exactly the same problem. The solution has been to
>manually separate the compile-time code (used by macros for syntax
>extensions) from the run-time code, originally with "eval-when"
>annotations, and later with module systems that automate the hurdles of
>phase separation. The phases can still share _code_, but they cannot
>affect each other's _state_.
>
>So how do offline native Forth compilers tackle the problem? Do they
>also require use of non-standard features in order to provide efficient
>compilation, or is there some other solution?

MPE and Forth Inc worked together on a large Forth project, and 
produced a set of proposals for cross-compiled Forths. The master
copies are usually on the Forth Inc web site. They were written by
Elizabeth Rather of Forth Inc. My copies are on the MPE web site:
  http://www.mpeforth.com/arena/XCtext5.PDF
  http://www.mpeforth.com/arena/XCapp5.PDF

Forth tries to insist that the various phases are accessible uniformly
at all times. Although it is possible to get close, in the limit you
have to separate the interpreted versions from the compiled versions.
In cross compiled systems, code to be interpreted is bracketed by

: foo ... ;
interpreter
: foo ... ;
target
foo  \ will execute interpreter version

It's a big topic. Leon and I will, I hope, get around to updating
the proposals. They date from the mid/late 1990s and reflect the 
cross compilers of that period. 

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#19229

Stephen Pelc wrote:
> Forth tries to insist that the various phases are accessible uniformly
> at all times. Although it is possible to get close, in the limit you
> have to separate the interpreted versions from the compiled versions.
> In cross compiled systems, code to be interpreted is bracketed by
> 
> : foo ... ;
> interpreter
> : foo ... ;
> target
> foo  \ will execute interpreter version
> 
> It's a big topic. Leon and I will, I hope, get around to updating
> the proposals. They date from the mid/late 1990s and reflect the
> cross compilers of that period.

Jens Wilke had implemented a quite nice idea to solve this problem: 
Compile to the host and the cross system in parallel, using some 
specially tweaked "primitives" for the host - usually, you have those, 
anyways, like @ and !, which address the cross compiler memory, but 
incomplete.  Unfortunately, he left some parts of the source code at 
G&D, so it didn't go into Gforth's cross compiler.

I.e. the host system emulates the cross system on Forth level.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19380

On Mon, 28 Jan 2013 17:19:56 -0000, Stephen Pelc <stephenXXX@mpeforth.com>  
wrote:
>
> MPE and Forth Inc worked together on a large Forth project, and
> produced a set of proposals for cross-compiled Forths. The master
> copies are usually on the Forth Inc web site. They were written by
> Elizabeth Rather of Forth Inc. My copies are on the MPE web site:
>   http://www.mpeforth.com/arena/XCtext5.PDF
>   http://www.mpeforth.com/arena/XCapp5.PDF

Yet, when we attempted to adopt these documents into the Forth 200x  
standard both MPE and Forth Inc., said "no, no, no, we would like to  
rewrite it first".  Thus the proposed Forth 2012 document contains only a  
passing reference to cross-compiling, which, true be told, could easily be  
removed without any detrimental affect. (2.1 Definitions of terms, A.3.4  
The Forth text interpreter and A.5 Compliance and labeling.)

-- 
Peter Knaggs

[toc] | [prev] | [next] | [standalone]

#19382

On Sun, 03 Feb 2013 10:50:40 -0000, "Peter Knaggs" <pjk@bcs.org.uk>
wrote:

>On Mon, 28 Jan 2013 17:19:56 -0000, Stephen Pelc <stephenXXX@mpeforth.com>  
>wrote:
>>
>> MPE and Forth Inc worked together on a large Forth project, and
>> produced a set of proposals for cross-compiled Forths. The master
>> copies are usually on the Forth Inc web site. They were written by
>> Elizabeth Rather of Forth Inc. My copies are on the MPE web site:
>>   http://www.mpeforth.com/arena/XCtext5.PDF
>>   http://www.mpeforth.com/arena/XCapp5.PDF
>
>Yet, when we attempted to adopt these documents into the Forth 200x  
>standard both MPE and Forth Inc., said "no, no, no, we would like to  
>rewrite it first".  Thus the proposed Forth 2012 document contains only a  
>passing reference to cross-compiling, which, true be told, could easily be  
>removed without any detrimental affect. (2.1 Definitions of terms, A.3.4  
>The Forth text interpreter and A.5 Compliance and labeling.)

When the cross-compiler documents were first written, both the MPE and
Forth Inc compilers used similar implementation techniques. Since that

time I have found or designed cross-compiler architectures for which
the proposed notation would be unnatural.

In addition, over the 15 years FI and MPE probably agree on the
meaning of the HOST and INTERPRETER scopes, but not on the
COMPILER scope.

Rewriting the documents to satisfy bothe FI and MPE will take a while
without considering other people with cross compilers.

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#19385

On 03.02.2013 13:11, Stephen Pelc wrote:
> In addition, over the 15 years FI and MPE probably agree on the
> meaning of the HOST and INTERPRETER scopes, but not on the
> COMPILER scope.
>
> Rewriting the documents to satisfy bothe FI and MPE will take a while
> without considering other people with cross compilers.
>
> Stephen
>

Then, when in decades nobody needed a Forth cross-compiler directive, 
what would be its payback now?

I do NOT want to sound discouraging, but given the limited amount of 
non-retired Forthers I'd rather vote for a simple generic reference 
implementation instead. F.ex. of a umbilical eforth system for 80% of 
the "hot" embedded processors.

But then you need to show a benefit over embedded Java or Lua, to name 
two. Forth "ideology" does not count here.

[toc] | [prev] | [next] | [standalone]

#19386

A. K. wrote:
> But then you need to show a benefit over embedded Java or Lua, to name
> two. Forth "ideology" does not count here.

This "ideology" thingy reminds me on a project a friend of mine had at 
Giesecke&Devrient.  They implemented a web server in a smart card, 
complete with SSL to have secure communication.  The SSL+TCP used some C 
libraries, the rest was in Forth; a total of three persons worked for a 
few months to get it running.

Then, the management came and said they don't want this Forth "ideology" 
and rather use embedded Java.  After 200 people tried for four years, 
they canceled the project.  All the time, they had a "running demo", as 
they called it, written in Forth.  The two employees who did that in 
Forth (the third one was a consultant) had long left the company for 
better opportunities.

The "ideology" here is that the mainstream tools are just as good, and 
that mediocre programmers are much better than good programmers, because 
you can get a dozen a dime (or so).  Gerald Wodny has a similar story: 
Two full-time indians are taking over of his job (he's doing 10h/week 
part time consulting for Honywell).  They are more expensive than he is.  
They don't get anything done.  This is not about Forth vs. something, 
because this is all done in .NET.  A good programmer simply is 20 times 
more efficient than a bad programmer.  And a good programmer is one who 
choses his tools as an informed decision.  The uninformed management 
likes to call that "ideology" if it doesn't fit with their own ideology 
(judging from themselves, therefore, any decision must be ideology).

In any case: a bad programmer in Forth is usually even worse than in Lua 
or Java.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19394

On 2013-02-03, Bernd Paysan <bernd.paysan@gmx.de> wrote:

> In any case: a bad programmer in Forth is usually even worse than in Lua 
> or Java.
>
Nothing beats bad management. 

I was once in project which very limited memory. Sorry. No Forth in this post.

My working solution in C was something like this in C
struct datastorage{
  unsigned int data;
  srtuct datastorage * next;
};

and then some functions(datastorage * s, char a); accessing parts of 
that unsigned int with data checks.

I knew my data limits and most of them were below 10. So 4 bits was 
enough every needed bit of information and everything worked.

After damagement reviewed the code, it was rewritten to
struct datastorage{
  unsigned int a;
  ...
  unsigned int k;
  struct datastorage * next;
}
And no checks for data was done.

That lead three things. First was the system which was out of memory, 
secondly the hardware was changed to have more memory. Third I gave 
ultimatum of that stupidity and I have to leave that project.

Hannu

[toc] | [prev] | [next] | [standalone]

#19466

On Sunday, February 3, 2013 8:59:22 AM UTC-5, A. K. wrote:

> Then, when in decades nobody needed a Forth cross-compiler directive, 
> what would be its payback now?

> I do NOT want to sound discouraging, but given the limited amount of 
> non-retired Forthers I'd rather vote for a simple generic reference 
> implementation instead. F.ex. of a umbilical eforth system for 80% of 
> the "hot" embedded processors.

I for one would welcome a standard.  My cross-compiler follows the old documents but there are bits that needed updating.  Having the major vendors and the general community agree allows programmers to move from one system to another with minimal mistakes.  
My work is 100% embedded, typically on small processors.  The cross-compilation process, and handling the assorted vocabularies needed for this this sort of thing, is not trivial.  It can be time consuming until one becomes facile with the system. Being able to use a variety of different tools/compilers without a large learning curve would be a plus.

Gary

[toc] | [prev] | [next] | [standalone]

#19426

"Peter Knaggs" <pjk@bcs.org.uk> writes:
>On Mon, 28 Jan 2013 17:19:56 -0000, Stephen Pelc <stephenXXX@mpeforth.com>  
>wrote:
>>
>> MPE and Forth Inc worked together on a large Forth project, and
>> produced a set of proposals for cross-compiled Forths. The master
>> copies are usually on the Forth Inc web site. They were written by
>> Elizabeth Rather of Forth Inc. My copies are on the MPE web site:
>>   http://www.mpeforth.com/arena/XCtext5.PDF
>>   http://www.mpeforth.com/arena/XCapp5.PDF
>
>Yet, when we attempted to adopt these documents into the Forth 200x  
>standard both MPE and Forth Inc., said "no, no, no, we would like to  
>rewrite it first".  Thus the proposed Forth 2012 document contains only a  
>passing reference to cross-compiling

Hmm, given that there are apparently still changes in the way that
cross-compiling is done (otherwise that text would not have to be
rewritten), it's probably too early to standardize cross-compilation.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2013: http://www.euroforth.org/ef13/

[toc] | [prev] | [next] | [standalone]

#19230

On 1/28/13 1:54 AM, Lauri Alanko wrote:
> Hello.
>
> I recently started getting acquainted with Forth. I'm quite impressed
> that such a simple core language can provide Lisp-like syntax
> manipulation power. However, I'm wondering about how this power
> constrains the implementation options.
>
> Forth is often used in embedded systems. I'm not very familiar with
> them, but I'd expect resources to be so scarce that cross-compilation to
> the target's native code would be the preferred implementation
> technique: interpretation would waste cycles on an already slow
> microcontroller, and there wouldn't be enough memory for a native
> non-cross-compiler, JIT or otherwise.

I have been involved with Forth since its development in the early 70's. 
In those days, Forth ran standalone (no host OS) on a "minicomputer" 
with 32 Kbytes of memory and a 1.25 Mb hard drive. This computer 
supported four users on separate "dumb terminals" one of which had 
graphic capabilities. At the same time, it was controlling a 36" radio 
telescope, controlling its motion and pointing accuracy to 0.1 degrees 
of arc, moving the observatory dome as necessary, and doing data 
acquisition at 1 kHz. Response time on the terminals was instantaneous.

The implementation strategy in use was ITC ("Indirect Threaded Code"), 
which was the most common implementation strategy for Forth until the 
1990's. The ITC compiler constructs definitions as sequences of absolute 
addresses of other definitions. These sequences of addresses are 
"interpreted" by a bit of code which could be as little as 2 machine 
instructions per address. On this machine (a PDP-11) that was slightly 
faster than a subroutine call instruction.

The system also supported a resident assembler for the host processor, 
so you could define primitives as desired, but typically there were only 
20-30 such primitives necessary.

So, you can see that the efficiency of Forth supports a very high degree 
of performance even on very primitive hardware.

> However, standard Forth is specified as an interpreted language, where
> execution semantics and compilation semantics can be interleaved, each
> affecting the other. For instance:
>
> variable counter
> 0 counter !
> : inc counter @ dup 1+ counter ! ;
> inc .
> : foo inc postpone literal ; immediate
> : bar foo . ;
> inc .
> bar
> bye
>
> With such a tight interdependency between compilation and execution,
> it's hard to see how standard Forth could be efficiently compiled
> statically in the general case. Of course a compiler could try to
> recognize easily-compilable (and presumably common) patterns, but I
> suspect such analysis to be very hard.

As I described above, the "compiler" was a great deal simpler than 
optimizing compilers that you may be familiar with. It was always 
resident, in order to support the kind of interaction in your example.

> Lisps have had exactly the same problem. The solution has been to
> manually separate the compile-time code (used by macros for syntax
> extensions) from the run-time code, originally with "eval-when"
> annotations, and later with module systems that automate the hurdles of
> phase separation. The phases can still share _code_, but they cannot
> affect each other's _state_.

I am not familiar with Lisp, although I understand that it is much 
slower than Forth.

> So how do offline native Forth compilers tackle the problem? Do they
> also require use of non-standard features in order to provide efficient
> compilation, or is there some other solution?

I am not sure what you mean by a "non-standard feature". I have 
described the traditional Forth ITC implementation. As Stephen Pelc 
describes, since the 1990s Forths have been developed using more 
classical compiler techniques that generate optimized machine code. As 
you might expect, these compilers are a good deal larger than the simple 
ITC compiler. They are used on Forths that run under OSs such as **nix 
and Windows, and for embedded systems they are usually implemented as 
cross-compilers for target systems that are connected to the host with 
an umbilical for interactive development that looks much like your example.

Cheers,
Elizabeth


-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#19240

Lauri Alanko <la@iki.fi> writes:
>Forth is often used in embedded systems. I'm not very familiar with
>them, but I'd expect resources to be so scarce that cross-compilation to
>the target's native code would be the preferred implementation
>technique: interpretation would waste cycles on an already slow
>microcontroller, and there wouldn't be enough memory for a native
>non-cross-compiler, JIT or otherwise.

Simple native-code compilers can be quite small.  Maybe Bernd Paysan
can provide numbers for bigForth.  It just copies a bit of native code
for every word that is compiled, and then performs some peephole
optimization; and the table for peephole optimization is not that big,
either, AFAIK.

>However, standard Forth is specified as an interpreted language, where
>execution semantics and compilation semantics can be interleaved, each
>affecting the other. For instance:
>
>variable counter
>0 counter !
>: inc counter @ dup 1+ counter ! ;
>inc .
>: foo inc postpone literal ; immediate
>: bar foo . ;
>inc .
>bar
>bye
>
>With such a tight interdependency between compilation and execution,
>it's hard to see how standard Forth could be efficiently compiled
>statically in the general case.

I assume that with "statically" you mean that there is a hard line
between compilation and run-time.  Forth cross compilers have that,
and yes, if you use a typical cross compiler, you cannot draw the line
before the last colon definition in the example above.  So yes, cross
compilers generally do not work on all standard Forth programs.

Such code does not appear to be that common, and maybe the limitations
of cross-compilers are a contributing factor.

The other alternative is to have some kind of code generator in the
target, even if it produces threaded code.  I don't think that a
different code generation strategy has been used for Forth, but the
GNU Java implementation used native code (GCC-based) for "static code"
(before the line) and a bytecode interpreter (GIJ) for "dynamic code"
(after the line).

>So how do offline native Forth compilers tackle the problem? Do they
>also require use of non-standard features in order to provide efficient
>compilation, or is there some other solution?

Cross-compilers just don't support the features that would require too
much memory to implement.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2012: http://www.euroforth.org/ef12/

[toc] | [prev] | [next] | [standalone]

#19252

Anton Ertl wrote:

> Lauri Alanko <la@iki.fi> writes:
>>Forth is often used in embedded systems. I'm not very familiar with
>>them, but I'd expect resources to be so scarce that cross-compilation
>>to the target's native code would be the preferred implementation
>>technique: interpretation would waste cycles on an already slow
>>microcontroller, and there wouldn't be enough memory for a native
>>non-cross-compiler, JIT or otherwise.
> 
> Simple native-code compilers can be quite small.  Maybe Bernd Paysan
> can provide numbers for bigForth.  It just copies a bit of native code
> for every word that is compiled, and then performs some peephole
> optimization; and the table for peephole optimization is not that big,
> either, AFAIK.

Yes, the copy-macro stuff is about 4 screens long, and the table itself 
11 screens, or as compiled table 1332 bytes.  I would put it into an 
embedded Forth (e.g. ColdFire or such, which would be an easy porting 
target for the 68k version of bigForth) today, because the space saved 
by the peephole compression is worth more than the peephole table and 
the more complicated COMPILE, itself.

An analytic native code compiler is in the order of 5000 lines of code.  
Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is 
about 5000, as is iForth.  Such a compiler probably won't break even on 
a small embedded system with less than 64k RAM, even though the analytic 
compilation reduces space as well - at least possibly (depending on how 
aggressive you inline - the "space constrained" rule would be "if 
inlining leads to more compact code than calling").

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19279

Bernd Paysan <bernd.paysan@gmx.de> writes:
> An analytic native code compiler is in the order of 5000 lines of code.  
> Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is 
> about 5000, as is iForth.  Such a compiler probably won't break even on 
> a small embedded system with less than 64k RAM, 

I'd hope most of the compiler could run from program flash rather than
ram, on systems where those are separate.  Example: I have a couple of
the TI Stellaris Launchpad (ARM Cortex M4) boards from their promotion
last year.  The cpu has 256k of program flash and 32K of RAM, which
seems squarely in Forth's comfort zone, so I'd been thinking of porting
an interpreter.  It hadn't occurred to me before, but maybe one of those
fancy compilers could actually fit in there.

[toc] | [prev] | [next] | [standalone]

#19280

On Wed, 30 Jan 2013 00:42:20 -0800, Paul Rubin
<no.email@nospam.invalid> wrote:

>I'd hope most of the compiler could run from program flash rather than
>ram, on systems where those are separate.  Example: I have a couple of
>the TI Stellaris Launchpad (ARM Cortex M4) boards from their promotion
>last year.  The cpu has 256k of program flash and 32K of RAM, which
>seems squarely in Forth's comfort zone, so I'd been thinking of porting
>an interpreter.  It hadn't occurred to me before, but maybe one of those
>fancy compilers could actually fit in there.

Yes, they could fit in the flash. VFX needs about 3 kb of RAM. But in
todays micros, the limiting resource is RAM. For the applications we
see, the authors use all the RAM long before they use all the Flash.
Losing 10% of the RAM for a WIBNI (Wouldn't It Be Nice If ...) is
not a good trade-off.

The question is why would you need one of those "fancy compilers" in
a deep embedded application. For those embedded applications which did
use a simple Forth compiler on the target, compilation speed and
performance of the code compiled by the target were never an issue.

We already know that VFX code generators are cross compilable, since
the x86 version is cross-compiled as part of the hosted VFX Forth
kernels.

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#19305

On Jan 30, 8:42 am, Paul Rubin <no.em...@nospam.invalid> wrote:
> Bernd Paysan <bernd.pay...@gmx.de> writes:
> > An analytic native code compiler is in the order of 5000 lines of code.
> > Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is
> > about 5000, as is iForth.  Such a compiler probably won't break even on
> > a small embedded system with less than 64k RAM,
>
> I'd hope most of the compiler could run from program flash rather than
> ram, on systems where those are separate.  Example: I have a couple of
> the TI Stellaris Launchpad (ARM Cortex M4) boards from their promotion
> last year.  The cpu has 256k of program flash and 32K of RAM, which
> seems squarely in Forth's comfort zone, so I'd been thinking of porting
> an interpreter.  It hadn't occurred to me before, but maybe one of those
> fancy compilers could actually fit in there.

An embedded box wouldn't really need its own compiler. That's what PCs
are for ;-)

[toc] | [prev] | [next] | [standalone]

#19311

In article <d8cd6236-871c-415e-92bd-26224bc5d206@m12g2000yqp.googlegroups.com>,
Mark Wills  <forthfreak@gmail.com> wrote:
>On Jan 30, 8:42 am, Paul Rubin <no.em...@nospam.invalid> wrote:
>> Bernd Paysan <bernd.pay...@gmx.de> writes:
>> > An analytic native code compiler is in the order of 5000 lines of code.
>> > Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is
>> > about 5000, as is iForth.  Such a compiler probably won't break even on
>> > a small embedded system with less than 64k RAM,
>>
>> I'd hope most of the compiler could run from program flash rather than
>> ram, on systems where those are separate.  Example: I have a couple of
>> the TI Stellaris Launchpad (ARM Cortex M4) boards from their promotion
>> last year.  The cpu has 256k of program flash and 32K of RAM, which
>> seems squarely in Forth's comfort zone, so I'd been thinking of porting
>> an interpreter.  It hadn't occurred to me before, but maybe one of those
>> fancy compilers could actually fit in there.
>
>An embedded box wouldn't really need its own compiler. That's what PCs
>are for ;-)

Well, if it comes to that. Humans don't really need PCs, or embedded
computers.

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#19379

Albert van der Horst wrote:
> In article <d8cd6236-871c-415e-92bd-26224bc5d206@m12g2000yqp.googlegroups.com>,
> Mark Wills  <forthfreak@gmail.com> wrote:
> ...
> >An embedded box wouldn't really need its own compiler. That's what PCs
> >are for ;-)
>
> Well, if it comes to that. Humans don't really need PCs, or embedded
> computers.

How long can you keep away from your PC ?  In addition, governments,
utilities, banks, schools etc all demand that you use them.  PC's are the
best tool the Devil has invented.  Legend has it that Sin began with Jobs'
first byte-ing of the Apple.

[toc] | [prev] | [next] | [standalone]

#19747

On Saturday, February 2, 2013 7:49:49 PM UTC-6, Ed wrote:
> Albert van der Horst wrote:
> 
> > In article <d8cd6236-871c-415e-92bd-26224bc5d206@m12g2000yqp.googlegroups.com>,
> 
> > Mark Wills  <forthfreak@gmail.com> wrote:
> 
> > ...
> 
> > >An embedded box wouldn't really need its own compiler. That's what PCs
> 
> > >are for ;-)
> 
> >
> 
> > Well, if it comes to that. Humans don't really need PCs, or embedded
> 
> > computers.
> 
> 
> 
> How long can you keep away from your PC ?  In addition, governments,
> 
> utilities, banks, schools etc all demand that you use them.  PC's are the
> 
> best tool the Devil has invented.  Legend has it that Sin began with Jobs'
> 
> first byte-ing of the Apple.

Nice!

[toc] | [prev] | [next] | [standalone]

#19406

Bernd Paysan <bernd.paysan@gmx.de> wrote Re: Offline compilation of Forth
[..]
> An analytic native code compiler is in the order of 5000 lines of code.  
> Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is 
> about 5000, as is iForth.  

Where did you get that number? The iForth metacompiler processes 91,936
LoCs to generate the Linux image (see #CLINES in the on-line manual).

The largest source file is the target-dependent optimizer -- it has 5,662 
LoCs, but doesn't contain the floating-point optimizer and the first 
stage macro's.

-marcel

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | comp.lang.forth

csiph-web