Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #19218 > unrolled thread

Offline compilation of Forth

Back to article view | Back to comp.lang.forth

Contents

  Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-01-28 11:54 +0000
    Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-01-28 17:19 +0000
      Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-01-28 22:08 +0100
      Re: Offline compilation of Forth "Peter Knaggs" <pjk@bcs.org.uk> - 2013-02-03 10:50 +0000
        Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 12:11 +0000
          Re: Offline compilation of Forth "A. K." <akk@nospam.org> - 2013-02-03 14:59 +0100
            Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-03 15:14 +0100
              Re: Offline compilation of Forth Hannu Vuolasaho <hannu.vuolasaho@nospam.tut.fi.invalid> - 2013-02-03 15:25 +0000
            Re: Offline compilation of Forth Gary Bergstrom <g.bergstrom@ieee.org> - 2013-02-05 08:13 -0800
        Re: Offline compilation of Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-02-04 16:23 +0000
    Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-28 12:26 -1000
    Re: Offline compilation of Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-01-29 08:55 +0000
      Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-01-29 17:49 +0100
        Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-01-30 00:42 -0800
          Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-01-30 11:58 +0000
          Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-01-30 23:16 -0800
            Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-01-31 13:29 +0000
              Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-03 12:49 +1100
                Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 20:37 -0800
        Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-03 21:15 +0200
          Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-04 02:04 +0100
        Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-04 13:22 +0000
          Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-04 20:07 +0100
            Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-04 23:07 +0200
              Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-05 18:33 +0100
                Re: Offline compilation of Forth mhx@iae.nl (Marcel Hendrix) - 2013-02-06 22:53 +0200
                  Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-07 00:30 +0100
      Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-01-30 00:36 -0800
    Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-01-30 18:12 +0000
      Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-30 09:36 -1000
      Re: Offline compilation of Forth "A. K." <akk@nospam.org> - 2013-01-31 07:45 +0100
      Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-01-31 03:37 -0600
        Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-02-01 00:01 +0000
          Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 14:59 -1000
            Re: Offline compilation of Forth Lauri Alanko <la@iki.fi> - 2013-02-05 15:17 +0000
              Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-02-05 10:46 -0600
                Re: Offline compilation of Forth Bernd Paysan <bernd.paysan@gmx.de> - 2013-02-05 18:47 +0100
      Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-01-31 21:47 +1100
        Re: Offline compilation of Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-01-31 06:12 -0600
          Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-01-31 13:32 +0000
          Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 08:27 -1000
            Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-01 10:23 +1100
              Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 13:53 -1000
                Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-02-01 03:17 +0000
                  Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 18:43 -1000
                  Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-01 08:45 -0800
                Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-01-31 23:05 -0800
                  Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-01-31 21:25 -1000
                  Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-01 08:55 -0800
                    Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-01 09:18 -1000
                      Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-02-01 13:04 -0800
                Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-03 11:20 +1100
                  Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 12:20 +0000
                    Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 07:22 -0800
                      Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-03 17:54 +0000
                        Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 10:17 -0800
                        Re: Offline compilation of Forth albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-02-03 19:33 +0000
                          Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-03 11:53 -0800
                            Re: Offline compilation of Forth Coos Haak <chforth@hccnet.nl> - 2013-02-04 00:59 +0100
                          Re: Offline compilation of Forth Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2013-02-04 11:38 +0100
                            Re: Offline compilation of Forth Paul Rubin <no.email@nospam.invalid> - 2013-02-24 21:26 -0800
                              Re: Offline compilation of Forth Mark Wills <markrobertwills@yahoo.co.uk> - 2013-02-24 23:56 -0800
                              Re: Offline compilation of Forth Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2013-02-25 12:08 +0100
                                Re: Offline compilation of Forth Mark Wills <markrobertwills@yahoo.co.uk> - 2013-02-25 04:31 -0800
                                  Re: Offline compilation of Forth Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2013-02-25 15:54 +0100
                        Re: Offline compilation of Forth Brad Eckert <hwfwguy@gmail.com> - 2013-02-06 09:13 -0800
                          Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 21:02 -0800
                        Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 20:55 -0800
                      Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-03 08:26 -1000
                    Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-05 13:04 +1100
                      Re: Offline compilation of Forth stephenXXX@mpeforth.com (Stephen Pelc) - 2013-02-05 10:33 +0000
                        Re: Offline compilation of Forth "Elizabeth D. Rather" <erather@forth.com> - 2013-02-05 08:56 -1000
                          Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-08 02:07 +1100
                            Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-07 19:03 +0100
                              Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-09 22:45 +1100
                                Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-09 18:56 +0100
                                  Re: Offline compilation of Forth Roelf Toxopeus <rt4all@notthis.hetnet.nl> - 2013-02-11 00:19 +0100
                                Re: Offline compilation of Forth Alex McDonald <blog@rivadpm.com> - 2013-02-09 11:59 -0800
                                  Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-13 13:13 +1100
                                    Re: Offline compilation of Forth Alex McDonald <blog@rivadpm.com> - 2013-02-12 19:19 -0800
                          Re: Offline compilation of Forth "Clyde W. Phillips Jr." <cwpjr02@gmail.com> - 2013-02-14 21:01 -0800
        Re: Offline compilation of Forth Roberto Waltman <usenet@rwaltman.com> - 2013-02-03 16:26 -0500
          Re: Offline compilation of Forth "Ed" <invalid@nospam.com> - 2013-02-05 15:32 +1100
            Re: Offline compilation of Forth Mark Wills <forthfreak@gmail.com> - 2013-02-04 23:18 -0800

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

#19413

Marcel Hendrix wrote:

> Bernd Paysan <bernd.paysan@gmx.de> wrote Re: Offline compilation of
> Forth
> [..]
>> An analytic native code compiler is in the order of 5000 lines of
>> code. Lars Krüger's is 4000 LoCs (that includes an assembler), VFX
>> AFAIK is about 5000, as is iForth.
> 
> Where did you get that number? The iForth metacompiler processes
> 91,936 LoCs to generate the Linux image (see #CLINES in the on-line
> manual).

What of these 91kLoCs is actual analytic native code compiler, and not 
stuff that would be in your Forth system anyways?

> The largest source file is the target-dependent optimizer -- it has
> 5,662 LoCs, but doesn't contain the floating-point optimizer and the
> first stage macro's.

Ok, so iForth is considerably more.  But I didn't include the floating 
point stuff in the VFX, either.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19420

On Tue, 29 Jan 2013 17:49:24 +0100, Bernd Paysan <bernd.paysan@gmx.de>
wrote:

>An analytic native code compiler is in the order of 5000 lines of code.  
>Lars Krüger's is 4000 LoCs (that includes an assembler), VFX AFAIK is 
>about 5000, as is iForth.

The primary VFX code generator file (no floating point) is about 
6900 lines. The assembler (with FP) is required and is about 4000
lines. The disassemblers are about 2500 lines.

You should also not that these files include the documentation that
ends up in the VFX Forth manual.

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[toc] | [prev] | [next] | [standalone]

#19430

Stephen Pelc wrote:

> On Tue, 29 Jan 2013 17:49:24 +0100, Bernd Paysan <bernd.paysan@gmx.de>
> wrote:
> 
>>An analytic native code compiler is in the order of 5000 lines of
>>code. Lars Krüger's is 4000 LoCs (that includes an assembler), VFX
>>AFAIK is about 5000, as is iForth.
> 
> The primary VFX code generator file (no floating point) is about
> 6900 lines. The assembler (with FP) is required and is about 4000
> lines. The disassemblers are about 2500 lines.
> 
> You should also not that these files include the documentation that
> ends up in the VFX Forth manual.

4000 lines for an x86 assembler?  My bigForth assembler is 630 lines, 
lacking SSE stuff, though, but it's written to be deliberately short.  
Lars' is 1000 lines.

When we count source code on a apple:apple base, we probably should only 
count the sources as such, not the comments, not the documentation.  I'm 
quite sure, Marcel also counts the documentation, while Lars Krüger 
didn't include much documentation in his source code.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19434

Bernd Paysan <bernd.paysan@gmx.de> writes Re: Offline compilation of Forth
>> Stephen Pelc wrote:
[..]
>> The primary VFX code generator file (no floating point) is about
>> 6900 lines. The assembler (with FP) is required and is about 4000
>> lines. The disassemblers are about 2500 lines.
>> 
>> You should also not that these files include the documentation that
>> ends up in the VFX Forth manual.

> 4000 lines for an x86 assembler?  My bigForth assembler is 630 lines, 
> lacking SSE stuff, though, but it's written to be deliberately short.  
> Lars' is 1000 lines.

That remark has only value as a personal comment or opinion.

I guess the VfxForth assembler must fullfil the expectations of MPE's
customers, which probably means all 16/32 bit instructions minus the
protected mode and SSE/SSE2 stuff.

The iForth assembler (428 lines framework + 2305 lines x86_64 specific 
including comments) has all 64/32 instructions and a few 16 bit ones, 
no protected mode ones, all FPU instructions, SSE/SSE2. The disassembler 
(1879 lines) is a bit more powerful than the assembler.

> When we count source code on a apple:apple base, we probably should only 
> count the sources as such, not the comments, not the documentation.  I'm 
> quite sure, Marcel also counts the documentation, while Lars Krüger 
> didn't include much documentation in his source code.

It is much easier to count all the lines than the lines of actual code.

-marcel

[toc] | [prev] | [next] | [standalone]

#19472

Marcel Hendrix wrote:

> Bernd Paysan <bernd.paysan@gmx.de> writes Re: Offline compilation of
> Forth
>> 4000 lines for an x86 assembler?  My bigForth assembler is 630 lines,
>> lacking SSE stuff, though, but it's written to be deliberately short.
>> Lars' is 1000 lines.
> 
> That remark has only value as a personal comment or opinion.

Yes.  I think that shorter programs are both easier to write, to 
maintain, and to debug as longer programs, even when everything else is 
the same.  Apparently, this includes the amount of documentation you 
need: David Kühling (see below) added SSE/SSE2 to my very terse 
assembler, using the same style, and he didn't need to ask me anything.  
All opcodes and registers have the same name as in Intel's manuals, so 
the only documentation needed is how to transform them to postfix style.  
That's a page or so.

> I guess the VfxForth assembler must fullfil the expectations of MPE's
> customers, which probably means all 16/32 bit instructions minus the
> protected mode and SSE/SSE2 stuff.
> 
> The iForth assembler (428 lines framework + 2305 lines x86_64 specific
> including comments) has all 64/32 instructions and a few 16 bit ones,
> no protected mode ones, all FPU instructions, SSE/SSE2. The
> disassembler (1879 lines) is a bit more powerful than the assembler.

For x64, having SSE/SSE2 is indeed a must.  David Kühling added SSE/SSE2 
to the x64 version of my assembler, and it's in total now 730 lines.  
What's now missing are only recent additions like AVX and AES-NI.

My disassembler is even smaller than the assembler... generating Intel 
syntax.

> It is much easier to count all the lines than the lines of actual
> code.

Yes.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19518

Bernd Paysan <bernd.paysan@gmx.de> writes Re: Offline compilation of Forth
[..]
> For x64, having SSE/SSE2 is indeed a must.  David Kühling added SSE/SSE2 
> to the x64 version of my assembler, and it's in total now 730 lines.  
> What's now missing are only recent additions like AVX and AES-NI.

Can you give a hint where to find it? The bigForth distribution
(sourceforge/subversion) shows assem486.fb (a screen file) which only seems to have a 
handful of 3Dnow and mmx words (no SSE/SSE2).

I am very curious how David got a full SSE/SSE2 assembler in 100 lines,
given that most instructions have tiny abnormalities and never quite the
same addressing possibilities. For example, the cvt(t)xx2yyy instruction 
needs 50 lines just to enter all the different opcode name varieties
(the SSE/SSE2 part, covering all instructions, is 510 lines in total).

-marcel

[toc] | [prev] | [next] | [standalone]

#19519

Marcel Hendrix wrote:

> Bernd Paysan <bernd.paysan@gmx.de> writes Re: Offline compilation of
> Forth
> [..]
>> For x64, having SSE/SSE2 is indeed a must.  David Kühling added
>> SSE/SSE2 to the x64 version of my assembler, and it's in total now
>> 730 lines. What's now missing are only recent additions like AVX and
>> AES-NI.
> 
> Can you give a hint where to find it? The bigForth distribution
> (sourceforge/subversion) shows assem486.fb (a screen file) which only
> seems to have a handful of 3Dnow and mmx words (no SSE/SSE2).

It's in Gforth, arch/amd64/asm.fs.  I should really backport this to 
bigForth, when I find time for it.

> I am very curious how David got a full SSE/SSE2 assembler in 100
> lines, given that most instructions have tiny abnormalities and never
> quite the same addressing possibilities.

By using the Forth principle that you don't have to solve all problems 
for the programmer, like "if you use the instruction wrong, it will 
assemble something that follows the logic of the instruction encoding, 
but the CPU won't like it".

> For example, the cvt(t)xx2yyy
> instruction needs 50 lines just to enter all the different opcode name
> varieties (the SSE/SSE2 part, covering all instructions, is 510 lines
> in total).

We have 14 variants, parts are covered by size prefixes (that's the way 
bigForth's assembler works).  By looking through the opcodes, comparing 
with

http://softpixel.com/~cwright/programming/simd/sse2.php

this seems not to be fully complete.  But adding the remaining 
instructions in the same way would not result in an awful lot of code.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19278

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> The other alternative is to have some kind of code generator in the
> target, even if it produces threaded code.  ... the GNU Java
> implementation used native code (GCC-based) for "static code" (before
> the line) and a bytecode interpreter (GIJ) for "dynamic code" (after
> the line).

That approach has also been used for Lisp, to deal with the "eval"
function, which lets you generate and execute code at runtime.  Of
course some Forth targets are too small for this to be practical.  I've
been thinking about cross-compilation because of the TI MSP430 Launchpad
board, which comes with two plug-in cpus.  The bigger cpu has 16k of
program flash and 512 bytes of ram and runs a resident Forth interpreter
(4e4th.eu) with reasonable space for user code, but the smaller one has
just 2k of flash and 128 bytes of ram, which would seem to call for a
cross-compiler.

[toc] | [prev] | [next] | [standalone]

#19289

Thanks to everyone for the responses. I'm overjoyed to find a newsgroup
that is still thriving so well!

The "Cross-compiler word set" seems to be pretty much exactly what I was
looking for: explicit phase annotations to separate between run-time and
compile-time code. I'm not sure about its naming, though: to my mind the
essential thing is not so much that the target program is run on a
different architecture, but that it is run at a different _time_, and
thus cannot access compile-time state. So the proposal seems useful even
when compiling for the host architecture.

Indeed, I don't quite understand the recommendation that this word set
shouldn't be supported outside cross-compilers. Compiling independent
applications seems like a useful feature in general, and even when an
implementation doesn't support phase separation, no-op scope words would
serve as useful annotation to the reader of the code to distinguish the
meta-programming parts of the code.

Even without phases, the namespace separation that the proposal
introduces between interpretation, execution and immediate words might
be useful even in "normal" Forth. Currently there are words with
undefined interpretation semantics. Surely it would be cleaner if those
words were not even visible in interpretation state?

The paper doesn't explicate how one might share common utility code that
could be useful both at compile-time and at run-time. Presumable one
would just place it in a separate file and INCLUDE it twice in different
scopes?


I think the technical issues regarding implementation on low-end
hardware are fairly minor. Sure, a JIT will take a bit of space and
bytecode will have a bit of overhead in both time and energy, but I'm
willing to believe these are within reasonable limits. A microprocessor,
after all, won't have huge pipelines to flush at every indirect jump.

Still, I would expect embedded programming to be so resource-tight that
even these small costs would occasionally be unacceptable. Perhaps this
is mostly a cultural issue: is the run-time interpreter seen to be such
an essential part of Forth that it cannot be left out in the name of
efficiency even from deployed production versions of the program? I know
no other language where QUIT means "begin interaction". :)


Lauri

[toc] | [prev] | [next] | [standalone]

#19291

On 1/30/13 8:12 AM, Lauri Alanko wrote:
> Thanks to everyone for the responses. I'm overjoyed to find a newsgroup
> that is still thriving so well!
>
> The "Cross-compiler word set" seems to be pretty much exactly what I was
> looking for: explicit phase annotations to separate between run-time and
> compile-time code. I'm not sure about its naming, though: to my mind the
> essential thing is not so much that the target program is run on a
> different architecture, but that it is run at a different _time_, and
> thus cannot access compile-time state. So the proposal seems useful even
> when compiling for the host architecture.
>
> Indeed, I don't quite understand the recommendation that this word set
> shouldn't be supported outside cross-compilers. Compiling independent
> applications seems like a useful feature in general, and even when an
> implementation doesn't support phase separation, no-op scope words would
> serve as useful annotation to the reader of the code to distinguish the
> meta-programming parts of the code.

Traditionally, operational Forth programs include the entire system, 
including the compiler and other development tools. There are simple 
mechanisms for preventing access to these for security reasons if 
desired. The benefit is that by removing the separate 
"edit-compile-link-load-test" steps you achieve a much more intimate 
relationship between the programmer and code under development, which 
greatly facilitates the programming process.

Most resident Forth systems include a provision for making a 
distributable turnkey version of the system+application, which may or 
may not include access to the programming tools. Since Forth generates 
such compact programs, these programs are usually still much smaller 
than comparable capabilities generated by more conventional languages.

> Even without phases, the namespace separation that the proposal
> introduces between interpretation, execution and immediate words might
> be useful even in "normal" Forth. Currently there are words with
> undefined interpretation semantics. Surely it would be cleaner if those
> words were not even visible in interpretation state?
>
> The paper doesn't explicate how one might share common utility code that
> could be useful both at compile-time and at run-time. Presumable one
> would just place it in a separate file and INCLUDE it twice in different
> scopes?

Well, if you're talking about different host and target processors, that 
code isn't really sharable, is it? In the resident programming 
environment, such utilities are naturally shared.

> I think the technical issues regarding implementation on low-end
> hardware are fairly minor. Sure, a JIT will take a bit of space and
> bytecode will have a bit of overhead in both time and energy, but I'm
> willing to believe these are within reasonable limits. A microprocessor,
> after all, won't have huge pipelines to flush at every indirect jump.
>
> Still, I would expect embedded programming to be so resource-tight that
> even these small costs would occasionally be unacceptable. Perhaps this
> is mostly a cultural issue: is the run-time interpreter seen to be such
> an essential part of Forth that it cannot be left out in the name of
> efficiency even from deployed production versions of the program? I know
> no other language where QUIT means "begin interaction". :)

"Embedded systems" vary considerably, from environments with 8-bit 
microcontrollers where every byte counts to 32-bit (or even 64-bit) 
processors with megabytes of memory and a host OS. Forth is scalable, 
with tools and versions for the entire range, though they are likely 
*different* tools and versions.

The expectation that you will have a resident Forth that is interactive 
is obviously inappropriate in a microcontroller managing an automobile's 
ignition, for example. Such a program would be cross-compiled and would 
not contain any provision for interactivity, omitting not only the 
interpreter and compiler but also even the heads of dictionary entries. 
It's quite possible to cross-compile a reasonable application in under 1K.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#19302

On 30.01.2013 19:12, Lauri Alanko wrote:
> Thanks to everyone for the responses. I'm overjoyed to find a newsgroup
> that is still thriving so well!
>
> The "Cross-compiler word set" seems to be pretty much exactly what I was
> looking for: explicit phase annotations to separate between run-time and
> compile-time code. I'm not sure about its naming, though: to my mind the
> essential thing is not so much that the target program is run on a
> different architecture, but that it is run at a different _time_, and
> thus cannot access compile-time state. So the proposal seems useful even
> when compiling for the host architecture.

We used not exactly a cross-compiler, but a small C program that 
generated a bytecode image in one pass from Forth source text files. 
This was done offline on a PC.

The target system (a controller card) hosted the Forth bytecode 
interpreter to run the uploaded image.

[toc] | [prev] | [next] | [standalone]

#19306

Lauri Alanko <la@iki.fi> wrote:
> Thanks to everyone for the responses. I'm overjoyed to find a newsgroup
> that is still thriving so well!
> 
> The "Cross-compiler word set" seems to be pretty much exactly what I
> was looking for: explicit phase annotations to separate between
> run-time and compile-time code. I'm not sure about its naming,
> though: to my mind the essential thing is not so much that the
> target program is run on a different architecture, but that it is
> run at a different _time_, and thus cannot access compile-time
> state. So the proposal seems useful even when compiling for the host
> architecture.
> 
> Indeed, I don't quite understand the recommendation that this word
> set shouldn't be supported outside cross-compilers. Compiling
> independent applications seems like a useful feature in general, and
> even when an implementation doesn't support phase separation, no-op
> scope words would serve as useful annotation to the reader of the
> code to distinguish the meta-programming parts of the code.
> 
> Even without phases, the namespace separation that the proposal
> introduces between interpretation, execution and immediate words
> might be useful even in "normal" Forth.

Probably not.  It's not rare in Forth for compiling words to generate
code that is more compiling words that generate code ... you get the
idea.  Where is the runtime/compile-time split then?  Cross-compilers
force you to have a hard boundary, and this results in a less powerful
language.

> Currently there are words with undefined interpretation
> semantics. Surely it would be cleaner if those words were not even
> visible in interpretation state?

Not really.  They might be defined on some systems.

Andrew.

[toc] | [prev] | [next] | [standalone]

#19335

In article <M8udnV_U7KPNppfMnZ2dnUVZ_rudnZ2d@supernews.com>,
Andrew Haley  <andrew29@littlepinkcloud.invalid> wrote:
> > Even without phases, the namespace separation that the proposal
> > introduces between interpretation, execution and immediate words
> > might be useful even in "normal" Forth.
> 
> Probably not.  It's not rare in Forth for compiling words to generate
> code that is more compiling words that generate code ... you get the
> idea.  Where is the runtime/compile-time split then?

Well, the most rigorous way to do this would be to have a fully
stratified tower of phases: run-time words, compiling words for
run-time words, compiling words for compiling words, etc. This is the
way macro expansion works in e.g. Racket (formerly PLT Scheme). When
the infrastructure is in place, it's quite painless to use: a module
(that may export both normal definitions and macros) can be used by
any module at any phase, or even at multiple phases. The system makes
sure that the instantiations of a module at distinct phases remain
segregated and cannot interfere with each other.

The simpler approach is just to have two phases: run-time and
compile-time. All compiling words are executed at the compile-time
phase, and once defined, they may be used both in run-time and
compile-time words. This isn't quite as beautiful as the fully
stratified approach, but should give most of the benefits.

The cross-compiler word set proposal goes a tiny bit further than
this: compiling words for run-time are defined in COMPILER scope,
whereas compiling words for compile-time are defined in HOST scope as
IMMEDIATE, as normal.


Lauri

[toc] | [prev] | [next] | [standalone]

#19336

On 1/31/13 2:01 PM, Lauri Alanko wrote:
> In article <M8udnV_U7KPNppfMnZ2dnUVZ_rudnZ2d@supernews.com>,
> Andrew Haley  <andrew29@littlepinkcloud.invalid> wrote:
>>> Even without phases, the namespace separation that the proposal
>>> introduces between interpretation, execution and immediate words
>>> might be useful even in "normal" Forth.
>>
>> Probably not.  It's not rare in Forth for compiling words to generate
>> code that is more compiling words that generate code ... you get the
>> idea.  Where is the runtime/compile-time split then?
>
> Well, the most rigorous way to do this would be to have a fully
> stratified tower of phases: run-time words, compiling words for
> run-time words, compiling words for compiling words, etc. This is the
> way macro expansion works in e.g. Racket (formerly PLT Scheme). When
> the infrastructure is in place, it's quite painless to use: a module
> (that may export both normal definitions and macros) can be used by
> any module at any phase, or even at multiple phases. The system makes
> sure that the instantiations of a module at distinct phases remain
> segregated and cannot interfere with each other.

Forth style prefers to avoid rigorous stratification of anything :-)

Words exist, and do what they do, and it's up to the user to use them 
appropriately. It's all very relaxed and informal.

> The simpler approach is just to have two phases: run-time and
> compile-time. All compiling words are executed at the compile-time
> phase, and once defined, they may be used both in run-time and
> compile-time words. This isn't quite as beautiful as the fully
> stratified approach, but should give most of the benefits.

That's in effect the way it works, except these phases are atomized: 
"compile time" really refers only to the time between when : (or 
equivalent) executes and ; (or equivalent) terminates compilation. 
Defining words such as CREATE are just executed and make whatever 
objects they're defined to make.

> The cross-compiler word set proposal goes a tiny bit further than
> this: compiling words for run-time are defined in COMPILER scope,
> whereas compiling words for compile-time are defined in HOST scope as
> IMMEDIATE, as normal.

The cross-compiler uses the scope words because it's dealing with two 
systems, the host and target, whereas a resident Forth has only itself. 
The existence of the separate target creates additional options. Whereas 
in the resident Forth you're only either compiling or executing all on 
the same computer, in a cross-compiler you might be:

* compiling host words to execute on the host
* executing host words, some of which may manipulate host memory
* compiling words that will manipulate the target image (compile its 
code, allocate its memory, etc.)
* compiling target words to execute on the target
* executing words that will compile target words
* executing words that manipulate the target image

If you have an umbilical connection to a target system, you may also be 
in a mode in which you appear to be executing target words on the host, 
but actually the host is sending a command to the target to execute the 
word, passing stack items to the target and back as appropriate, so that 
the operation seems transparent.

This level of complexity is unnecessary and undesirable in a resident Forth.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#19465

In article <_NCdnTkA4ucQjpbMnZ2dnUVZ_sadnZ2d@supernews.com>,
Elizabeth D. Rather <erather@forth.com> wrote:
> The existence of the separate target creates additional options. Whereas
> in the resident Forth you're only either compiling or executing all on
> the same computer, in a cross-compiler you might be:
>
> * compiling host words to execute on the host
> * executing host words, some of which may manipulate host memory
> * compiling words that will manipulate the target image (compile its
> code, allocate its memory, etc.)
> * compiling target words to execute on the target
> * executing words that will compile target words
> * executing words that manipulate the target image

This last one surprised me a bit. I see I read the spec (and the
accompanying paper) too hastily. So the idea is that if we have, say,

TARGET
VARIABLE foo
42 foo !

...then the host will allocate a cell from a target image, make foo
return the target-relative address of that cell, and store a
target-architecture representation of 42 in that cell. The output of the
compiler will be the image resulting from executing all interpreted
TARGET code, and there will be some (implementation-specific?) way of
specifying an entry point (a word?) in the image. Have I understood this
much correctly?

If so, this seems like a strange approach:

- If cells on host are untagged, converting them correctly to the target
  architecture seems impossible. Suppose the host is 32-bit and the
  target is 64-bit. The numeric literal -1 will place $FFFFFFFF on the
  stack. If we then try to execute ! in TARGET scope to store it in
  IData, how do we know if the cell is meant to be signed
  ($FFFFFFFFFFFFFFFF) or unsigned ($00000000FFFFFFFF)?

- Interpreted I/O words, even in the TARGET scope, will be executed on
  the host at compile time, not on the target.

I would have expected TARGET definitions to be visible when interpreting
in TARGET scope, and their interpretation semantics would be to add
their execution semantics to a "top-level script" which would then be
executed when the target binary is run. So TARGET-scope interpreted ! or
I/O words or whatever would only be executed on the target, not on the
host. This would make the compiled program more faithfully replicate the
semantics of interpreting the source of the program.

If the above approach is not used in Forth compilers, what is the reason
for that?

> This level of complexity is unnecessary and undesirable in a resident
> Forth.

This is very much a matter of preference, convention and culture.
Personally, I think that metaprogramming brings a huge amount of
complexity to a program (although the rewards are often worth it). The
conceptual difference between first-order code (that actually does
something useful) and higher-order code (that will just generate some
more code) needs to be heeded, even if both levels are written in the
same language, which makes them less distinct visually.

If the language encourages or even requires the programmer to annotate
the different levels of code, it doesn't _create_ complexity, it just
forces the programmer to _face_ existing complexity instead of
pretending it isn't there.


Lauri

[toc] | [prev] | [next] | [standalone]

#19467

Lauri Alanko <la@iki.fi> wrote:
> In article <_NCdnTkA4ucQjpbMnZ2dnUVZ_sadnZ2d@supernews.com>,
> Elizabeth D. Rather <erather@forth.com> wrote:
>> The existence of the separate target creates additional options. Whereas
>> in the resident Forth you're only either compiling or executing all on
>> the same computer, in a cross-compiler you might be:
>>
>> * compiling host words to execute on the host
>> * executing host words, some of which may manipulate host memory
>> * compiling words that will manipulate the target image (compile its
>> code, allocate its memory, etc.)
>> * compiling target words to execute on the target
>> * executing words that will compile target words
>> * executing words that manipulate the target image
> 
> This last one surprised me a bit. I see I read the spec (and the
> accompanying paper) too hastily. So the idea is that if we have, say,
> 
> TARGET
> VARIABLE foo
> 42 foo !
> 
> ...then the host will allocate a cell from a target image, make foo
> return the target-relative address of that cell, and store a
> target-architecture representation of 42 in that cell.

Not exactly.  The cross-compiler is building up an image that will
live at some address in the target's memory.  That image might be in
ROM on the target.  To store anything in foo, you must specify IDATA .

> - If cells on host are untagged, converting them correctly to the target
>  architecture seems impossible. Suppose the host is 32-bit and the
>  target is 64-bit.

That's impossible because the host wouldn't be able to form an address
for the target.  The only time I ever saw a real 16 -> 32 bit cross
compiler, it used a 32-bit Forth running on a 16-bit x86 to target the
68000.  It's a very rare need, to say the least, because you're
generally compiling from a desktop computer (big) to an embedded
widget (small).

> - Interpreted I/O words, even in the TARGET scope, 

Words in TARGET scope are target-executable; they are not host-
executable.

> will be executed on the host at compile time, not on the target.

Correct, unless there's some sort of umbilical mechanism.

> I would have expected TARGET definitions to be visible when
> interpreting in TARGET scope, and their interpretation semantics
> would be to add their execution semantics to a "top-level script"
> which would then be executed when the target binary is run.

That's not what happens.  If you need initialization, you can do that
at the start of your application.

> If the above approach is not used in Forth compilers, what is the reason
> for that?

There's no need for it.

>> This level of complexity is unnecessary and undesirable in a
>> resident Forth.
> 
> This is very much a matter of preference, convention and culture.

Well, yes, that's true.  We're talking about the Forth culture.  You
can't talk about the Forth language in isolation from that.

> Personally, I think that metaprogramming brings a huge amount of
> complexity to a program (although the rewards are often worth it). The
> conceptual difference between first-order code (that actually does
> something useful) and higher-order code (that will just generate some
> more code) needs to be heeded, even if both levels are written in the
> same language, which makes them less distinct visually.

I think you're adding barriers where none are needed.  Most
metaprogramming in Forth is just not a big deal, it's ordinary meat 'n
potatoes work.  Look at this:

: array ( n --)  create  cells allot
   does> ( n -- a)  swap cells + ;

Is that metaprogramming?  Yes: the first half (up to DOES>) executes
while compiling, the second half at runtime.  Do we need to put
warning beacons on this very ordinary code?  No.

> If the language encourages or even requires the programmer to annotate
> the different levels of code, it doesn't _create_ complexity, it just
> forces the programmer to _face_ existing complexity instead of
> pretending it isn't there.

You have to face complexity anyway, but there's no point dressing it
up in drag.

Andrew.

[toc] | [prev] | [next] | [standalone]

#19473

Andrew Haley wrote:
>> - If cells on host are untagged, converting them correctly to the
>> target
>>  architecture seems impossible. Suppose the host is 32-bit and the
>>  target is 64-bit.
> 
> That's impossible because the host wouldn't be able to form an address
> for the target.  The only time I ever saw a real 16 -> 32 bit cross
> compiler, it used a 32-bit Forth running on a 16-bit x86 to target the
> 68000.  It's a very rare need, to say the least, because you're
> generally compiling from a desktop computer (big) to an embedded
> widget (small).

Nowadays, the Gforth cross compiler usually runs on a 64 bit machine, 
cross compiling 32 bits there.  But when we started, it was the other 
way round: 32 bit machine, 64 bit target.

The situation is much better than it looks, because addresses are 
offsets from the image start, and therefore, you don't need 64 bits for 
them.

Some parts are a bit tricky, like the name headers (cell sized length, 
flags in the top bits), but most literals are 32 bits.  The tricky ones 
get special treatment.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#19307

Lauri Alanko wrote:
> ...
> Even without phases, the namespace separation that the proposal
> introduces between interpretation, execution and immediate words might
> be useful even in "normal" Forth.

There do exist "normal" Forths which are implemented such that compiler
can be discarded when an application is turnkeyed e.g. Win32Forth &
DX-Forth.  For embedded apps there was RSC-Forth, Inner Access
Super-8 etc.  The latter employed a "development ROM" located in high
memory which held the compiler/interpreter.  Once the app was written
and debugged, the development ROM was discarded.  There's a pdf
manual for RSC-Forth on the internet which explains how it works.

Does one *need* to have the Forth compiler/interpreter in the final
application?  In my experience, almost never.

[toc] | [prev] | [next] | [standalone]

#19309

Ed <invalid@nospam.com> wrote:
> 
> Does one *need* to have the Forth compiler/interpreter in the final
> application?  In my experience, almost never.

It depends what you're doing.  An open interpreter can be really
useful: OpenBoot is a good example.  So are application-oriented
languages used for, say, sequence control.

Andrew.

[toc] | [prev] | [next] | [standalone]

#19312

In article <TJmdnY0N8fgIwpfMnZ2dnUVZ_u6dnZ2d@supernews.com>,
Andrew Haley  <andrew29@littlepinkcloud.invalid> wrote:
>Ed <invalid@nospam.com> wrote:
>>
>> Does one *need* to have the Forth compiler/interpreter in the final
>> application?  In my experience, almost never.
>
>It depends what you're doing.  An open interpreter can be really
>useful: OpenBoot is a good example.  So are application-oriented
>languages used for, say, sequence control.

My euler solutions often end with

: doit 1 ARG[] EVALUATE euler410
  "And the solution .... " TYPE total ? CR ;

A typical call

euler411 '7 9 **'

>
>Andrew.

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

Page 2 of 5 — ← Prev page 1 [2] 3 4 5  Next page →

Back to top | Article view | comp.lang.forth

csiph-web