Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #48923

Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth

Show all headers | View raw

On 07/07/2016 20:48, Alex Dowad wrote:
> Greetings all,
>
> I'm sure there is a wealth of information on this topic out there on the web, but my Google power may not be strong enough. If anyone could be so kind as to give me a few links to quality material, that would be appreciated.
>
> I've built a Forth implementation and am interested in getting it to compile Forth programs to standalone binaries. The binaries should not go through the process of building up word definitions and adding them to the dictionary at run-time; all that should happen at compile-time (including executing immediates like IF), and just the completed word definitions should go into the output binary.
>
> Better yet is if only the word definitions which are actually needed go into the output. The dictionary itself should also be omitted, meaning that words like FIND cannot be used at run-time (only at compile-time, either at the top level or in an immediate word).
>
> Better still is if the output is fully relocatable and thus can be linked freely with libraries (which might be written in a different language). It turns out this is non-trivial to do, unless you make some undesirable compromises:
>
> Fixing up the addresses in the code field of a word's definition (this is DTC) and making them relocatable is trivial. Fixing up BRANCHes (which could possibly cross over into a different word's code field) is also more or less trivial.
>
> Dealing with ALLOTed addresses may be trickier, depending on what compromises you want to make. VARIABLE, VALUE, ARRAY, and so on all rely on ALLOT, compiling an ALLOTed address as the argument of LIT. It would be best if they work seamlessly at both compile-time and run-time.
>
> It is easy enough to record how much memory is ALLOTed and arrange for an equivalent amount of memory to be reserved at run-time. But since we want the code to be relocatable, either you must emit a relocation for every place where an ALLOTed address appears in a word definition, or significant extra indirection (and cost) will be incurred each time those addresses are accessed at runtime.
>
> Again, it's easy to add a special primitive which tells the compiler to emit a relocation, and use that primitive in VARIABLE, VALUE, DEFER, etc. It's unfortunate that ALL words, including custom ones, which include addresses will have to use that special primitive, but that's not so bad.
>
> What is much trickier is that ALLOTed addresses may also be stored in a VARIABLE, an ARRAY, a table of some kind, etc. If you allow VARIABLEs to be initialized at compile-time, and then carry the values through to run-time by storing the values in the binary, those values could be pointers. Those pointers will not be valid at run-time, unless they are relocated with the rest of the code.
>
> But when looking at raw data, it's very hard to know what is and is not a pointer. Unless the programmer writes some special incantation every time (s)he stores a pointer somewhere, in the general case, the compiler will not be able to identify all the pointers in pre-initialized data and emit relocations for all of them.
>
> A similar problem exists with tick; it's not hard to deal with written inside a colon definition, but very hard to deal with when exec tokens are stored in arbitrary places and used at run-time.
>
> A few possible solutions:
>
> - No pre-initialized data. All data (VARIABLEs, ARRAYs, etc etc) must all be initialized at run-time, using code inside a colon definition. Perhaps they can also be used at compile-time, but the values do not carry through.
> - Abandon the idea that the output should be relocatable. Emit code which only works when loaded at a specific address.
> - Add an offset to addresses when accessing them at run-time from within @, etc. The offset will have to be kept in a global variable. Likewise in EXECUTE.
> - Abandon the idea that the same VARIABLE should work at both compile-time and run-time. Use a completely different word, with different semantics.
> - ???
>
> I'm sure I'm not the first person to run into these difficulties. What are other Forth AOT compilers "out there" doing? (My guess: probably emitting non-relocatable code.)

I believe one solution to identifying addresses, although I haven't 
tried it myself, is to compile the program twice to different addresses 
and compare the two results. As they are compiled to different addresses 
anything different between the two must be an address (unless you're 
compiling time varying data which is, presumably, unlikely).

-- 
Gerry

Back to comp.lang.forth | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-07 12:48 -0700
  Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2016-07-07 21:26 +0100
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-07 13:48 -0700
  Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Rod Pemberton <NoHaveNotOne@bcczxcfrze.cam> - 2016-07-07 19:48 -0400
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-07 21:02 -0700
      Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Paul Rubin <no.email@nospam.invalid> - 2016-07-08 00:05 -0700
        Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-08 00:42 -0700
          Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Lars Brinkhoff <lars.spam@nocrew.org> - 2016-07-08 09:57 +0200
  Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth "Elizabeth D. Rather" <erather@forth.com> - 2016-07-07 14:57 -1000
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Cecil Bayona <cbayona@cbayona.com> - 2016-07-07 20:09 -0500
      Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth "Elizabeth D. Rather" <erather@forth.com> - 2016-07-07 16:35 -1000
        Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Richard Owlett <rowlett@cloud85.net> - 2016-07-08 09:31 -0500
  Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Lars Brinkhoff <lars.spam@nocrew.org> - 2016-07-08 08:03 +0200
  Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Chris Curl <ccurl609@gmail.com> - 2016-07-08 05:49 -0700
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Chris Curl <ccurl609@gmail.com> - 2016-07-08 05:58 -0700
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-08 06:53 -0700
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth hughaguilar96@gmail.com - 2016-07-08 07:11 -0700
      Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Alex Dowad <alexinbeijing@gmail.com> - 2016-07-08 07:50 -0700
    Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Cecil Bayona <cbayona@cbayona.com> - 2016-07-08 10:17 -0500
      Re: Ahead-of-time compilation (as well as tree-shaking, etc.) for Forth Chris Curl <ccurl609@gmail.com> - 2016-07-08 08:34 -0700

csiph-web