Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #3603 > unrolled thread

macro expansion ?

Back to article view | Back to comp.lang.forth

Contents

  macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 14:43 +0100
    Re: macro expansion ? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-06-28 10:58 -0500
      Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 18:34 +0100
        Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 18:51 +0100
          Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 19:02 +0100
            Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 19:11 +0100
              Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 19:17 +0100
                Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 19:47 +0100
                  Re: macro expansion ? Josh Grams <josh@qualdan.com> - 2011-06-29 15:34 +0000
                    Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-29 19:21 +0100
              Re: macro expansion ? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-06-29 03:15 -0500
                Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-29 13:11 +0100
              Re: macro expansion ? Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-06-29 17:17 +0100
                Re: macro expansion ? Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-06-29 18:09 +0100
                  Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-29 19:32 +0100
                    Re: macro expansion ? Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-07-02 08:48 +0100
      Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-29 00:44 +0100
      Re: macro expansion ? Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2011-06-29 20:39 +0100
    Re: macro expansion ? Elizabeth D Rather <erather@forth.com> - 2011-06-28 08:59 -1000
      Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 20:39 +0100
        Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-28 21:34 +0100
          Re: macro expansion ? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-06-29 03:26 -0500
    Re: macro expansion ? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-06-29 11:25 +0000
      Re: macro expansion ? Chris Hinsley <chris.hinsley@gmail.com> - 2011-06-29 13:24 +0100
        Re: macro expansion ? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2011-07-03 14:02 +0000
      Re: macro expansion ? Albert van der Horst <albert@spenarnc.xs4all.nl> - 2011-06-29 18:48 +0000
    Re: macro expansion ? BruceMcF <agila61@netscape.net> - 2011-06-30 11:11 -0700
    Re: macro expansion ? Ian Osgood <iano@quirkster.com> - 2011-07-02 16:22 -0700
    Re: macro expansion ? Hugh Aguilar <hughaguilar96@yahoo.com> - 2011-07-05 17:35 -0700

Page 2 of 2 — ← Prev page 1 [2]

#3615

On 2011-06-28 20:39:15 +0100, Chris Hinsley said:

> On 2011-06-28 19:59:06 +0100, Elizabeth D Rather said:
> 
>> On 6/28/11 3:43 AM, Chris Hinsley wrote:
>>> From the recent thread on IMMEDIATE I quote this word:
>>> 
>>> \ macros
>>> : MACRO ( "name" <ccccc"> -- )
>>> CREATE ," IMMEDIATE DOES> COUNT EVALUATE ;
>>> 
>>> When I first added this word to my Forth project I belived it would
>>> allow me to define words that were 'macro expanded' in a text
>>> substitution manner.
>>> 
>>> Is this indead the case ? The call to EVALUATE made me think this would
>>> act as if it was text substitution rather than just a IMMIDEATE word
>>> that would execute during compilation. My intention was somthing that
>>> would be expanded during compilation and therefore may get optimised as
>>> inlined code into the word curently deing defined, so no subroutine call
>>> overhead.
>> 
>> Whether it compiles something would depend on whether it's used in 
>> compile state, obviously.  The use of EVALUATE has a potential 
>> side-effect in that if a word in the string has been redefined, you'll 
>> get the most recent version, whereas when you compile a definition you 
>> always have the version that was current at the time it was compiled.
> 
> I see the possibility of redefintion during expansion as a feature, not 
> a problem. !
> 
>> 
>>> Clearly it suffers from haveing a single " being scanned for the end of
>>> the evaluated string, but that could be improved maybe ?
>>> 
>>> But this led me on to thinking how would I do a more Lisp like macro ?
>>> Somthing that would be expanded and then that expantion expanded till
>>> there was no more expantions. One of the thoughs that's bothering me on
>>> this is what happens if the expansion leads to new words needing to be
>>> defined within a partial expansion, and then on the next expantion pass
>>> they are executed, all this while not yet finishing the definition of
>>> the original word that used the macro in the first place ?
>>> 
>>> I would like to resolve these thoughts with any help from the Forth
>>> GURU's out there. :)
>> 
>> My big question is, why do you want to do this?  If it's an academic 
>> exercise, to just see if it's possible, that's one thing: enjoy. But 
>> I'm not very enamored of macros in general, because they're often a 
>> substitute for "programming by copy/paste" which is a pernicious 
>> practice.  People do it in languages where a CALL (and surrounding 
>> setup, etc.) is costly, but given that the entire architecture of Forth 
>> is designed to minimize the cost of a call, I really don't see the 
>> point as a practical matter.
>> 
>> Cheers,
>> Elizabeth
> 
> It's not an academic exersise ! At least not the way I see it. I 
> disagree that CALL is cheap ! It is on your own Forth CPU, it's not on 
> a general RISC chip.
> 
> I would like to have the ability to have a powerfull macro system that 
> works at the source level. Somthing like what LISP has, or (forgive me 
> mentioning it) C++ templates. Before the Forth compiler starts to lay 
> down the definition of a word I wish to have text substitution happen, 
> I want the full power of Forth to be available during the expansion to 
> manipulate the source. I don't seam to be able to do this in a simple 
> way useing what's available within the language itself ?
> 
> Chris

I'll try to ilustrate why this is important to me.

In assembler language I've written a _lot_ of pixel bashing code over 
the years while codeing arcade games, I've also written around 5 GUI 
libraries during the last 15 years, again all in assembler not C.

So I certainly didn't have an optimising compiler to do the hard work 
of producing optimal pixel conversion code (for example) for blit 
routines from one surface type to another. After doing my n'th hand 
coded assembler inner loop for some new customer graphics card surface 
format I thought, 'stuff this for a game of soilders I'm going to write 
a macro to do this for me'. I put all the smarts of how to shift and 
mask and up/down convert colour channels, detect if there was an 
unneeded channel or if I could omit an and if it was the top bits of a 
channel, plus it would shedule the sequence of channel operations to 
allow for results not being available for 1 and somtimes 2 cycles etc.

After I'd done this macro, I then recoded every surface type blit class 
to just a few macro calls, removeing huge gobs of repetative code, 
along with minnor bugs too I might add. And we had a system that would 
produce optimal _quality_ code, passed all the test suits, was now 
about 20 lines of source per surface class. And any old idiot (sorry 
any Tao guys out there reading this, cause none of you where idiots :) 
) could knock up a new format for a customer without any need to know 
more than a few equates to define the channel layout.

Now I find myself in a similar situation with Forth. I like the 
language a lot, I've ported and improved a simple compiler, but I 
really don't think I'm going to expend all the energy I need to do a 
fully optimising compiler. But I don't think I should need to ! 5% of 
the code (sucking eggs time guys) get's executted 90% of the time. I 
really don't care much about 95% of the code being optimised or not. 
But what I do care about is if I can create a macro in Forth that has 
the smarts my assembler macro had, and produces inline code for each 
surface to surface blit function without haveing a call for each 
operation on each colour channel for each pixel !

Chris

[toc] | [prev] | [next] | [standalone]

#3621

Chris Hinsley <chris.hinsley@gmail.com> wrote:
> On 2011-06-28 20:39:15 +0100, Chris Hinsley said:
> 
>> I would like to have the ability to have a powerfull macro system that 
>> works at the source level. Somthing like what LISP has, or (forgive me 
>> mentioning it) C++ templates.

But Forth already has this in spades.  Immediate words do everything
that you're asking for AFAICS, and without the pain of template
metaprogramming.

> But what I do care about is if I can create a macro in Forth that has 
> the smarts my assembler macro had, and produces inline code for each 
> surface to surface blit function without haveing a call for each 
> operation on each colour channel for each pixel !

This is really easy in Forth.  See
http://groups.google.com/group/comp.lang.forth/msg/0e9f575ad26b70af
for an example of mine.

Andrew.

[toc] | [prev] | [next] | [standalone]

#3623

Chris Hinsley <chris.hinsley@gmail.com> writes:
>From the recent thread on IMMEDIATE I quote this word:
>
>\ macros
>: MACRO ( "name" <ccccc"> -- )
>   CREATE ," IMMEDIATE  DOES> COUNT EVALUATE ;
>
>When I first added this word to my Forth project I belived it would 
>allow me to define words that were 'macro expanded' in a text 
>substitution manner.

In some sense, which is why it is a bad idea.

>Is this indead the case ? The call to EVALUATE made me think this would 
>act as if it was text substitution rather than just a IMMIDEATE word 
>that would execute during compilation.

It's an immediate word that executes during compilation, so the text
substitution part happens during compilation.  Leave away the
IMMEDIATE and you get a word where the text substitution happens at
run-time.  That is (for both), in the usual case.  When you tick or
POSTPONE the word, you get something STATE-dependend (because the text
interpreter used by EVALUIATE is STATE-dependend), and the behaviour
you intended will probably only happen in one of the two states.

There are also other dependencies in this kind of word: dependence on
BASE, on the search order, on the words in the wordlists on the search
order, and other global state.

It's very easy to write buggy programs with these kinds of words, and
hard to test for that stuff, so I call this concept bugros.

As an illustration of one of the dangers:

decimal
macro ten 10 "

hex
: foo ten ;
decimal

foo .

>My intention was somthing that 
>would be expanded during compilation and therefore may get optimised as 
>inlined code into the word curently deing defined, so no subroutine 
>call overhead.

If you want optimization, get an optimizing compiler and write colon
defintions.  The compiler will inline the colon definitions by itself.
You will get better performance and fewer bugs.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2011: http://www.euroforth.org/ef11/

[toc] | [prev] | [next] | [standalone]

#3625

On 2011-06-29 12:25:33 +0100, Anton Ertl said:

> Chris Hinsley <chris.hinsley@gmail.com> writes:
>> From the recent thread on IMMEDIATE I quote this word:
>> 
>> \ macros
>> : MACRO ( "name" <ccccc"> -- )
>> CREATE ," IMMEDIATE  DOES> COUNT EVALUATE ;
>> 
>> When I first added this word to my Forth project I belived it would
>> allow me to define words that were 'macro expanded' in a text
>> substitution manner.
> 
> In some sense, which is why it is a bad idea.
> 
>> Is this indead the case ? The call to EVALUATE made me think this would
>> act as if it was text substitution rather than just a IMMIDEATE word
>> that would execute during compilation.
> 
> It's an immediate word that executes during compilation, so the text
> substitution part happens during compilation.  Leave away the
> IMMEDIATE and you get a word where the text substitution happens at
> run-time.  That is (for both), in the usual case.  When you tick or
> POSTPONE the word, you get something STATE-dependend (because the text
> interpreter used by EVALUIATE is STATE-dependend), and the behaviour
> you intended will probably only happen in one of the two states.
> 
> There are also other dependencies in this kind of word: dependence on
> BASE, on the search order, on the words in the wordlists on the search
> order, and other global state.
> 
> It's very easy to write buggy programs with these kinds of words, and
> hard to test for that stuff, so I call this concept bugros.
> 
> As an illustration of one of the dangers:
> 
> decimal
> macro ten 10 "
> 
> hex
> : foo ten ;
> decimal
> 
> foo .
> 
>> My intention was somthing that
>> would be expanded during compilation and therefore may get optimised as
>> inlined code into the word curently deing defined, so no subroutine
>> call overhead.
> 
> If you want optimization, get an optimizing compiler and write colon
> defintions.  The compiler will inline the colon definitions by itself.
> You will get better performance and fewer bugs.
> 
> - anton

OK, I'm convinced this MACRO word does more harm that good.

I don't agree that I should have to have an optimizing compiler to 
produce good code ! For my specific example about surface to surface 
blit routines, it's perfectly possible to do them without an optimizing 
compiler provided you have a good technique for source generation.

I'm only asking for somthing that I could achive in a macro assembler, 
I wouldn't have thought that was asking too much of Forth.

I guess the best way to go on this is to write an IMMEDIATE word that 
compiles the inner loop of the blit directly, takes the surface types 
as compile time paramaters and just places the code in the dictionary 
directly.

Chris

[toc] | [prev] | [next] | [standalone]

#3743

Chris Hinsley <chris.hinsley@gmail.com> writes:
>I don't agree that I should have to have an optimizing compiler to 
>produce good code !

In case of inlining, that's the best general way.

>For my specific example about surface to surface 
>blit routines, it's perfectly possible to do them without an optimizing 
>compiler provided you have a good technique for source generation.

For specific applications where certain restrictions are acceptable,
there are also other techniques.  E.g., if you want macros and can
accept that they don't work interpretively, you can use POSTPONE in
combination with IMMEDIATE; example:

: my-2dup postpone over postpone over ; immediate

: my-1+ 1 postpone literal postpone + ; immediate

long sequences of POSTPONEs are not very readable, so you can use the
]] .. [[ syntax implemented in standard Forth in
<http://www.complang.tuwien.ac.at/viewcvs/cgi-bin/viewcvs.cgi/*checkout*/gforth/compat/macros.fs>

: my-2dup ]] over over [[ ; immediate
: my-1+ 1 ]]L + [[ ; immediate

The advantage over EVALUATE-based macros is that the word names are
bound at macro defintion time (so you know which version of the word
is compiled), and likewise for BASE and possibly other dependences.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2011: http://www.euroforth.org/ef11/

[toc] | [prev] | [next] | [standalone]

#3636

In article <2011Jun29.132533@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>Chris Hinsley <chris.hinsley@gmail.com> writes:
>>My intention was somthing that 
>>would be expanded during compilation and therefore may get optimised as 
>>inlined code into the word curently deing defined, so no subroutine 
>>call overhead.
>
>If you want optimization, get an optimizing compiler and write colon
>defintions.  The compiler will inline the colon definitions by itself.
>You will get better performance and fewer bugs.

And if you want ultimate control there is always assembler.
That is very well integrated in most Forth's.

>
>- anton

Groetje Albert

-- 

-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#3673

On Jun 28, 9:43 am, Chris Hinsley <chris.hins...@gmail.com> wrote:
> From the recent thread on IMMEDIATE I quote this word:
>
> \ macros
> : MACRO ( "name" <ccccc"> -- )
>    CREATE ," IMMEDIATE  DOES> COUNT EVALUATE ;
>
> When I first added this word to my Forth project I believed it would
> allow me to define words that were 'macro expanded' in a text
> substitution manner.

> Is this indeed the case?

At a single EVALUATE call level, yes. But in macro expansions where
there are "macros in macros", you have an end result that is the full
string of text with all substitutions ~ which if it is source could
then be evaluated ~ where with the above approach, if the EVALUATE
macro has macros contained in it ~ call them "submacros" ~ the
submacros would be expanded and executed sequentially, rather than all
expanded and then executed when there were no further expansions to
perform.


> act as if it was text substitution rather than just a IMMIDEATE word
> that would execute during compilation. My intention was somthing that
> would be expanded during compilation and therefore may get optimised as
> inlined code into the word curently deing defined, so no subroutine
> call overhead.
>
> Clearly it suffers from haveing a single " being scanned for the end of
> the evaluated string, but that could be improved maybe ?

For the delimiter, if you are content with a single character
delimiter, I think you can use CHAR PARSE and S,:

\ macros
: MACRO ( "name" "d" <cccccd> -- )
   CREATE CHAR PARSE S,  DOES> COUNT EVALUATE ;

... which requires S, as ( ca u -- ) to place a string at HERE and
ALLOT the result.

Note that the semantics of CHAR implies that:

MACRO foo | act1 act2 act3 |

works, but also that:

MACRO foo |bar act1 act2 act3 |

... would have exactly the same effect. Define the second token to be
a single white-space delimited character, or define a more precise
PARSE-DELIMITED that infers the trailing delimiter from the first non-
ws character in the input stream.

> But this led me on to thinking how would I do a more Lisp like macro ?
> Somthing that would be expanded and then that expantion expanded till
> there was no more expantions. One of the thoughs that's bothering me on
> this is what happens if the expansion leads to new words needing to be
> defined within a partial expansion, and then on the next expantion pass
> they are executed, all this while not yet finishing the definition of
> the original word that used the macro in the first place ?

Each expansion has to be "complete" in the sense of working only on
its own string of words and the state of the stack. An EVALUATE macro
cannot, in other words, be used as a defining word in the context of
the input state that called the evaluate.

Given that, some of the tangles you are forseeing might be ruled out
from the outset in any event.

Note also that inheriting compiler optimizations will *only* apply at
the "node" macro ~ the above approach makes a "macro of submacros"
into a sequence, with each "submacro" completed before the next
"submacro" is evaluated, so optimizations that would occur if the low
level source was laid end to end involving words from distinct
"submacros" will not be taken advantage of.

[toc] | [prev] | [next] | [standalone]

#3725

On Jun 28, 6:43 am, Chris Hinsley <chris.hins...@gmail.com> wrote:
> From the recent thread on IMMEDIATE I quote this word:
>
> \ macros
> : MACRO ( "name" <ccccc"> -- )
>    CREATE ," IMMEDIATE  DOES> COUNT EVALUATE ;
>
> When I first added this word to my Forth project I belived it would
> allow me to define words that were 'macro expanded' in a text
> substitution manner.
>
> Is this indead the case ? The call to EVALUATE made me think this would
> act as if it was text substitution rather than just a IMMIDEATE word
> that would execute during compilation. My intention was somthing that
> would be expanded during compilation and therefore may get optimised as
> inlined code into the word curently deing defined, so no subroutine
> call overhead.
>
> Clearly it suffers from haveing a single " being scanned for the end of
> the evaluated string, but that could be improved maybe ?
>
> But this led me on to thinking how would I do a more Lisp like macro ?
> Somthing that would be expanded and then that expantion expanded till
> there was no more expantions. One of the thoughs that's bothering me on
> this is what happens if the expansion leads to new words needing to be
> defined within a partial expansion, and then on the next expantion pass
> they are executed, all this while not yet finishing the definition of
> the original word that used the macro in the first place ?
>
> I would like to resolve these thoughts with any help from the Forth
> GURU's out there. :)
>
> Chris

In my chess program, I used a similar word ":inline", from Wil Baden:

: :inline ( "name <char> ccc<char>" -- )
  : [CHAR] ; PARSE  POSTPONE SLITERAL  POSTPONE EVALUATE
  POSTPONE ; IMMEDIATE
;

which allowed me to selectively inline words that profiling determined
were hot-spots. Like pixel-shaders, a chess program can never be too
fast, so it is useful to be able to micro-optimize while retaining the
ability to factor out snippets into well-named words.  Typically, the
things I inlined were straight line code snippets less than five words
long.

Ian

[toc] | [prev] | [next] | [standalone]

#3832

On Jun 28, 7:43 am, Chris Hinsley <chris.hins...@gmail.com> wrote:
> From the recent thread on IMMEDIATE I quote this word:
>
> \ macros
> : MACRO ( "name" <ccccc"> -- )
>    CREATE ," IMMEDIATE  DOES> COUNT EVALUATE ;
>
> When I first added this word to my Forth project I belived it would
> allow me to define words that were 'macro expanded' in a text
> substitution manner.
>
> Is this indead the case ? The call to EVALUATE made me think this would
> act as if it was text substitution rather than just a IMMIDEATE word
> that would execute during compilation. My intention was somthing that
> would be expanded during compilation and therefore may get optimised as
> inlined code into the word curently deing defined, so no subroutine
> call overhead.
>
> Clearly it suffers from haveing a single " being scanned for the end of
> the evaluated string, but that could be improved maybe ?
>
> But this led me on to thinking how would I do a more Lisp like macro ?
> Somthing that would be expanded and then that expantion expanded till
> there was no more expantions. One of the thoughs that's bothering me on
> this is what happens if the expansion leads to new words needing to be
> defined within a partial expansion, and then on the next expantion pass
> they are executed, all this while not yet finishing the definition of
> the original word that used the macro in the first place ?
>
> I would like to resolve these thoughts with any help from the Forth
> GURU's out there. :)
>
> Chris

I have MACRO: in my novice package. Note that it works for multi-line
macros. It is used to replace colon. You don't want to use other
delimiters than semicolon, because then you can't just replace colon
with MACRO: without also modifying the delimiter. My MACRO: has the
expected limitations, such as that the macros don't work in
interpretive mode, and if you use the name of the macro inside of it
you will get infinite recursion.

For the most part, I just use MACRO: as a replacement for colon to
speed up the code. This is done after the colon word has already been
tested interpretively, so the fact that the macro doesn't work
interpretively isn't an issue. I also occasionally use macros for code
that can't be a colon word because it includes incomplete control
structures or uses the return stack. For example, in LIST.4TH I have:

macro: next>r ( node -- node )   \ r: -- next-node
    dup .fore @ >r ;

macro: each[                     \ toucher: i*x node -- j*x
    begin  dup while
        next>r ;

macro: ]each
        r> repeat drop ;

Notice that MACRO: has a colon on the end of the word. This helps you
to look up definitions in a text editor. If you can't remember if xxx
is a colon word or a macro, just look up ": xxx" and your text editor
search will find the definition either way.

If you want your "macro" to have parameters, then you should just
write an immediate word. Use POSTPONE or EVALUATE inside of the
immediate word to generate your code. My FIELD is an example of this;
there are other examples as well.

BTW, I originally wrote MACRO: to look up the xt of each word and
compile it. This turned out to be a problem in ANS-Forth because some
words don't necessarily have an xt value (!!!). For example, this
worked okay in SwiftForth but then failed under Gforth because the
semicolon doesn't have an xt value. I consider this to be a bug in the
ANS-Forth design --- every word should have an xt value. Anyway, to
maintain ANS-Forth compatibility for the novice package, I switched
over to doing a string expansion, which is what I have now.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.forth

csiph-web