Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #101235 > unrolled thread

Macro Expansion

Back to article view | Back to comp.lang.c

Contents

  Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 18:35 +0000
    Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-01-07 11:23 -0800
      Re: Macro Expansion Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-01-07 21:28 +0000
    Re: Macro Expansion Siri Cruise <chine.bleu@yahoo.com> - 2017-01-07 11:45 -0800
      Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 20:16 +0000
        Re: Macro Expansion Siri Cruise <chine.bleu@yahoo.com> - 2017-01-07 12:23 -0800
          Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 20:31 +0000
            Re: Macro Expansion Siri Cruise <chine.bleu@yahoo.com> - 2017-01-07 12:42 -0800
            Re: Macro Expansion Ian Collins <ian-news@hotmail.com> - 2017-01-08 14:42 +1300
              Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 13:18 +0000
    Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 20:15 +0000
      Re: Macro Expansion Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-01-07 20:42 +0000
        Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 22:50 +0000
          Re: Macro Expansion Ian Collins <ian-news@hotmail.com> - 2017-01-08 11:59 +1300
            Re: Macro Expansion Malcolm McLean <malcolm.mclean5@btinternet.com> - 2017-01-07 15:10 -0800
          Re: Macro Expansion supercat@casperkitty.com - 2017-01-07 17:24 -0800
            Re: Macro Expansion Tim Rentsch <txr@alumni.caltech.edu> - 2017-01-15 21:55 -0800
              Re: Macro Expansion supercat@casperkitty.com - 2017-01-16 09:06 -0800
                Re: Macro Expansion Tim Rentsch <txr@alumni.caltech.edu> - 2017-01-31 08:51 -0800
                  Re: Macro Expansion supercat@casperkitty.com - 2017-01-31 13:06 -0800
                    Re: Macro Expansion Tim Rentsch <txr@alumni.caltech.edu> - 2017-02-03 07:49 -0800
                      Re: Macro Expansion supercat@casperkitty.com - 2017-02-03 12:03 -0800
                        Re: Macro Expansion Tim Rentsch <txr@alumni.caltech.edu> - 2017-02-09 09:00 -0800
          Re: Macro Expansion Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-01-08 01:37 +0000
            Re: Macro Expansion Jean-Marc Bourguet <jm@bourguet.org> - 2017-01-08 10:22 +0100
    Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-01-07 12:18 -0800
      Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 20:28 +0000
        Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-01-07 12:36 -0800
          Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-07 22:59 +0000
    Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-01-08 00:20 -0800
      Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 13:30 +0000
        Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-10 14:00 +0000
          Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-01-10 07:43 -0800
            Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-10 17:47 +0000
              Re: Macro Expansion Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-01-10 22:49 +0000
                Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-10 23:28 +0000
                Re: Macro Expansion "James R. Kuyper" <jameskuyper@verizon.net> - 2017-01-10 18:51 -0500
                  Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-11 00:06 +0000
            Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-10 20:30 +0000
              Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-04-26 10:04 -0700
                Re: Macro Expansion Keith Thompson <kst-u@mib.org> - 2017-04-26 12:00 -0700
                  Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-04-26 12:24 -0700
                Re: Macro Expansion bartc <bc@freeuk.com> - 2017-04-26 20:12 +0100
                  Re: Macro Expansion Thiago Adams <thiago.adams@gmail.com> - 2017-04-26 12:34 -0700
                  Re: Macro Expansion jameskuyper@verizon.net - 2017-04-26 13:46 -0700
    Re: Macro Expansion Jean-Marc Bourguet <jm@bourguet.org> - 2017-01-08 10:13 +0100
      Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 13:10 +0000
        Re: Macro Expansion Jean-Marc Bourguet <jm@bourguet.org> - 2017-01-08 15:51 +0100
          Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 17:17 +0000
            Re: Macro Expansion supercat@casperkitty.com - 2017-01-08 11:04 -0800
              Re: Macro Expansion Malcolm McLean <malcolm.mclean5@btinternet.com> - 2017-01-08 11:10 -0800
                Re: Macro Expansion supercat@casperkitty.com - 2017-01-08 14:08 -0800
              Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 19:49 +0000
                Re: Macro Expansion supercat@casperkitty.com - 2017-01-08 14:17 -0800
            Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 19:19 +0000
            Re: Macro Expansion Jean-Marc Bourguet <jm@bourguet.org> - 2017-01-08 21:43 +0100
        Re: Macro Expansion supercat@casperkitty.com - 2017-01-08 10:43 -0800
        Re: Macro Expansion Keith Thompson <kst-u@mib.org> - 2017-01-08 12:39 -0800
    Re: Macro Expansion antispam@math.uni.wroc.pl - 2017-01-08 22:54 +0000
      Re: Macro Expansion "James R. Kuyper" <jameskuyper@verizon.net> - 2017-01-08 18:36 -0500
        Re: Macro Expansion antispam@math.uni.wroc.pl - 2017-01-09 01:15 +0000
          Re: Macro Expansion "James R. Kuyper" <jameskuyper@verizon.net> - 2017-01-08 20:49 -0500
      Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-08 23:53 +0000
        Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-09 01:11 +0000
        Re: Macro Expansion antispam@math.uni.wroc.pl - 2017-01-09 01:31 +0000
          Re: Macro Expansion BartC <bc@freeuk.com> - 2017-01-10 11:28 +0000

Page 4 of 4 — ← Prev page 1 2 3 [4]

#101325

James R. Kuyper <jameskuyper@verizon.net> wrote:
> On 01/08/2017 05:54 PM, antispam@math.uni.wroc.pl wrote:
> > BartC <bc@freeuk.com> wrote:
> >> I've knocked up a short script program to implement CPP macro expansion.
> >> It works with many of the examples I've tried, but I have little
> >> confidence that whatever I end up with will always work, as I'm guessing
> >> at the algorithms needed.
> >>
> >> The descriptions I've seen of the process seem to me ambiguous and
> >> difficult to understand (maybe they're striving to be as cryptic as the
> >> language they are describing!).
> >
> > I looked at 2011 draft (n1570.pdf).  The description looks clear.
> > First, they say that different implementations (possibly giving
> > differing results are allowed).  So standard is unambigous:
> > the implementer of preprocessor has free hand for choosing
> > details.
> 
> Could you please cite the exact text you're referring to? I'm not aware 
> of anything that the C standard says about the preprocessor that matches 
> that description. As with any other part of the C standard, when it 
> imposes specific requirements for pre-processing, they must be met.

Did I say otherwise?  It should be clear that "details" in my statement
mean "details not specified by the standard" (again "not specified"
beeing shortcat for undefined behaviour, implementatin defined (where
the "not specified" means choice remaing to implementer after
requremnent for "implementatin defined" are met) and unspecified
(again with standard disclaimer).

And standard is several places gives a lot of freedom _how_
implementer satisfies requirements.  For example implementation
may convert function like macros to functions -- if it can
determine when it is safe (does not change observable behaviour
mandated in the standard).  Purely within macro expansion
there are lot of ways to arrive at token seqence satifying
requrements.

> Producing differing results on different implementations is certainly 
> possible, but only when the C standard gives permission to so so. When 
> the behavior is unspecified, an implementation must choose one of the 
> permitted possibilities.

Specifically 6.10.3.2 at the end of point 2 says:

: The order of evaluation of # and ## operators is unspecified.

So implementer has free choce of evaluation order.  AFAICS
for valid code all evaluation orders will give the same
result and only on invalid code diagnostics may differ
(but implementation has a lot of freedom concerning diagnostics
anyway).  But implementer does not have to think if
all evaluation orders really give the same result:
standard explictly give choice here.

Concerning different results, see Example in point 4 of 6.10.3.4.
Apparently standard did not define what "nested replacement" is
and disagreement here may lead to differing results.
My own idea of "nested replacement" is that whole call
must be within replacent text, which would lead to
second expansion.  But Example shows that not all share
my view...

-- 
                              Waldek Hebisch

[toc] | [prev] | [next] | [standalone]

#101327

On 01/08/2017 08:15 PM, antispam@math.uni.wroc.pl wrote:
> James R. Kuyper <jameskuyper@verizon.net> wrote:
>> On 01/08/2017 05:54 PM, antispam@math.uni.wroc.pl wrote:
...
>>> I looked at 2011 draft (n1570.pdf).  The description looks clear.
>>> First, they say that different implementations (possibly giving
>>> differing results are allowed).  So standard is unambigous:
>>> the implementer of preprocessor has free hand for choosing
>>> details.
>>
>> Could you please cite the exact text you're referring to? ...

Could you please cite the precise text that you summarize above by 
saying "First, they say that different implementations (possibly giving 
differing results are allowed)."?  Regardless of anything else said 
below, I still want to know precisely what it is that you're referring to.

>> ... I'm not aware
>> of anything that the C standard says about the preprocessor that matches
>> that description. As with any other part of the C standard, when it
>> imposes specific requirements for pre-processing, they must be met.
>
> Did I say otherwise?

Your use of the term "free hand" is pretty much completely incompatible 
with the existence of any constraints on the implementation of 
preprocessing - and a great many such constraints do, in fact, exist.

...
> ...  Purely within macro expansion
> there are lot of ways to arrive at token seqence satifying
> requrements.

I suppose, but there's only a limited variety of token sequences that 
can actually be arrived at by those many different ways.

...
> Specifically 6.10.3.2 at the end of point 2 says:
>
> : The order of evaluation of # and ## operators is unspecified.
>
> So implementer has free choce of evaluation order.

No, it only has a choice concerning the order of # and ## operators.

> Concerning different results, see Example in point 4 of 6.10.3.4.
> Apparently standard did not define what "nested replacement" is
> and disagreement here may lead to differing results.

Yes, that is one point on which the standard allows for two different 
possibilities - but not complete freedom.

[toc] | [prev] | [next] | [standalone]

#101323

On 08/01/2017 22:54, antispam@math.uni.wroc.pl wrote:
> BartC <bc@freeuk.com> wrote:

>> * The result is now scanned again looking for new macros to expand.
>>   However, once any name (eg. M and W) has been expanded once, it is
>>   never expanded again (in a subsequent pass). So only XX is changed
>>   to result in:
>>
>>    10 20 "10" 99 M(a,b) W+W+20+x
>
> AFAICS this is your interetation, which is mostly correct but
> in corner cases may produce wrong result and seem to be much
> less efficient then what standard says:
>
> : After all parameters in the replacement list have been substituted
> : and # and ## processing has taken place, all placemarker preprocessing
> : tokens are removed. The resulting preprocessing token sequence is
> : then rescanned, along with all subsequent preprocessing tokens of
> : the source file, for more macro names to replace.
>
> So, after macro expanded parameters are substituted and # and ##
> handling is done you simply prepend the result to source text
> following macro call and start reading from the start of resulting
> token list.  In other words there are _no_ repeated rescanning,
> rescanning is done just once and during rescanning you may have
> macro call which begins withing replacement list but ends in the
> source text.

I don't understand. If you insert the expanded macro result back into 
the source, and re-start normal tokenising from the start of that new 
sequence, isn't that effectively doing a re-scan anyway? And if a new 
macro is identified, then the process repeats.

But an advantage I can see if that if a new macro is encountered near 
the end of the replacement sequence, you don't have to re-scan all of 
the sequence up to that point after that macro is expanded, only the 
last bit.

Also, it takes care of the f(2)(9) example from elsewhere, as this is 
awkward when the f(2) expands to macro g, but the g and the () are in 
different placed.

> Note: most of the time repeated rescanning just wastes time.
> But sometimes it gives wrong result (see examples at the end).

> Executive summary: rules in the standard are carefully written
> to allow fast one pass implementation, but give freedom to
> the implementer to choose other, less efficient way.
>
> Few examples:
>
> #define f a d d (x)
> #define g a b (c)
> #define h(x, y) x y
> #define hash #
> #define i j j
> #define j i
>
> // This form used to check for modern comment handling
>
> // No rescanning when reading from source
> a b (c)
> a d (c)
> // Expanded macro argument is rescanned
> d(a b (c))
> d(a d (c))

I don't get these. Have you left out definitions for a and d?

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#101324

On 08/01/2017 23:53, BartC wrote:
> On 08/01/2017 22:54, antispam@math.uni.wroc.pl wrote:
>> BartC <bc@freeuk.com> wrote:
>
>>> * The result is now scanned again looking for new macros to expand.
>>>   However, once any name (eg. M and W) has been expanded once, it is
>>>   never expanded again (in a subsequent pass). So only XX is changed
>>>   to result in:
>>>
>>>    10 20 "10" 99 M(a,b) W+W+20+x
>>
>> AFAICS this is your interetation, which is mostly correct but
>> in corner cases may produce wrong result and seem to be much
>> less efficient then what standard says:
>>
>> : After all parameters in the replacement list have been substituted
>> : and # and ## processing has taken place, all placemarker preprocessing
>> : tokens are removed. The resulting preprocessing token sequence is
>> : then rescanned, along with all subsequent preprocessing tokens of
>> : the source file, for more macro names to replace.
>>
>> So, after macro expanded parameters are substituted and # and ##
>> handling is done you simply prepend the result to source text
>> following macro call and start reading from the start of resulting
>> token list.  In other words there are _no_ repeated rescanning,
>> rescanning is done just once and during rescanning you may have
>> macro call which begins withing replacement list but ends in the
>> source text.

Yes, this is an good example of that:

#define A(x,y) if(x==y)
#define B A(

int main(void){
   B 10,20);
}

The expanded code should be 'if(10==20)'. But out of six compilers I 
tried, 3 had problems. (One crashed, one ran out of memory, another - 
MSVC 2008 - reported an error about A not having enough arguments.

However this variation:

#define A(x,y) if(x==y)
#define B A

int main(void){
   B (10,20);
}

was generally OK. The new macro call is still split like this:

   A / (10,20)         # A in replacement sequence, rest in main source

but that seems to be less objectionable.

Also, K&R2, page 230, says this:

"In both kinds of macro, the replacement token sequence is repeatedly 
rescanned for more defined identifiers." (Then going on about only 
replacing each identifier once.)

I think I might go with my current scheme for now (which has a tweak to 
fix the second case above where the replacement text ends with a new 
macro name, and the "(...)" follows in the main text). Because I've 
already solved the problems with controlling macro replacement, which 
will be harder when it's mixed with the main text.

(I need to get on with the rest of my compiler and can live with a not 
quite perfect preprocessor until I can return to it later on when 
everything else is finished, and try a different approach. I think if it 
was OK for Microsoft - in 2008 anyway - it'll do for me!)


-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#101326

BartC <bc@freeuk.com> wrote:
> On 08/01/2017 22:54, antispam@math.uni.wroc.pl wrote:
> > BartC <bc@freeuk.com> wrote:
> 
> >> * The result is now scanned again looking for new macros to expand.
> >>   However, once any name (eg. M and W) has been expanded once, it is
> >>   never expanded again (in a subsequent pass). So only XX is changed
> >>   to result in:
> >>
> >>    10 20 "10" 99 M(a,b) W+W+20+x
> >
> > AFAICS this is your interetation, which is mostly correct but
> > in corner cases may produce wrong result and seem to be much
> > less efficient then what standard says:
> >
> > : After all parameters in the replacement list have been substituted
> > : and # and ## processing has taken place, all placemarker preprocessing
> > : tokens are removed. The resulting preprocessing token sequence is
> > : then rescanned, along with all subsequent preprocessing tokens of
> > : the source file, for more macro names to replace.
> >
> > So, after macro expanded parameters are substituted and # and ##
> > handling is done you simply prepend the result to source text
> > following macro call and start reading from the start of resulting
> > token list.  In other words there are _no_ repeated rescanning,
> > rescanning is done just once and during rescanning you may have
> > macro call which begins withing replacement list but ends in the
> > source text.
> 
> I don't understand. If you insert the expanded macro result back into 
> the source, and re-start normal tokenising from the start of that new
                                  ^^^^^^^^^^
                                macro expansion
> sequence, isn't that effectively doing a re-scan anyway? And if a new 
> macro is identified, then the process repeats.
> 
> But an advantage I can see if that if a new macro is encountered near 
> the end of the replacement sequence, you don't have to re-scan all of 
> the sequence up to that point after that macro is expanded, only the 
> last bit.

Note: if you find new macro close to start of replacement sequence
you abort current scan, perform replacement and restart scan
starting form beginning of replacement text of the new macro.
So, effectively there is single rescan: if you looked at a token
during rescan and it was not expanded then you will not look at
it again.  If you did not look at a token, this means that the
token really did not take part in the current (re)scan and
will be handled in the future.

> 
> Also, it takes care of the f(2)(9) example from elsewhere, as this is 
> awkward when the f(2) expands to macro g, but the g and the () are in 
> different placed.
> 
> > Note: most of the time repeated rescanning just wastes time.
> > But sometimes it gives wrong result (see examples at the end).
> 
> > Executive summary: rules in the standard are carefully written
> > to allow fast one pass implementation, but give freedom to
> > the implementer to choose other, less efficient way.
> >
> > Few examples:
> >

Missing definitions:

#define a(x) mac_a(x)
#define b
#define d(x) (x)
#define e(x, y, z) x y ## z

> > #define f a d d (x)
> > #define g a b (c)
> > #define h(x, y) x y
> > #define hash #
> > #define i j j
> > #define j i
> >
> > // This form used to check for modern comment handling
> >
> > // No rescanning when reading from source
> > a b (c)
> > a d (c)
> > // Expanded macro argument is rescanned
> > d(a b (c))
> > d(a d (c))
> 
> I don't get these. Have you left out definitions for a and d?

Sorry, this was cut and paste error.

-- 
                              Waldek Hebisch

[toc] | [prev] | [next] | [standalone]

#101374

On 09/01/2017 01:31, antispam@math.uni.wroc.pl wrote:
> BartC <bc@freeuk.com> wrote:

> Missing definitions:
>
> #define a(x) mac_a(x)
> #define b
> #define d(x) (x)
> #define e(x, y, z) x y ## z
>
>>> #define f a d d (x)
>>> #define g a b (c)
>>> #define h(x, y) x y
>>> #define hash #
>>> #define i j j
>>> #define j i
>>>
>>> // This form used to check for modern comment handling
>>>
>>> // No rescanning when reading from source
>>> a b (c)
>>> a d (c)
>>> // Expanded macro argument is rescanned
>>> d(a b (c))
>>> d(a d (c))
>>
>> I don't get these. Have you left out definitions for a and d?
>
> Sorry, this was cut and paste error.

Thanks.

These examples have given enough problems with my algorithm that I 
decided to use another approach and to try the prepend scheme that you 
suggested.

-- 
Bartc

[toc] | [prev] | [standalone]

Page 4 of 4 — ← Prev page 1 2 3 [4]

Back to top | Article view | comp.lang.c

csiph-web