Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #146704 > unrolled thread

Preprocessor and "empty" substitutions

Back to article view | Back to comp.lang.c

Contents

  Preprocessor and "empty" substitutions Andrey Tarasevich <andreytarasevich@hotmail.com> - 2019-10-07 12:03 -0700
    Re: Preprocessor and "empty" substitutions Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-10-07 21:05 +0100
      Re: Preprocessor and "empty" substitutions Andrey Tarasevich <andreytarasevich@hotmail.com> - 2019-10-07 15:16 -0700
        Re: Preprocessor and "empty" substitutions Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-10-08 01:41 +0100
          Re: Preprocessor and "empty" substitutions Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-10-09 01:58 +0100
          Re: Preprocessor and "empty" substitutions Andrey Tarasevich <andreytarasevich@hotmail.com> - 2019-10-08 11:41 -0700
            Re: Preprocessor and "empty" substitutions Tim Rentsch <tr.17687@z991.linuxsc.com> - 2019-10-09 06:34 -0700

#146704 — Preprocessor and "empty" substitutions

Hello

Consider the following two simple tests

   #define OUTPUT(x) printf("%d\n", x)

   #define EMPTY
   #define TEST(x) OUTPUT EMPTY (x)

   int main(int argc, char **argv)
   {
     int i = 42;
     TEST(i);
   }

   http://coliru.stacked-crooked.com/a/fe355c96201c9fb1

and

   #define OUTPUT(x) printf("%d\n", x)

   #define TEST(x, y) OUTPUT y (x)

   int main(int argc, char **argv)
   {
     int i = 42;
     TEST(i, );
   }

   http://coliru.stacked-crooked.com/a/537978e2b5888de3

The idea is to inject something "empty" into the replacement list of 
macro 'TEST' between 'OUTPUT' and '(x)'. In the first case that 
"emptiness" arrives through an independently-defined object-like macro 
'EMPTY'. In the second case it is brought in by a macro parameter 'y'.

These two cases behave differently. In the first case the further rescan 
does not recognize 'OUTPUT(x)' invocation in 'TEST's replacement list. 
In the second case the 'OUTPUT(x)' invocation is recognized.

Apparently, in the first case the macro 'EMPTY' is not that "empty": it 
leaves something behind after replacement (an empty pp-token?). Whatever 
it leaves behind apparently prevents the preprocessor from seeing tokens 
'OUTPUT' and '(' as immediately adjacent. The invocation of 
function-like macro 'OUTPUT(x)' is not recognized. (Is this analysis 
correct?)

In the second case parameter 'y' is apparently truly "empty" - it 
disappears without a trace, turning 'OUTPUT' and '(' into immediate 
neighbors. Subsequent rescan sees 'OUTPUT(x)' invocation.

So, what exactly happens in the first case? What does 'EMPTY' leave 
behind after substitution? Which specific wording the the standard says 
that empty parameter values are truly "empty", while empty macros are not?

-- 
Best regards,
Andrey Tarasevich

[toc] | [next] | [standalone]

#146707

Andrey Tarasevich <andreytarasevich@hotmail.com> writes:

> Consider the following two simple tests
>
>   #define OUTPUT(x) printf("%d\n", x)
>
>   #define EMPTY
>   #define TEST(x) OUTPUT EMPTY (x)
>
>   int main(int argc, char **argv)
>   {
>     int i = 42;
>     TEST(i);
>   }

Here is the sequence (I'll put tokens in <> to make them stand out):

A function-like macro is detected because <TEST> is followed by <(>.
Arguments are collected.  We get <i>.  The token list

  <OUTPUT> <EMPTY> <(> <x> <)>

has parameter <x> replaced by <i> to get:

  <OUTPUT> <EMPTY> <(> <i> <)>

Now this list is scanned for further replacement, but the macro name
token <OUTPUT> is not followed by a <(> token so no function-like macro
invocation is seen.

<EMPTY> is seen as an object-like macro and is replaced by nothing, but
the scan does not go back to see if this makes any difference to the
decision not to recognise an invocation of a function-like macro.  No
further tokens can be replaced and, importantly, the resulting list:

  <OUTPUT> <(> <i> <)>
  
is /not/ scanned again.

<cut>
>   #define OUTPUT(x) printf("%d\n", x)
>
>   #define TEST(x, y) OUTPUT y (x)
>
>   int main(int argc, char **argv)
>   {
>     int i = 42;
>     TEST(i, );
>   }

In this case, the token list that results from parameter substitution is:

  <OUTPUT> <(> <x> <)>

and, since the list /is/ re-scanned after parameter substitution, the
invocation of the function-like macro is seen and replaced.

<cut>
> So, what exactly happens in the first case? What does 'EMPTY' leave
> behind after substitution? Which specific wording the the standard
> says that empty parameter values are truly "empty", while empty macros
> are not?

It's not really about what's empty and what isn't.  I hope the above
explains it though.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#146711

On 10/7/2019 1:05 PM, Ben Bacarisse wrote:
> 
> <EMPTY> is seen as an object-like macro and is replaced by nothing, but
> the scan does not go back to see if this makes any difference to the
> decision not to recognise an invocation of a function-like macro.  No
> further tokens can be replaced and, importantly, the resulting list:
> 
>    <OUTPUT> <(> <i> <)>
>    
> is /not/ scanned again.
> 

Interesting. Well, if we try something like

     #define A x
     #define B A
     #define C B
     #define D C

     int main(void)
     {
       int D = 42;
       printf("%d\n", x); // outputs '42'
     }

we'll see that the above occurrence of macro name 'D' is ultimately 
replaced by 'x', meaning that there is some kind of 
_repetitive/recursive_ rescan involved. Once 'D' is replaced with 'C', 
the result _is_ rescanned again, which leads to 'C's being replaced with 
'B' and so on until it reaches 'x'.

I presume that what you mean is that the rescan process is done 
_locally_. 6.10.3/9 (http://port70.net/~nsz/c/c11/n1570.html#6.10.3p9) says

    "The replacement list is then rescanned for more macro names as 
specified below."

I initially assumed that the term "replacement list" in this quote is 
referring to the entire replacement list of the top-level macro. But 
apparently it is not the case: it refers to the just-substituted 
replacement list. I.e. the [recursive] rescan is done locally.

On the other hand, the above 6.10.3/9 quote sounds like a mere reference 
("...as specified below") to a complete specification. Apparently, 
"6.10.3.4 Rescanning and further replacement" is intended to serve as a 
complete specification of the rescan procedure. However, it is not 
immediately obvious from 6.10.3.4 whether it is intended to mandate a 
repetitive global rescan or local rescans.

-- 
Best regards,
Andrey Tarasevich

[toc] | [prev] | [next] | [standalone]

#146720

Andrey Tarasevich <andreytarasevich@hotmail.com> writes:

> On 10/7/2019 1:05 PM, Ben Bacarisse wrote:
>>
>> <EMPTY> is seen as an object-like macro and is replaced by nothing, but
>> the scan does not go back to see if this makes any difference to the
>> decision not to recognise an invocation of a function-like macro.  No
>> further tokens can be replaced and, importantly, the resulting list:
>>
>>    <OUTPUT> <(> <i> <)>
>>    is /not/ scanned again.
>>
>
> Interesting. Well, if we try something like
>
>     #define A x
>     #define B A
>     #define C B
>     #define D C
>
>     int main(void)
>     {
>       int D = 42;
>       printf("%d\n", x); // outputs '42'
>     }
>
> we'll see that the above occurrence of macro name 'D' is ultimately
> replaced by 'x', meaning that there is some kind of
> _repetitive/recursive_ rescan involved.

If that's how you like to think of it, then go for it.  I just say that
when a macro is expanded, that expansion is scanned for more macros.

The reason I don't like using "repetitive" or "recursive" in this
context is that C's macros famously can't "loop" or "recurse" in the
conventional sense, as is permitted in many other macro processors.

> Once 'D' is replaced with 'C', the result _is_ rescanned again, which
> leads to 'C's being replaced with 'B' and so on until it reaches 'x'.

Yes.  Your stress on "_is_" suggests you thought I was saying something
that contradicts this.  I did not intend to, though I did skip a step,
because when EMPTY is replaced by nothing, that empty replacement list
is (logically) scanned for more macros.

> I presume that what you mean is that the rescan process is done
> _locally_.

Well, I meant what I said but it was obviously not clear.

> 6.10.3/9 (http://port70.net/~nsz/c/c11/n1570.html#6.10.3p9) says
>
>    "The replacement list is then rescanned for more macro names as
>    specified below."
>
> I initially assumed that the term "replacement list" in this quote is
> referring to the entire replacement list of the top-level macro.

To my mind, that would be an odd interpretation.  It seems obvious that
"the replacement list" refers to the one in the definition being
expanded.

The "re" in rescanned is maybe a bit odd because there has been (as far
as I can see) no previous scan of that list.  I would just have said
that "the replacement list is then scanned for more macro names".

I think if your first thought were the intended meaning, it would have
to be stated very clearly that the scan was not of this macro's
replacement but of the whole replacement list of the outermost expansion
(if there is one).

> But apparently it is not the case: it refers to the just-substituted
> replacement list. I.e. the [recursive] rescan is done locally.

I'm not keen on that way of describing it but if it works for you,
that's fine.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#146729

Andrey Tarasevich <andreytarasevich@hotmail.com> writes:

> On 10/7/2019 5:41 PM, Ben Bacarisse wrote:
>>>
>>> we'll see that the above occurrence of macro name 'D' is ultimately
>>> replaced by 'x', meaning that there is some kind of
>>> _repetitive/recursive_ rescan involved.
>>
>> If that's how you like to think of it, then go for it.  I just say that
>> when a macro is expanded, that expansion is scanned for more macros.
>
> Well, the text of the standard seem to draw a pretty specific
> distinction between how parameter values are substituted and how
> nested macro invocations are substituted:

Yes it does.  I'm not sure what your point is or if you still have a
question.  Do you understand your original example now?

<cut>
>>> 6.10.3/9 (http://port70.net/~nsz/c/c11/n1570.html#6.10.3p9) says
>>>
>>>     "The replacement list is then rescanned for more macro names as
>>>     specified below."
>>>
>>> I initially assumed that the term "replacement list" in this quote is
>>> referring to the entire replacement list of the top-level macro.
>>
>> To my mind, that would be an odd interpretation.  It seems obvious that
>> "the replacement list" refers to the one in the definition being
>> expanded.
>
> May be. But in that case it is not clear why wouldn't they just unify
> the specification of macro parameter substitution with the
> specification of macro invocation: first, exhaustively substitute
> everything that can be substituted in the replacement list and only
> then plant the result into the target location.

That would produce a macro processor with very different semantics.  Why
do you think it would be better?  The scan on the replacement text
allows the expansion to use following tokens like this:

  #define f g
  #define g(x) <x>

  f(1)

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#146767

On 10/7/2019 5:41 PM, Ben Bacarisse wrote:
>>
>> we'll see that the above occurrence of macro name 'D' is ultimately
>> replaced by 'x', meaning that there is some kind of
>> _repetitive/recursive_ rescan involved.
> 
> If that's how you like to think of it, then go for it.  I just say that
> when a macro is expanded, that expansion is scanned for more macros.

Well, the text of the standard seem to draw a pretty specific 
distinction between how parameter values are substituted and how nested 
macro invocations are substituted:

* In case of parameters it states that first of all, any macro 
substitutions inside the parameter value are processed. And only when 
this process is complete (i.e. everything is exhaustively substituted) 
the final result is planted into the parent macro.

* In case of nested macro invocations it is the other way around: the 
substitution list for the invoked macro is immediately planted into the 
parent macro, and only afterwards that planted list is rescanned for 
more macro invocations.

This appears to be done for some specific reason. And the first reason 
for that distinction that come to mind is that parameters are intended 
to be processed out-of-context, while macro invocations are intended to 
be handled (rescanned) in-context (in the context of surrounding tokens).

> 
>> I presume that what you mean is that the rescan process is done
>> _locally_.
> 
> Well, I meant what I said but it was obviously not clear.
> 
>> 6.10.3/9 (http://port70.net/~nsz/c/c11/n1570.html#6.10.3p9) says
>>
>>     "The replacement list is then rescanned for more macro names as
>>     specified below."
>>
>> I initially assumed that the term "replacement list" in this quote is
>> referring to the entire replacement list of the top-level macro.
> 
> To my mind, that would be an odd interpretation.  It seems obvious that
> "the replacement list" refers to the one in the definition being
> expanded.

May be. But in that case it is not clear why wouldn't they just unify 
the specification of macro parameter substitution with the specification 
of macro invocation: first, exhaustively substitute everything that can 
be substituted in the replacement list and only then plant the result 
into the target location.

-- 
Best regards,
Andrey Tarasevich

[toc] | [prev] | [next] | [standalone]

#146799

Andrey Tarasevich <andreytarasevich@hotmail.com> writes:

> On 10/7/2019 5:41 PM, Ben Bacarisse wrote:
>
>>> we'll see that the above occurrence of macro name 'D' is ultimately
>>> replaced by 'x', meaning that there is some kind of
>>> _repetitive/recursive_ rescan involved.
>>
>> If that's how you like to think of it, then go for it.  I just say that
>> when a macro is expanded, that expansion is scanned for more macros.
>
> Well, the text of the standard seem to draw a pretty specific
> distinction between how parameter values are substituted and how
> nested macro invocations are substituted:
>
> * In case of parameters it states that first of all, any macro
> substitutions inside the parameter value are processed.  And only
> when this process is complete (i.e. everything is exhaustively
> substituted) the final result is planted into the parent macro.
>
> * In case of nested macro invocations it is the other way around:
> the substitution list for the invoked macro is immediately planted
> into the parent macro, and only afterwards that planted list is
> rescanned for more macro invocations.
>
> This appears to be done for some specific reason.  And the first
> reason for that distinction that come to mind is that parameters
> are intended to be processed out-of-context, while macro invocations
> are intended to be handled (rescanned) in-context (in the context of
> surrounding tokens).
>
>>> I presume that what you mean is that the rescan process is done
>>> _locally_.
>>
>> Well, I meant what I said but it was obviously not clear.
>>
>>> 6.10.3/9 (http://port70.net/~nsz/c/c11/n1570.html#6.10.3p9) says
>>>
>>>     "The replacement list is then rescanned for more macro names as
>>>     specified below."
>>>
>>> I initially assumed that the term "replacement list" in this quote is
>>> referring to the entire replacement list of the top-level macro.
>>
>> To my mind, that would be an odd interpretation.  It seems obvious that
>> "the replacement list" refers to the one in the definition being
>> expanded.
>
> May be.  But in that case it is not clear why wouldn't they just
> unify the specification of macro parameter substitution with the
> specification of macro invocation:  first, exhaustively substitute
> everything that can be substituted in the replacement list and
> only then plant the result into the target location.

I think there are two points that are causing you confusion.  One
is the idea that macro calls are "nested".  The other is the idea
that how macro /arguments/ are expanded is somehow fundamentally
different than how macro /definitions/ are expanded.  Neither of
these ideas is really right.  The first is sometimes a convenient
way to think about defining a macro but doesn't exactly match
how expansion is done.  The second is simply wrong.  Let me see
if I can explain.

Macro calls are not "nested" because further calls are not
expanded until the top-level expansion for the current call has
been substituted.  That matters because the top-level expansion
may cause further calls to come into being, or it may cause
tokens that look like they would be calls not actually to be
calls.  A lot of times it helps to think about macro definitions
as having nested calls, but it's important to know that this view
is a simplification and sometimes it breaks down.

The second point may be more telling.  How macro arguments are
expanded is almost exactly the same as how macro definitions are
expanded.  There are some differences, and the differences are
not trivial.  However, it is not the case that macro arguments
are "exhaustively substituted" before being plugged into a macro
definition.  The process is exactly the same as expanding a
parameterless macro called at the end of a translation unit.
It can be hard to see the sameness, because macro arguments
are scanned a second time after being expanded, as part of
processing the macro definition.  Here is an example to help
see what's going on, using the OUTPUT/EMPTY method used in your
original posting:

    #define MYSTERY_GUEST   OUTPUT  EMPTY  (3)
    #define OUTPUT(x)       show x
    #define EMPTY

    #define TWO(x)  ONE(x)
    #define ONE(x)  ZERO(x)
    #define ZERO(x) #x

    ONE( MYSTERY_GUEST  )

    TWO( MYSTERY_GUEST )

Output of 'gcc -E -' (but leaving out the preprocessor line
directives):

    "OUTPUT (3)"

    "show 3"

This example shows that macro arguments are not exhaustively
expanded before being substituted in the expanded macro body.  Do
you see why?  Can you explain what is going on that causes this
result to be produced?

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web