Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #1850 > unrolled thread

Re: Parsing partial sentences

Back to article view | Back to comp.compilers

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Parsing partial sentences Martin Ward <martin@gkc.org.uk> - 2017-04-11 10:30 +0100
    Re: Parsing partial sentences Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2017-04-20 16:14 +0200
      Re: Parsing partial sentences George Neuner <gneuner2@comcast.net> - 2017-04-21 07:41 -0400

#1850 — Re: Parsing partial sentences

On 07/04/17 21:24, Hans-Peter Diettrich wrote:
>> It might be easiest to just run the C preprocessor as a 1st pass and
>> then attempt to convert the result.
>
> That would obfuscate the source code widely

On 07/04/17 21:24, John wrote:
> Since preprocessor macros are text macros, there's no reason to
> expect a macro's expansion can be parsed at all.

This is true: handling the full generality of what is possible
with #define macros without first expanding all the macros
is (almost) impossible.

However, in practice it depends on the purpose of your C to Pascal
convertor. If it is to be a general purpose tool which has to be
able to handle any C source code that anyone throws at it,
then you are stuck. If, on the other hand, there is a certain
defined body of C code which you want to translate
then you only need to be able to handle the types of macro
definitions which are actually used in that body of code:
which is likely to be a small subset of the possibilities.
Most macro definitions in most code (other than the original
Bourne shell source code, and some similar examples)
are written such that the code using the macro looks
and parses like ordinary C code.

An intermediate approach might be to keep all the macros
which look straightforward and fall back on expanding macros
which do weird things with the syntax.

--
			Martin

Dr Martin Ward STRL Principal Lecturer & Reader in Software Engineering
martin@gkc.org.uk  http://www.cse.dmu.ac.uk/~mward/  Erdos number: 4
G.K.Chesterton web site: http://www.cse.dmu.ac.uk/~mward/gkc/
Mirrors:  http://www.gkc.org.uk  and  http://www.gkc.org.uk/gkc

[toc] | [next] | [standalone]

#1860

Am 11.04.2017 um 11:30 schrieb Martin Ward:
> On 07/04/17 21:24, Hans-Peter Diettrich wrote:

> On 07/04/17 21:24, John wrote:
>> Since preprocessor macros are text macros, there's no reason to
>> expect a macro's expansion can be parsed at all.
>
> This is true: handling the full generality of what is possible
> with #define macros without first expanding all the macros
> is (almost) impossible.

After thinking more about macro expansion, interpretation of the macro
text should occur only when a macro is actually used, so that all
referenced identifiers (other macros, parameters, variables, functions)
are known to the parser/compiler. At that point the returned
non-terminal from the expanded macro can be used immediately to
determine a possible macro conversion.

Some attributes should be added to the parse tree, so that it will be
possible to determine the macro name for every token, if it results from
a macro expansion, and the kind, type and scope of all used identifiers.
As long as only literal constants occur in the parse tree, the macro can
be converted into a named constant[1]. If identifiers occur as well,
more checks are required to reject e.g. identifiers of non-global scope,
before the macro can be converted into a function[2].
If all tests are passed, the macro equivalent can be constructed and
stored, for later use. If such an equivalent has already been stored,
the new construct has to be compared with that first construct, to sort
out possible variations due to macro redefinition or varying macro
argument types. It may be possible to handle varying argument types by a
transformation into generic functions, or by type expansion.

Provisions also are required (and already implemented) to prevent
duplicate parsing of properly guarded header files, which otherwise
could confuse the automatism about preceding expansions of macros.

Did I miss something?


[1] Since macros can be defined in any order, it's not guaranteed that
all used identifiers have already been defined or declared before.
That's why their classification has to be delayed, at least until all
macros have been defined, e.g. at the end of a header file.

As long as all uses (actual expansions) of a macro have to be checked,
an additional check for constant definitions seems to be abdicable. OTOH
its desireable to also convert header files into the target language,
for later immediate use. Since typically no macro expansion occurs in
header files, and even in a source module not all macros are used, a
special scan for constant macros still looks very helpful to me.
Eventually a pseudo module can be constructed for every header file,
containing a function that uses all macros (without arguments?) which
are #defined in the header. The parser may have to be modified for that
purpose, to skip over macros which do not expand immediately into valid
source code.


[2] Eventually even local variables may become acceptable in converted
functions, when the resulting function becomes local to the scope of the
variables and invoking function. Here OPL puts a limit onto
"localization", because functions can be local only to another function,
not to arbitrary code blocks. Similarly local variables can only be
local to a function, not to code blocks. This case already is handled by
the convertor.


> However, in practice it depends on the purpose of your C to Pascal
> convertor. If it is to be a general purpose tool which has to be
> able to handle any C source code that anyone throws at it,
> then you are stuck.

See above, my convertor only accepts valid C code. In so far the user
can throw any valid C source file at it. It's only required that the
original compiler and its libraries are made known to the convertor, so
that all compiler-predefined macros are known, and all #included header
files are available for parsing.

This is not really different from the usual invocation of a compiler,
which must be given a search path to all used header files. A couple of
compiler specific extensions over the C standard are already
implemented, like the strange macro expansion of '/'##'/' into "//",
found in MSVC, or #include_next in gcc.


> An intermediate approach might be to keep all the macros
> which look straightforward and fall back on expanding macros
> which do weird things with the syntax.

That's the current implementation, which does not yet handle translated
#defines at all.


Thanks for all the inspiration on my project :-)

DoDi

[toc] | [prev] | [next] | [standalone]

#1861

On Thu, 20 Apr 2017 16:14:43 +0200, Hans-Peter Diettrich
<DrDiettrich1@netscape.net> wrote:

>After thinking more about macro expansion, interpretation of the macro
>text should occur only when a macro is actually used, so that all
>referenced identifiers (other macros, parameters, variables, functions)
>are known to the parser/compiler. At that point the returned
>non-terminal from the expanded macro can be used immediately to
>determine a possible macro conversion.
>
>Some attributes should be added to the parse tree, so that it will be
>possible to determine the macro name for every token, if it results from
>a macro expansion, and the kind, type and scope of all used identifiers.
>As long as only literal constants occur in the parse tree, the macro can
>be converted into a named constant[1]. If identifiers occur as well,
>more checks are required to reject e.g. identifiers of non-global scope,
>before the macro can be converted into a function[2].

So far, so good.  Actually converting "function" macros into functions
[where possible] should lead to improved maintenance.

However, in the past I have run into some bizarre macro systems where
some macros create data structures for other macros to manipulate.

I'm just guessing as I don't have an example handy - but considering
such entwined macros independently, I could see manipulation macros
being rejected by your translator for appearing to reference
"identifiers of non-global scope".


>If all tests are passed, the macro equivalent can be constructed and
>stored, for later use. If such an equivalent has already been stored,
>the new construct has to be compared with that first construct, to sort
>out possible variations due to macro redefinition or varying macro
>argument types. It may be possible to handle varying argument types by a
>transformation into generic functions, or by type expansion.
>
>Provisions also are required (and already implemented) to prevent
>duplicate parsing of properly guarded header files, which otherwise
>could confuse the automatism about preceding expansions of macros.
>
>Did I miss something?

Some actual things done with C macros.

http://stackoverflow.com/questions/652788/what-is-the-worst-real-world-macros-pre-processor-abuse-youve-ever-come-across

http://jhnet.co.uk/articles/cpp_magic

https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,-tips,-and-idioms

https://natecraun.net/articles/struct-iteration-through-abuse-of-the-c-preprocessor.html


> Martin Ward said:
>
>> However, in practice it depends on the purpose of your C to Pascal
>> convertor. If it is to be a general purpose tool which has to be
>> able to handle any C source code that anyone throws at it,
>> then you are stuck.
>
>See above, my convertor only accepts valid C code. In so far the user
>can throw any valid C source file at it. It's only required that the
>original compiler and its libraries are made known to the convertor, so
>that all compiler-predefined macros are known, and all #included header
>files are available for parsing.

Seemingly non-sensical macros can lead to valid C code.  In one
stackoverflow example they led to valid Java.


>> An intermediate approach might be to keep all the macros
>> which look straightforward and fall back on expanding macros
>> which do weird things with the syntax.
>
>That's the current implementation, which does not yet handle translated
>#defines at all.


George

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compilers

csiph-web