Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1351220 > unrolled thread

[PATCH v5 0/5] Introduce GCC plugin infrastructure

Back to article view | Back to linux.kernel

Contents

  [PATCH v5 0/5] Introduce GCC plugin infrastructure Emese Revfy <re.emese@gmail.com> - 2016-03-07 00:10 +0100
    [PATCH v5 3/5] Add Cyclomatic complexity GCC plugin Emese Revfy <re.emese@gmail.com> - 2016-03-07 00:10 +0100
      Re: [PATCH v5 3/5] Add Cyclomatic complexity GCC plugin Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-11 07:30 +0100
        Re: [PATCH v5 3/5] Add Cyclomatic complexity GCC plugin Emese Revfy <re.emese@gmail.com> - 2016-03-14 22:10 +0100
    [PATCH v5 5/5] Add sancov plugin Emese Revfy <re.emese@gmail.com> - 2016-03-07 00:10 +0100
      Re: [PATCH v5 5/5] Add sancov plugin Kees Cook <keescook@chromium.org> - 2016-03-07 22:10 +0100
        Re: [PATCH v5 5/5] Add sancov plugin Emese Revfy <re.emese@gmail.com> - 2016-03-07 22:40 +0100
        Re: [PATCH v5 5/5] Add sancov plugin Dmitry Vyukov <dvyukov@google.com> - 2016-03-08 12:00 +0100
    [PATCH v5 1/5] Shared library support Emese Revfy <re.emese@gmail.com> - 2016-03-07 00:10 +0100
      Re: [PATCH v5 1/5] Shared library support Kees Cook <keescook@chromium.org> - 2016-03-07 22:10 +0100
        Re: [PATCH v5 1/5] Shared library support Emese Revfy <re.emese@gmail.com> - 2016-03-07 22:40 +0100
      Re: [PATCH v5 1/5] Shared library support Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-11 07:30 +0100
        Re: [PATCH v5 1/5] Shared library support Emese Revfy <re.emese@gmail.com> - 2016-03-14 21:20 +0100
          Re: [PATCH v5 1/5] Shared library support Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-16 09:00 +0100
    Re: [kernel-hardening] [PATCH v5 2/5] GCC plugin infrastructure David Brown <david.brown@linaro.org> - 2016-03-09 10:10 +0100
      Re: [kernel-hardening] [PATCH v5 2/5] GCC plugin infrastructure Kees Cook <keescook@chromium.org> - 2016-03-09 22:00 +0100
        Re: [kernel-hardening] [PATCH v5 2/5] GCC plugin infrastructure Emese Revfy <re.emese@gmail.com> - 2016-03-09 23:10 +0100
      Re: [kernel-hardening] [PATCH v5 2/5] GCC plugin infrastructure Emese Revfy <re.emese@gmail.com> - 2016-03-09 23:10 +0100
    Re: [PATCH v5 2/5] GCC plugin infrastructure Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-11 07:30 +0100
      Re: [PATCH v5 2/5] GCC plugin infrastructure Emese Revfy <re.emese@gmail.com> - 2016-03-14 22:00 +0100
        Re: [PATCH v5 2/5] GCC plugin infrastructure Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-16 08:50 +0100
          Re: [PATCH v5 2/5] GCC plugin infrastructure Emese Revfy <re.emese@gmail.com> - 2016-03-16 22:10 +0100
      Re: [PATCH v5 2/5] GCC plugin infrastructure "PaX Team" <pageexec@freemail.hu> - 2016-03-14 22:30 +0100
        Re: [PATCH v5 2/5] GCC plugin infrastructure Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-16 08:40 +0100
          Re: [PATCH v5 2/5] GCC plugin infrastructure "PaX Team" <pageexec@freemail.hu> - 2016-03-16 14:00 +0100
            Re: [PATCH v5 2/5] GCC plugin infrastructure Masahiro Yamada <yamada.masahiro@socionext.com> - 2016-03-17 05:20 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1358599 — Re: [PATCH v5 2/5] GCC plugin infrastructure

Hi Emese,

2016-03-15 5:52 GMT+09:00 Emese Revfy <re.emese@gmail.com>:
> On Fri, 11 Mar 2016 15:25:19 +0900
> Masahiro Yamada <yamada.masahiro@socionext.com> wrote:
>
>> Maybe scripts/gcc-plugins/ is better than tools/gcc ?
>>
>> In the directory "scripts/", we have several tools used during
>> building the kernel image.
>> We have some optional programs in the directory "tools/", which are not used
>> for building the kernel image itself.
>>
>> Please correct me if I am wrong.
>
> I think scripts isn't the right place because there are tools there which don't get used
> during the building of vmlinux (e.g., coccinelle, checkpatch).

No, this is not what I meant.

What I want to say is:
All the host tools used for building vmlinux should be located in the
directory "scripts/",
but the directory may also other host tools that are not used for
building vmlinux (such as checkpatch).


> The scripts name also doesn't describe gcc plugins well.

With my suggestion "scripts/gcc-plugins/",
the sub-directory name describes it very well.


> Plugins take part in the image building process
> in a different way than these tools and scripts do.
> Since there doesn't seen to be a good place for compiler plugins, maybe we should create a new toplevel directory
> called "build". Compiler plugins and other existing build tools could live there. What do you think?

In my understanding, they are plug-in'ed into the cross-compiler that
compiles vmlinux.

If so, GCC plugins should be happy in "scripts/"
because Kbuild descends into "scripts/" before building any objects for vmlinux.


-- 
Best Regards
Masahiro Yamada

[toc] | [prev] | [next] | [standalone]

#1359364 — Re: [PATCH v5 2/5] GCC plugin infrastructure

On Wed, 16 Mar 2016 16:41:36 +0900
Masahiro Yamada <yamada.masahiro@socionext.com> wrote:

> > The scripts name also doesn't describe gcc plugins well.
> 
> With my suggestion "scripts/gcc-plugins/",
> the sub-directory name describes it very well.
> 
> 
> > Plugins take part in the image building process
> > in a different way than these tools and scripts do.
> > Since there doesn't seen to be a good place for compiler plugins, maybe we should create a new toplevel directory
> > called "build". Compiler plugins and other existing build tools could live there. What do you think?
> 
> In my understanding, they are plug-in'ed into the cross-compiler that
> compiles vmlinux.
> 
> If so, GCC plugins should be happy in "scripts/"
> because Kbuild descends into "scripts/" before building any objects for vmlinux.

Ok, I'll move them under scripts/ in the next patch set.

-- 
Emese

[toc] | [prev] | [next] | [standalone]

#1357636 — Re: [PATCH v5 2/5] GCC plugin infrastructure

On 11 Mar 2016 at 15:25, Masahiro Yamada wrote:

> > diff --git a/scripts/gcc-plugin.sh b/scripts/gcc-plugin.sh
> > new file mode 100644
> > index 0000000..eaa4fce
> > --- /dev/null
> > +++ b/scripts/gcc-plugin.sh
> > @@ -0,0 +1,51 @@
> > +#!/bin/sh
> > +srctree=$(dirname "$0")
> > +gccplugins_dir=$($3 -print-file-name=plugin)
> > +plugincc=$($1 -E -x c++ - -o /dev/null -I"${srctree}"/../tools/gcc -I"${gccplugins_dir}"/include 2>&1 <<EOF
> > +#include "gcc-common.h"
> 
> 
> Maybe <gcc-common.h> because it is not located at the same directory?

[snip]
> > +#include "emit-rtl.h"
> > +#include "debug.h"
> > +#include "target.h"
> > +#include "langhooks.h"
> > +#include "cfgloop.h"
> > +#include "cgraph.h"
> > +#include "opts.h"
> 
> All of these are included by "...", not <...>.
> 
> 
> As mentioned above, I want you to use "..." style
> when you need to use relative path from the source.
> 
> I do not see most of them in tools/gcc/.

no, that'd be incorrect for several reasons. first, the rule to use <...> vs. "..."
include directives is not about the header being in the same directory but whether
the header is a system header or one provided by the given program. roughly speaking,
system headers are those that are available through the compiler's default include
paths (gcc's own headers, those of glibc and other libraries under /usr/include, etc).
gcc plugin headers are *not* available by default, one has to query the compiler about
their path (see -print-file-name=plugin above) and explicitly add it to the compiler's
include search path.

second, regardless of whether plugin headers are available by default or not, we
still couldn't use them during cross-compilation as the plugin headers we want are
those of the target compiler (that will load the plugin eventually), not that of
the host compiler (which merely compiles the plugin and in theory doesn't even have
to be gcc or a plugin capable gcc).

for these reasons the correct include directive is "..." and not <...>. if it helps
to understand the situation better, consider that gcc plugins are to gcc as kernel
modules are to vmlinux and all kernel headers are included via "..." as well, regardless
of whether they're in the same directory or not.

cheers,
 PaX Team

[toc] | [prev] | [next] | [standalone]

#1358576 — Re: [PATCH v5 2/5] GCC plugin infrastructure

Hi PaX,


2016-03-15 6:25 GMT+09:00 PaX Team <pageexec@freemail.hu>:
> On 11 Mar 2016 at 15:25, Masahiro Yamada wrote:
>
>> > diff --git a/scripts/gcc-plugin.sh b/scripts/gcc-plugin.sh
>> > new file mode 100644
>> > index 0000000..eaa4fce
>> > --- /dev/null
>> > +++ b/scripts/gcc-plugin.sh
>> > @@ -0,0 +1,51 @@
>> > +#!/bin/sh
>> > +srctree=$(dirname "$0")
>> > +gccplugins_dir=$($3 -print-file-name=plugin)
>> > +plugincc=$($1 -E -x c++ - -o /dev/null -I"${srctree}"/../tools/gcc -I"${gccplugins_dir}"/include 2>&1 <<EOF
>> > +#include "gcc-common.h"
>>
>>
>> Maybe <gcc-common.h> because it is not located at the same directory?
>
> [snip]
>> > +#include "emit-rtl.h"
>> > +#include "debug.h"
>> > +#include "target.h"
>> > +#include "langhooks.h"
>> > +#include "cfgloop.h"
>> > +#include "cgraph.h"
>> > +#include "opts.h"
>>
>> All of these are included by "...", not <...>.
>>
>>
>> As mentioned above, I want you to use "..." style
>> when you need to use relative path from the source.
>>
>> I do not see most of them in tools/gcc/.
>
> no, that'd be incorrect for several reasons. first, the rule to use <...> vs. "..."
> include directives is not about the header being in the same directory but whether
> the header is a system header or one provided by the given program. roughly speaking,
> system headers are those that are available through the compiler's default include
> paths (gcc's own headers, those of glibc and other libraries under /usr/include, etc).
> gcc plugin headers are *not* available by default, one has to query the compiler about
> their path (see -print-file-name=plugin above) and explicitly add it to the compiler's
> include search path.

Are you sure?

I'd recommend you to read the following:
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html


scripts/gcc-plugin.sh adds
  - ${srctree}"/../tools/gcc
  - ${gccplugins_dir}/include

to the header search path with "-I" option.


GCC looks for headers requested with #include <file>
in directories specified with "-I" as well as in the default ones.



> second, regardless of whether plugin headers are available by default or not, we
> still couldn't use them during cross-compilation as the plugin headers we want are
> those of the target compiler (that will load the plugin eventually), not that of
> the host compiler (which merely compiles the plugin and in theory doesn't even have
> to be gcc or a plugin capable gcc).

So, how could this be the reason why we should avoid #include <file>?



> for these reasons the correct include directive is "..." and not <...>. if it helps
> to understand the situation better, consider that gcc plugins are to gcc as kernel
> modules are to vmlinux and all kernel headers are included via "..." as well, regardless
> of whether they're in the same directory or not.

Of course, you could use #include "..." as well to include kernel headers,
but you should not do that.

You should do so only when you need to include headers that are local
to your directory.


-- 
Best Regards
Masahiro Yamada

[toc] | [prev] | [next] | [standalone]

#1358963 — Re: [PATCH v5 2/5] GCC plugin infrastructure

On 16 Mar 2016 at 16:34, Masahiro Yamada wrote:

Hi,

> >> As mentioned above, I want you to use "..." style
> >> when you need to use relative path from the source.
> >>
> >> I do not see most of them in tools/gcc/.
> >
> > no, that'd be incorrect for several reasons. first, the rule to use <...> vs. "..."
> > include directives is not about the header being in the same directory but whether
> > the header is a system header or one provided by the given program. roughly speaking,
> > system headers are those that are available through the compiler's default include
> > paths (gcc's own headers, those of glibc and other libraries under /usr/include, etc).
> > gcc plugin headers are *not* available by default, one has to query the compiler about
> > their path (see -print-file-name=plugin above) and explicitly add it to the compiler's
> > include search path.
> 
> Are you sure?

sure about which part? ;) the use of <...> vs "..." is a matter of convention (to
differentiate between 'system' headers from the program's own) and for gcc the plugin
headers don't count as 'system' headers, gcc uses "..." throughout its codebase itself
(so does clang/llvm). it'd look weird and inconsistent if compiler plugins used <...>.

at the end of the day this comes down to whether gcc plugins are considered kernel
code and thus the kernel style applies to them (the kernel itself uses <...> because
it's a freestanding program so its 'system' headers are mostly what it provides
itself, not those of the compiler let alone userland libs) or gcc extensions and thus
the gcc include style applies to them. as you can guess, i'm arguing for the latter,
because gcc plugins are meant for the compiler, not the kernel per se, and in fact
we have several plugins already that can be used in userland as well (e.g., Emese's
size overflow plugin).

the one thing for plugins that i borrowed from the kernel instead of gcc is the code
formatting style as the GNU one is as bad as Linus argued decades ago ;).

> > second, regardless of whether plugin headers are available by default or not, we
> > still couldn't use them during cross-compilation as the plugin headers we want are
> > those of the target compiler (that will load the plugin eventually), not that of
> > the host compiler (which merely compiles the plugin and in theory doesn't even have
> > to be gcc or a plugin capable gcc).
> 
> So, how could this be the reason why we should avoid #include <file>?

because <...> is meant for headers that are on the compiler's default search path
and even if gcc plugin headers were there (they aren't) they'd be the wrong ones
to use, you'd have to specifically override the include path with that of the target
compiler which is a sign that you're no longer including 'system' headers (again,
this convention holds for hosted programs, freestanding ones don't have anything
but 'system' headers usually).

> > for these reasons the correct include directive is "..." and not <...>. if it helps
> > to understand the situation better, consider that gcc plugins are to gcc as kernel
> > modules are to vmlinux and all kernel headers are included via "..." as well, regardless
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sorry, i take that back, the kernel uses <...> but for a reason that in my opinion
doesn't apply to compiler plugins (see above).

> > of whether they're in the same directory or not.
> 
> Of course, you could use #include "..." as well to include kernel headers,
> but you should not do that.
> 
> You should do so only when you need to include headers that are local
> to your directory.

sure, a header in the same directory is a sign that it's not a 'system' header but
i'm also saying that there can be other non-system headers in a program in other
directories as well. FWIW, the kernel itself has several "..." directives that
reference headers outside of the same directory, e.g., try this in a kernel tree:

  grep "#include.*\"\." -rn

cheers,
 PaX Team

[toc] | [prev] | [next] | [standalone]

#1359572 — Re: [PATCH v5 2/5] GCC plugin infrastructure

2016-03-16 21:49 GMT+09:00 PaX Team <pageexec@freemail.hu>:
> On 16 Mar 2016 at 16:34, Masahiro Yamada wrote:
>
> Hi,
>
>> >> As mentioned above, I want you to use "..." style
>> >> when you need to use relative path from the source.
>> >>
>> >> I do not see most of them in tools/gcc/.
>> >
>> > no, that'd be incorrect for several reasons. first, the rule to use <...> vs. "..."
>> > include directives is not about the header being in the same directory but whether
>> > the header is a system header or one provided by the given program. roughly speaking,
>> > system headers are those that are available through the compiler's default include
>> > paths (gcc's own headers, those of glibc and other libraries under /usr/include, etc).
>> > gcc plugin headers are *not* available by default, one has to query the compiler about
>> > their path (see -print-file-name=plugin above) and explicitly add it to the compiler's
>> > include search path.
>>
>> Are you sure?
>
> sure about which part? ;)

You said "that'd be incorrect", but
there is no "correct" or "incorrect" in either way as long as it works.

This decision varies from project to project.

At least in Linux, it generally uses #include <file> style
except ones in local directories.
I guess it is debatable if this rule also applies to host tools or not.


> the use of <...> vs "..." is a matter of convention (to
> differentiate between 'system' headers from the program's own) and for gcc the plugin
> headers don't count as 'system' headers, gcc uses "..." throughout its codebase itself
> (so does clang/llvm). it'd look weird and inconsistent if compiler plugins used <...>.
>
> at the end of the day this comes down to whether gcc plugins are considered kernel
> code and thus the kernel style applies to them (the kernel itself uses <...> because
> it's a freestanding program so its 'system' headers are mostly what it provides
> itself, not those of the compiler let alone userland libs) or gcc extensions and thus
> the gcc include style applies to them. as you can guess, i'm arguing for the latter,
> because gcc plugins are meant for the compiler, not the kernel per se, and in fact
> we have several plugins already that can be used in userland as well (e.g., Emese's
> size overflow plugin).
>
> the one thing for plugins that i borrowed from the kernel instead of gcc is the code
> formatting style as the GNU one is as bad as Linus argued decades ago ;).
>
>> > second, regardless of whether plugin headers are available by default or not, we
>> > still couldn't use them during cross-compilation as the plugin headers we want are
>> > those of the target compiler (that will load the plugin eventually), not that of
>> > the host compiler (which merely compiles the plugin and in theory doesn't even have
>> > to be gcc or a plugin capable gcc).
>>
>> So, how could this be the reason why we should avoid #include <file>?
>
> because <...> is meant for headers that are on the compiler's default search path
> and even if gcc plugin headers were there (they aren't) they'd be the wrong ones
> to use, you'd have to specifically override the include path with that of the target
> compiler which is a sign that you're no longer including 'system' headers (again,
> this convention holds for hosted programs, freestanding ones don't have anything
> but 'system' headers usually).
>
>> > for these reasons the correct include directive is "..." and not <...>. if it helps
>> > to understand the situation better, consider that gcc plugins are to gcc as kernel
>> > modules are to vmlinux and all kernel headers are included via "..." as well, regardless
>                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> sorry, i take that back, the kernel uses <...> but for a reason that in my opinion
> doesn't apply to compiler plugins (see above).
>
>> > of whether they're in the same directory or not.
>>
>> Of course, you could use #include "..." as well to include kernel headers,
>> but you should not do that.
>>
>> You should do so only when you need to include headers that are local
>> to your directory.
>
> sure, a header in the same directory is a sign that it's not a 'system' header but
> i'm also saying that there can be other non-system headers in a program in other
> directories as well. FWIW, the kernel itself has several "..." directives that
> reference headers outside of the same directory, e.g., try this in a kernel tree:
>
>   grep "#include.*\"\." -rn
>

To sum up, your suggestion for GCC plugins is

  - Use #include <...> to include header files provided by the host compiler
  - Use #include "..." to include header files provided by the cross
compiler (plugin directory)
  - Use #include "..." to include header files in the local project

Correct?

I leave the final decision to others.


-- 
Best Regards
Masahiro Yamada

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web