Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #103363

Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization

Show all headers | View raw

On 07/02/17 15:34, Rick C. Hodgin wrote:
> I had the idea today of instrumenting the generated binary with meta
> data capture abilities.  These would be little counters and flags
> added by the compiler to collect runtime information, the purpose of
> which would be to help it determine how best to optimize code in
> future compiles.
> 
> By instrumenting the binary with this information, it generates a
> live meta file at runtime based on a data set being processed, or a
> use pattern if it's involves human interaction.  These can be fed
> back in to the compiler for subsequent compilations, helping it make
> choices which best serve that data set, or that user.
> 
> In such a case, and with the speed of computers growing by leaps and
> bounds, and with the open-source nature of source code becoming far
> more widely used, it may be such that a user could download the
> instrumented version of their software, run it for a while, and then
> submit their live meta file to a build bot, which then generates for
> them a custom build tailored to their data set or use pattern.
> 
> Multiple live meta files could be used as input to create multiple
> static versions people could download without having to give their
> own feedback, etc.
> 

<https://en.wikipedia.org/wiki/Profile-guided_optimization>

It's been available in compilers such as MSVC and gcc for decades.  Some
people find it useful, others find it to inconvenient to use.  A
particular problem is that it can severely limit the optimisation and
code generation - it is very hard to instrumentalise functions without
affecting inlining, cloning, and other optimisations.  It gets even
worse if you want to instrumentalise loops, blocks, or other parts of
the code.  Thus your profiling could give data that simply does not
match what happens in optimised code.

Alternatives to instrumenting code are to use sampling to build up
statistics (use a timer-based interrupt to sample the program counter in
the main program), and "hardware performance counters" that can count
the number of times a particular piece of code is run without needing
software changes.  These methods have their own advantages and
disadvantages, of course.

All in all, profiling like this can be useful for some types of
software, but it is an art - it is not something you can just "switch
on" and it will result in better code.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Instrumenting binaries with meta-gatherers for subsequent recompilation optimization "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-02-07 06:34 -0800
  Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization David Brown <david.brown@hesbynett.no> - 2017-02-07 16:14 +0100
    Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-02-07 07:22 -0800
      Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-02-07 07:30 -0800
        Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization David Brown <david.brown@hesbynett.no> - 2017-02-08 09:40 +0100
          Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-02-08 03:14 -0800
            Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization David Brown <david.brown@hesbynett.no> - 2017-02-08 14:20 +0100
              Re: Instrumenting binaries with meta-gatherers for subsequent recompilation optimization "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-02-08 05:24 -0800

csiph-web