Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #2045

Language standards vs. implementation, was Re: A right alternative to IEEE-754's format

Show all headers | View raw

 [[ this string is copied from comp.arch because your moderation found it interesting ]]

On 2018-04-06 4:01 AM, Tim Rentsch wrote:
> Walter Banks <walter@bytecraft.com> writes:
>>
>> As someone who has done a significant amount of compiler development
>> there really never enough testing.  Once past sanity tests and
>> detailed test suites a lot can be gained just by running regression
>> tests even if the tests are not executed.  Detailed metrics alone
>> can be a very revealing indication of significant compiler problems.
>>
>> This can be especially true while developing optimization a
>> surprising number of new optimizations do not have the intended
>> effect on old functioning programs.
>
> I think what I meant didn't come across the way I meant it.  I
> agree with what (I think) you are saying:  it's hard to test
> compilers effectively, and it is much harder to test heavily
> optimizing compilers effectively.  In fact this raises an
> interesting question, namely, what is a good methodology for
> testing compilers in the presence of heavy optimization and
> programs that may be "broken" in the sense of having undefined
> behavior.  That strikes me as a topic worth pursuing (along
> with all the other topics on that list, which wasn't short to
> begin with).
>
> However all that is orthogonal to what I was trying to say in my
> previous comments.  My comments are not about verifying how a
> compiler behaves but about validating the requirements for how
> the compiler should behave.  More specifically, in what cases do
> we /want/ a compiler to take advantage, or be allowed to take
> advantage, of the freedom that "undefined behavior" provides?  I
> am convinced that in some cases compilers take greater advantage
> than is helpful, in the sense of total cost of ownership.  Let me
> emphasize here:  not greater advantage than the Standard allows,
> but greater advantage than is helpful when considering the whole
> picture.  I don't know what is the solution to this problem, but
> I have no doubt that there is a problem, and one that needs to
> be addressed at a high level rather than a patchwork quilt of
> local fixes.
>
> Did that make things any clearer?  I hope it did.
>

I think we are all on the same page. Add in Terje's comments of
optimizations that work well with a dozen micro-tests and find when
testing larger amounts of code fail in many ways to help and often for
various reason make the application less effective in some way.

Most of my comments are related to embedded systems tools and
applications which partly changes the situation. Embedded systems are
compile once and run a lot of times as opposed to run a few times. This
makes it attractive to spend more cycles at compile time.

We do a lot of regression tests using real application code. In many
cases it isn't practical to execute this code to get execution data. It
is still worth building the code and looking at the compilation metrics.

What comes out of the metrics are clues when a specific optimization
should not be used. We in many cases still keep Terje's micro-tests and
add the cases that did not help that should not be used.

There are cases where two different optimizations each work well but
together are performance reducing. The general case for this type of
optimization failure is generically resource competition. It is not the
general case of the caching example but something more direct for things
like processors with multiple parallel functionally diverse alu's
competing for a specialized rare resource, for example a multiply and
accumulate.

We burn a lot of compiler cycles managing target resources (embedded
systems) We do whole application strategy passes very early in the
compiling process essentially doing two things application data
management and execution flow. Both of these help a lot to decide when
and where an optimization should happen when the application is being built.

Test new optimizations one addition at a time with regression testing
otherwise many surprises get hidden.

w..

Back to comp.compilers | Previous | Next | Find similar

Thread

Language standards vs. implementation, was Re: A right alternative to IEEE-754's format "Walter Banks" <walter@bytecraft.com> - 2018-04-10 11:08 -0400

csiph-web