Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #140046

Re: Compiler tests

From David Brown <david.brown@hesbynett.no>
Newsgroups comp.lang.c
Subject Re: Compiler tests
Date 2019-01-21 09:44 +0100
Organization A noiseless patient Spider
Message-ID <q240pf$ips$1@dont-email.me> (permalink)
References (2 earlier) <m24la3llvv.fsf@despina.home> <q21mhu$jm6$1@dont-email.me> <lp_0E.182687$1D1.4718@fx29.am4> <q227i9$r1s$1@dont-email.me> <y421E.109019$ns2.47564@fx26.am4>

Show all headers | View raw

On 20/01/2019 18:19, Bart wrote:
> On 20/01/2019 16:27, David Brown wrote:
>> On 20/01/2019 14:08, Bart wrote:
>>> On 20/01/2019 11:37, David Brown wrote:
>>>> On 20/01/2019 10:27, Pascal J. Bourguignon wrote:
>>>
>>>>> Well, obviously some compilers try to optimize duplicate expressions,
>>>>> and get lost in the number of them.  Not a realistic test, but
>>>>> definitely a good try-to-break-the-compiler test.
>>>>>
>>>>
>>>> Unfortunately for Bart, compilers are usually targeted to real code.
>>>> And in real code, duplicate expressions and structures /do/ turn up
>>>> (especially in template-based C++ coding), and optimising them is
>>>> very important to the efficiency of the code generation.
>>>
>>> Except that the problem timings were with /unoptimised/ compiles.
>>> With optimisation enabled, they were much faster.
>>>
>>
>> Surely you don't /still/ think that optimisation is a black-or-white
>> issue,
> 
> The conjecture was that the rapid slowdown was because of looking for
> common sub-expressions. Since optimisation was off, that would be less
> likely to be the cause.

Optimisations are not an on/off thing.  Compilers do lots of
optimisations, even when you have "-O0" (or whatever).  They also do
lots of analysis.  And some of that process may scale badly with the
size of function, and the size of expressions, and may or may not be
worse for repeated identical expressions.  You might even find that for
some tests, optimised compilation (i.e., where you have asked the
compiler for greater effort) are faster because stronger analysis allows
more code elimination.

So gcc may well analyse for common sub-expressions even when not
optimising - it certainly does /some/ analysis.

The one thing I know (since I have seen bug reports about it) that
affects gcc speed on large functions is register allocation.  gcc will
attempt to optimise the use of registers throughout the function.  This
certainly is an issue with large functions when optimising - I don't
know if it applies for -O0 code.  And it is certainly something that has
been improved in later gcc versions (as always, you have omitted the gcc
version in your posts).

> 
> Note that clang performed much better than gcc in the test, for
> unoptimised compiles. There might be reasons for that that are easy to
> fix. But the real problem might be that there is no one at gcc looking
> into such things.
> 
>>>    Pelles C        T/O      ---
>>>    lccwin         13.0      T/O  seconds to compile
>>>    dmc             0.4      OOM
>>>    gcc            25.0    307.0
>>>    gcc opt        39.0      ---
>>>    tiny c          0.2      0.7
>>>    msvc            3.0     34.0
>>>    clang           6.3     30.0
>>>    (bcc            0.3     1.6)
>>>
>>> They still seem to fare very differently.
>>
>> Again, so what?  If these tests help you improve your own tools in
>> ways that you want, then great.  But it means nothing to anyone else.
> 
> Really? Pelles C took 4000 times as long as DMC. That suggests an issue
> that ought to be fixed. One that /could/ come up in real programs one day.

No, it suggests that Pelles C is not a good choice for compiling huge
functions full of duplicate statements.  It does /not/ suggest an issue
that ought to be fixed unless compiling such functions is a reasonable
use-case for that compiler.  And do you think your test functions are
typical use-cases of typical Pelles C users?

Don't get me wrong here - I think it is a good thing for compilers to be
able to handle large source efficiently, of whatever kind.  But I don't
think it is a sensible priority.

To put it in very simplistic terms, it is probably easy (relatively
speaking) to make a compiler that has O(n) performance in the number of
lines/statements/expressions in a function.  That could be done with a
"direct translation" compilation, and give fast compiles for your test
code.  On the other hand, analysing the source to give better quality
output (object code and/or warnings) might be O(n²).  That would be fast
enough for normal sized functions, but be miserable on your artificial
test functions.  But that algorithm is not an "issue that ought to be
fixed" - it is a /good/ choice, giving better results for real users and
real code.  Optimising for an artificial benchmark would be to the
detriment of the real use of the tool.

Of course, it would be possible to have a hybrid compiler - switching
from the O(n²) algorithm to the O(n) algorithm at a certain size of
function.  That would give you the best of both worlds - except that it
would complicate and enlarge the compiler, meaning more development
effort and a greater risk of errors.


Compilers are tools for a handling certain kinds of tasks.  Sometimes
that is just "turn this source code into object code".  Sometimes it is
"give me the fastest possible code from this source code".  Often it is
"help me write correct code" (that is a key reason I prefer gcc).  It is
/never/ "compile this million line artificial function as fast as
possible".  So why should any compiler writer put effort into achieving
this last goal?  They might happen to achieve it as a side effect of
other work, but it should not be a goal in itself.

> 
> I've already mentioned about gcc's 307 seconds in another thread.
> 
> I mean, just compare gcc and tcc: the latter is usually acknowledged to
> compile quicker and generate slower code, especially when gcc uses -O3 ,
> but 400 times slower to compile and still end up with code as slow as tcc?
> 
> (I used to have a problem with a test like this, when there were very
> large numbers of similar identifiers. Things got very slow. It turned
> out to be a poor hashing function. But I wouldn't have discovered it
> otherwise.)
> 
>

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Compiler tests Bart <bc@freeuk.com> - 2019-01-20 00:03 +0000
  Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-20 16:52 +1300
    Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-20 10:27 +0100
      Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 12:37 +0100
        Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 13:08 +0000
          Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 15:02 +0000
            Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:31 +0100
              Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:08 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 21:40 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 22:41 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 00:39 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 00:36 +0000
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-21 16:33 +1300
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 11:58 +0000
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 04:15 -0800
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-22 07:53 +1300
                Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 01:15 -0600
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 12:00 +0000
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-24 08:13 +1300
                Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 13:30 -0600
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 08:22 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:23 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 14:42 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 14:17 +0000
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:46 -0800
                You guys really, really need to get a room (Was: Compiler tests) gazelle@shell.xmission.com (Kenny McCormack) - 2019-01-24 14:05 +0000
          Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:27 +0100
            Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:19 +0000
              Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 09:44 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:49 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:31 +0100
                Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 14:38 +0100
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 15:15 +0100
                Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 16:05 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 16:39 +0100
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:51 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 00:16 +0000
                Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 08:43 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:31 +0000
                Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 17:49 +0100
                Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-23 11:04 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 11:18 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:35 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 15:49 +0000
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 17:42 +0100
                Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 03:18 -0600
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 11:11 +0100
                Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 12:18 -0600
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 13:56 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 10:55 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:07 +0000
                Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 04:29 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:52 +0000
                Re: Compiler tests mark.bluemel@gmail.com - 2019-01-22 05:04 -0800
                Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:18 -0800
                Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:45 -0800
                Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:35 -0800
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 15:10 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 17:30 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 23:02 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 23:14 +0000
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 15:45 -0800
                Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:57 -0600
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 16:36 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 01:37 +0000
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 18:53 -0800
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:42 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:22 +0100
                Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:52 -0500
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 15:13 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 14:33 +0000
                Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 07:30 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:05 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:17 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:29 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:04 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 21:50 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 10:45 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 11:55 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 13:58 +0100
                Re: Compiler tests mark.bluemel@gmail.com - 2019-01-24 05:07 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:10 +0000
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:18 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:51 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:44 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 17:32 +0000
                Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-24 19:07 +0000
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 20:14 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 22:24 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 22:46 +0000
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 12:34 +1300
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 01:01 +0000
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 14:15 +1300
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:05 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:51 +0000
                Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 09:48 +0000
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:56 +0100
                Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 12:08 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:28 +0100
                Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 13:33 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:38 +0100
                Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 15:08 +0000
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 13:53 +0000
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:25 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:03 +0100
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:08 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:14 +0100
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 12:02 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:37 +0100
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 13:58 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:26 +0100
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 15:01 +0100
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:42 +0100
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:25 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:37 +0000
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 16:33 +0100
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:44 -0800
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 17:53 +0100
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:33 -0800
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:43 +0100
                Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 12:17 -0800
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 18:31 +0000
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:41 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 21:19 +0000
                Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-26 10:24 +1300
                Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 22:27 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:39 +0000
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:46 +0000
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:43 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:37 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:19 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:47 +0100
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:17 -0800
                Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 07:28 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 12:35 +0100
                Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 21:47 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 22:57 +0100
                Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-24 06:47 -0800
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 20:22 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:12 +0100
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 21:54 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 11:41 +0100
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-24 23:17 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:11 +0100
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 12:04 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:05 +0100
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:40 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 22:36 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:15 +0000
                Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:48 +0000
                Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:46 +0000
                Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 17:48 +0000
                Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:23 -0800
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 16:41 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:18 +0000
                Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:28 +0100
                Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 20:23 +0000
                Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:52 -0600
          Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 20:17 +0000
    Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-21 00:13 -0600

csiph-web