Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #106087
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: C Compilers QoI |
| Date | 2017-03-14 21:41 +0100 |
| Organization | A noiseless patient Spider |
| Message-ID | <oa9kbm$sph$1@dont-email.me> (permalink) |
| References | <oa8pdp$nok$1@dont-email.me> <oa8st4$4a6$1@dont-email.me> <oa9221$mpf$1@dont-email.me> |
On 14/03/17 16:28, BartC wrote:
> On 14/03/2017 14:00, David Brown wrote:
>> On 14/03/17 14:01, BartC wrote:
>>> I was doing some very routine tests but came across surprising issues
>>> with a few of my C compilers that I didn't expect.
>>>
>>> This was the test code:
>>>
>>> int main(void) {
>>> int a,b,c,d;
>>>
>>> a = b+c*d;
>>> }
>>>
>>
>> That is undefined behaviour.
>
> That is not relevant to any of my OP.
>
> If I initialise b,c,d, then I get exactly the same results. In the case
> of gcc, running out of memory after 30 seconds.
That's better. It is still a completely daft exercise, but at least it
is valid code.
>
> If I make a volatile, then I almost the same result; the only difference
> is that now gcc crashes out after 40 seconds!
Now you are actually asking the compiler to do something - it has to
generate code that writes to "a" a million times (it does not have to do
the calculations, if the other variables are not also volatile).
>
>>> except that the 'a=b+c*d;' line was repeated a million times. (Also
>>> those lines are in an include file, but I don't think that matters.)
>>
>> That is undefined behaviour, repeated a pointless 999,999 times.
>
> You really don't get it, do you?
>
> I'm stress-testing a C compiler by giving it a contrived input that
> happens to repeat the same construct many times.
Stress-testing a compiler is fair enough - but make it a sensible or
realistic test. This is not realistic by any stretch of the imagination.
It is not even close to doing a test of a million line compile, because
you are using a million line /function/ - something that is many orders
of magnitude away from reality. And while your obsessions about "lines
of code per second" imply that you seem to view all lines as being
equal, that is not the case in compilers.
>
> That your favourite compiler gcc performs poorly (in terms of memory
> usage and processing speed), you try and excuse the result because the
> test input was 'meaningless'.
The test /is/ meaningless - regardless of the compiler. Neither I nor
anyone else cares a whit how much memory gcc, or any other compiler,
takes to run this test.
It makes a certain amount of sense to do speed comparisons of compilers
on different /real/ code inputs. gcc usually does quite badly on such
tests - that's okay for me, because I don't consider compile speed to be
a particular important feature of a compiler (within reason, of course).
If compile speed is all you care about, then you will want to do such
tests - but do them on sensible code, not such totally pointless
artificial tests.
>
> When you do unit tests, do the tests always mirror real life code?
Yes.
> When
> testing a memory allocator, do the allocations always to put to real use
> in some actual real-life code?
You make the tests reflect the expected usage.
You are asking the compiler to compile a million lines in /one
function/. For a compiler that does analysis and optimisation within a
function (yes, even with no "-O" flag), that is a completely different
task from compiling a thousand functions at a thousand lines each.
>
> Sometimes the content of the test data to a program does not matter;
> just the quantity of it does.
Lots of noise in means lots of noise out.
>
>>> MSVC 2008: in about 8 seconds; peak memory 0.6GB
>
>> Maybe MS have fine-tuned their compilers to be better able to deal with
>> long reams of pointless crap...
>
> Well, it does seem that MS' product manages to do the job with a
> fraction of the memory that gcc (or some of the other compilers) would
> have needed.
They will have different internal structures. I can't compare them,
because I don't know the details - it is quite possible that MS has
algorithms here that scale better for function size.
Compilers are usually designed to work well with the kind of code they
expect to meet - compilers do not expect to work with huge functions,
because people don't normally write such functions in normal code. (An
exception, perhaps, is the traditional Windows event loop function which
was often written as a huge switch statement - maybe MS tuned their
tools to work well with such functions.)
>
> (By simplifying the task to 250Kloc so that gcc completes, and
> extrapolating, it seems that bcc would have used about 3.7GB and taken
> about 2 minutes to compile the example in my OP.)
>
>>> And Tiny C still did it effortlessly in one second!
>>>
>>
>> So a very limited and simplistic compiler that does practically no
>> analysis or optimisation on the code can handle lines with greater speed
>> and less memory than a compiler that does a good deal of analysis and
>> optimisation (even with no optimisation flags).
>
> *Considerably* less memory.
>
> And for all its optimisation and analysis, gcc isn't spectacularly
> faster than my own crappy compiler. [When both are used to compile my
> test C compiler app to 64-bit]
As you have been told before, gcc does a great deal more work than such
compilers - even if /you/ don't find other features relevant. So it is
not reasonable to expect any similarity in resource usage.
I don't really understand why you bother with compilers like gcc or
MSVC. You are interested primarily in compilation speed and small
installations, and view features such as static error checking,
warnings, optimisations, standards compliance, extensions, etc., as
disadvantages in a compiler. If you don't like gcc, don't use it -
stick to TCC or whatever else works for you. It makes no more sense for
you to rant about how bad gcc is, than it would for me to rant about how
TCC can't generate code for the ARM, or that lccwin does not support the
"-Wswitch-enum" flag (which found a bug in my code for me today).
>
>>> Anyway, people will say, use a more up-to-date compiler built as a
>>> 64-bit application.
>>
>> No, people will say "stopping doing such stupid tests" !
>
> You don't think that such tests give any insight whatsoever into the
> memory needs and processing efficiency of different compilers?
No. They give absolutely no useful insight whatsoever.
If you were compiling a vaguely /realistic/ large file - full of macros,
struct definitions, inline code, variables, and lots of functions, then
it could be a little interesting. For actual applications in real life,
it is often the link stage - especially with link-time optimisation -
where memory and processing resources are relevant. To a lesser extent,
it is also important when doing massively parallel builds ("make -j 32",
etc.) on big SMP systems. That's why gcc and clang both view LTO builds
of Firefox, LibreOffice, and similar large projects as benchmarks. In
the release notes for gcc 4.9, for example, it notes "memory usage
building Firefox with debug enabled was reduced from 15 GB to 3.5 GB,
link time from 1700 seconds to 350 seconds". /That/ sort of big build
gives insight.
>
> After all the overheads of the data structures required to represent
> several million terms, operations and statements might be the same
> whether they are arranged to represent a real program or a contrived one.
>
>>> If they used memory more
>>> efficiently, then they would have worked (MSVC only used 0.6GB; Tiny C
>>> apparently uses no memory at all!).
>>
>> Why would anyone bother making their compiler more efficient at at test
>> like this?
>
> <Palm to forehead> Because maybe, just maybe, if it can be made more
> efficient at such a test, then it might be efficient at real code too?
I severely doubt it - there is more to compiling than chewing through
lines and mindlessly converting to assembly. The kind of "compiling"
you are trying to do with this test could probably be handled by an awk
script.
>
> Why do they bother testing cars around an unobstructed circular track,
> when real-life usage is so utterly different?
>
> If there is a problem, then it is much easier to isolate the bits of
> code responsible when a very limited set of features are being tested,
> and easier to to see the results of any tweaks. Compared with testing
> large of amounts of real code which make it harder to spot the bottlenecks.
>
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 13:01 +0000
Re: C Compilers QoI mark.bluemel@gmail.com - 2017-03-14 06:16 -0700
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 06:42 -0700
Re: C Compilers QoI "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-03-14 06:56 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 15:58 +0000
Re: C Compilers QoI "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2017-03-14 09:22 -0700
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-14 16:54 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 10:24 -0700
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-14 17:58 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 14:12 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-14 16:34 -0500
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-14 17:36 -0500
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 23:06 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 16:20 -0700
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-15 02:32 +0000
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-15 12:49 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-15 11:56 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-15 23:09 +0000
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-16 18:17 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 11:47 -0700
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 07:50 +1300
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 12:59 -0700
Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-16 21:24 +0100
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 14:23 -0700
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:30 +1300
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:38 +1300
Re: C Compilers QoI Gareth Owen <gwowen@gmail.com> - 2017-03-17 05:07 +0000
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 20:26 +1300
Re: C Compilers QoI Gareth Owen <gwowen@gmail.com> - 2017-03-17 07:59 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 16:14 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-16 22:54 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-17 09:01 -0700
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-26 14:16 +1300
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-26 14:19 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-26 22:43 +0100
Re: C Compilers QoI GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-03-27 01:28 +0200
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-26 22:07 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 11:28 +0100
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 07:38 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 16:23 +0100
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 08:55 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-27 17:01 +0100
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 10:03 -0700
Re: C Compilers QoI GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-03-27 17:58 +0200
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-27 03:54 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 08:46 -0700
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-27 17:47 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 11:48 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 15:37 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 14:46 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 20:28 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-28 01:18 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-28 18:24 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-29 07:40 -0700
Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-29 17:05 +0200
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-29 11:07 -0700
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-31 04:04 +0000
Re: C Compilers QoI bartc <bc@freeuk.com> - 2017-03-31 10:53 +0100
Re: C Compilers QoI Malcolm McLean <malcolm.mclean5@btinternet.com> - 2017-03-31 07:35 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 08:07 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 07:54 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-31 07:43 -0700
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:21 +1300
Re: C Compilers QoI Ian Collins <ian-news@hotmail.com> - 2017-03-17 11:24 +1300
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 22:15 +0000
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-17 12:31 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-16 15:24 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 16:03 -0700
Re: C Compilers QoI scott@slp53.sl.home (Scott Lurndal) - 2017-03-17 12:39 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-17 08:27 -0700
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-16 01:35 +0000
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 02:18 +0000
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-18 00:45 +0000
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-18 13:12 +0000
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-18 20:31 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-18 17:31 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-16 09:22 -0700
Re: C Compilers QoI Nick Bowler <nbowler@draconx.ca> - 2017-03-24 16:13 +0000
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-24 12:32 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-24 22:13 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-25 13:05 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 15:44 -0500
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-27 14:51 -0700
Re: C Compilers QoI Robert Wessel <robertwessel2@yahoo.com> - 2017-03-27 20:31 -0500
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:26 +0000
Re: C Compilers QoI Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-03-15 12:02 -0500
Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 15:00 +0100
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 07:36 -0700
Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 15:57 +0100
Re: C Compilers QoI Andrey Tarasevich <andreytarasevich@hotmail.com> - 2017-03-14 08:11 -0700
Re: C Compilers QoI BGB <cr88192@hotmail.com> - 2017-03-29 23:51 -0500
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 15:28 +0000
Re: C Compilers QoI David Brown <david.brown@hesbynett.no> - 2017-03-14 21:41 +0100
Re: C Compilers QoI Keith Thompson <kst-u@mib.org> - 2017-03-14 09:26 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 18:23 +0000
Re: C Compilers QoI Keith Thompson <kst-u@mib.org> - 2017-03-14 12:05 -0700
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 09:35 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:08 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 10:24 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 17:34 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 11:55 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:03 +0000
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:06 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 12:10 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 21:14 +0000
Re: C Compilers QoI already5chosen@yahoo.com - 2017-03-14 14:42 -0700
Re: C Compilers QoI supercat@casperkitty.com - 2017-03-14 14:53 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 23:20 +0000
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-15 02:43 +0000
Re: C Compilers QoI jacobnavia <jacob@jacob.remcomp.fr> - 2017-03-14 19:39 +0100
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-14 19:10 +0000
Re: C Compilers QoI antispam@math.uni.wroc.pl - 2017-03-14 19:29 +0000
Re: C Compilers QoI Noob <root@127.0.0.1> - 2017-03-15 09:27 +0100
Re: C Compilers QoI ed_davis2@yahoo.com - 2017-03-16 15:10 -0700
Re: C Compilers QoI BartC <bc@freeuk.com> - 2017-03-16 23:19 +0000
csiph-web