Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #140095
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: Compiler tests |
| Date | 2019-01-22 23:02 +0100 |
| Organization | A noiseless patient Spider |
| Message-ID | <q283te$1fj$1@dont-email.me> (permalink) |
| References | (10 earlier) <pSl1E.188124$FF1.29988@fx06.am4> <q26paq$plc$1@dont-email.me> <QHD1E.216796$1D1.179078@fx29.am4> <q27894$a00$1@dont-email.me> <2rI1E.118890$dp2.55232@fx18.am4> |
On 22/01/2019 18:30, Bart wrote:
> On 22/01/2019 14:10, David Brown wrote:
>> On 22/01/2019 13:07, Bart wrote:
>>> On 22/01/2019 09:55, David Brown wrote:
>
>>> That has to be balanced against other approaches such as reading 110,000
>>> lines from a text file.
>>
>> That is going to be a great deal less than 0.1 seconds, at least with an
>> SSD or an OS that can cache files properly.
>
>> And it has to be balanced against doing it properly in C with an
>> initialised table.
>
> Sometimes the data needs to be embedded in the source code. And
> sometimes the linear block data from a file needs to be arranged into
> the data structures needed.
I regularly embed files as data in C code. Given your aversion to build
systems, you'll be horrified to know that for a number of my projects,
"make" uses Python scripts to turn things like images or html files into
C files with const arrays of data. These are defined and initialised as
arrays like
const uint8_t index_html = {
"\x48\x54\x54\x50\x2f\x31 ...
It is written as an initialised array, declared const. Anything else -
such as using assignments in a function - would be insane in a small
embedded system.
>
>> Just to please you, I made a file with a 1,000,000 entry array of ints,
>> arranged randomly but initialised properly rather than with a series of
>> assignments.
>>
>> int table[1000000] = {
>> 128689,
>> 586686,
>> 436487,
>> ...
>>
>> Compilation (optimised) is 2 seconds with gcc, or 5 if I used designated
>> initialisers in a random order:
>>
>> int table[1000000] = {
>> [323574] = 128689,
>> [463931] = 586686,
>> [114617] = 436487,
>> ...
>>
>> You'll be pleased to know I gave up waiting for gcc to attempt
>> assignment:
>>
>>
>> int table[1000000];
>>
>> int main(void) {
>> table[323574] = 128689;
>> table[463931] = 586686;
>> table[114617] = 436487;
>> ...
>>
>>
>> Assuming your "push" method is linear (which is unreasonable, as cache
>> effects would come into play), a million entry table will take about 1
>> second of startup time. How much startup time does my array
>> initialisation take? Less than a millisecond - too short to be
>> measurable by a simple "time" command.
>
> Further tests with that interpreted code show that running "hello" takes
> 40ms (including process overheads); a program initialising a table of
> 109K strings takes 70ms; and one for 500K strings takes 180ms. So about
> 140ms to initialise.
>
> String tables are a little bit harder because it's not just a linear
> block of data.
>
> If I do the equivalent test with C, then a program like this:
>
> #include <stdio.h>
>
> char *table[524288];
>
> int main(void) {
> int t;
>
> puts("Building table...");
> t=clock();
>
> table[0]="ångström";
> table[1]="ångströms";
> table[2]="übermensch";
> ....
> table[501207]="zyzzyvas";
> table[501208]="zz";
> table[501209]="zzt";
> table[501210]="zzz";
>
> t=clock()-t;
> printf("Done; time:%d %s\n",t,table[123456]);
> }
>
> took 1.6 seconds to build with Tiny C. Running it took about 100ms
> overall. Output is:
>
> Building table...
> Done; time:6 dornocks
>
> which suggests the actual initialisation only took 6ms. This is after
> all the ultimate unrolled loop. (It's possible that large data incurs
> additional overheads when loading the executable to start the process.)
>
> (Don't ask me what 'dornocks' means; this list of words includes
> everything.)
>
> However, If I try it with gcc, it runs out of memory.
What gcc version? I had no hints of memory issues with my testing with
gcc 7 and gcc 8. It just took time.
>
> (My bcc will compile it to .asm in 2.2 seconds, but I need to fix the
> problem with its assembler to complete it.
>
> Testing for extreme programs and sorting out the problems that come up
> is kinda fun. Just keep me away from PE file formats.)
As I've said before, "fun" is a perfectly good reason, and trumps pretty
much anything else. Initialising a table using a huge function of
assignments is crazy and massively inefficient regardless of the
compiler, and regardless of whether you are talking about compile time
or run time, and code generators that make their tables this way should
be fixed. But if testing this stuff is fun, go for it. I've found a
lot weirder things fun. (And while I remain convinced that these tests
have no practical purpose, I have to admit having learned a little here.
Useless things, but Quite Interesting™ none the less.)
>
>> Oh, and just for fun I timed python reading these C files (with their
>> million lines) - 20 milliseconds. Reading the file and separating them
>> into lines is 70 milliseconds.
>
> Doesn't that make you wonder what the hell gcc is doing for 15 minutes,
> for processing the same input?
>
I have some idea.
gcc does not translate statements one by one as they come in. It builds
up trees in a variety of formats (it has at least two internal formats
used at different stages in compilation). It does many passes over
these, even with -O0, for analysis, checking, sorting and re-arranging
them. Register assignment involves working back and forth through these
trees. With code like this, especially on a relatively register-poor
architecture like x86 (x86-64 is better than 32-bit mode, but poor
compared to many other architectures), this amounts to a great deal of
effort for very little purpose. For more normal functions, it can make
a huge difference to the efficiency of the code.
I am actually quite surprised that the times scaled linearly with the
size of the functions, rather than worse.
>> A large clump of assignments, instead of an array initialisation, is a
>> /worse/ way to write the code in all reasonable interpretations of
>> "worse".
>
> If there was a better alternative then I would agree. But if using C as
> a backend target, then it may simply be more convenient to do it that
> way. Maybe the data tables aren't static. Maybe some elements could be
> runtime expressions.
>
Laziness on the part of the code generator is not an excuse, IMHO,
unless it is only designed to handle small and limited cases.
> So long as its valid C (and there is a LOT worse C written using
> macros), then that should be it.
>
If you didn't have your irrational prejudice against macros, your code
generator would be much simpler, resulting in neater generated code that
would compile to more efficient object code on a wider range of compilers.
> Some people /want/ to use C as a kind of portable assembler where you
> throw anything at it, including large blocks of the same thing, and
> expect it to just work.
>
Everything has its limits.
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Compiler tests Bart <bc@freeuk.com> - 2019-01-20 00:03 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-20 16:52 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-20 10:27 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 12:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 13:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 15:02 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:31 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 21:40 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 22:41 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 00:39 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 00:36 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-21 16:33 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 11:58 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 04:15 -0800
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-22 07:53 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 01:15 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 12:00 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-24 08:13 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 13:30 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 08:22 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:23 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 14:42 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 14:17 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:46 -0800
You guys really, really need to get a room (Was: Compiler tests) gazelle@shell.xmission.com (Kenny McCormack) - 2019-01-24 14:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 09:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:49 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:31 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 14:38 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 15:15 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 16:05 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 16:39 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:51 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 00:16 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 08:43 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:31 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 17:49 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-23 11:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 11:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:35 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 15:49 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 17:42 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 03:18 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 11:11 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 12:18 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 13:56 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 10:55 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:07 +0000
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 04:29 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:52 +0000
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-22 05:04 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:18 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:45 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:35 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 15:10 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 17:30 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 23:02 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 23:14 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 15:45 -0800
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:57 -0600
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 16:36 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 01:37 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 18:53 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:42 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:22 +0100
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:52 -0500
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 15:13 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 14:33 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 07:30 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:17 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:29 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 21:50 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 10:45 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 11:55 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 13:58 +0100
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-24 05:07 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:10 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:51 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 17:32 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-24 19:07 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 20:14 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 22:24 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 22:46 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 12:34 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 01:01 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 14:15 +1300
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:05 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:51 +0000
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 09:48 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:56 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 12:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:28 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 13:33 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:38 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 15:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 13:53 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:25 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:03 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:08 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:14 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 12:02 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:37 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 13:58 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:26 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 15:01 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:42 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:25 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:37 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 16:33 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:44 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 17:53 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:33 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:43 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 12:17 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 18:31 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 21:19 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-26 10:24 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 22:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:39 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:46 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:43 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:47 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:17 -0800
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 07:28 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 12:35 +0100
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 21:47 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 22:57 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-24 06:47 -0800
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 20:22 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:12 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 21:54 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 11:41 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-24 23:17 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:11 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 12:04 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:05 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:40 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 22:36 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:15 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:48 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:46 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 17:48 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:23 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 16:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:28 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 20:23 +0000
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:52 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 20:17 +0000
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-21 00:13 -0600
csiph-web