Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #140005 > unrolled thread
| Started by | Bart <bc@freeuk.com> |
|---|---|
| First post | 2019-01-20 00:03 +0000 |
| Last post | 2019-01-21 00:13 -0600 |
| Articles | 20 on this page of 160 — 17 participants |
Back to article view | Back to comp.lang.c
Compiler tests Bart <bc@freeuk.com> - 2019-01-20 00:03 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-20 16:52 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-20 10:27 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 12:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 13:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 15:02 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:31 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 21:40 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 22:41 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 00:39 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 00:36 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-21 16:33 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 11:58 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 04:15 -0800
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-22 07:53 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 01:15 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 12:00 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-24 08:13 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 13:30 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 08:22 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:23 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 14:42 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 14:17 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:46 -0800
You guys really, really need to get a room (Was: Compiler tests) gazelle@shell.xmission.com (Kenny McCormack) - 2019-01-24 14:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 09:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:49 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:31 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 14:38 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 15:15 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 16:05 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 16:39 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:51 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 00:16 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 08:43 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:31 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 17:49 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-23 11:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 11:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:35 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 15:49 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 17:42 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 03:18 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 11:11 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 12:18 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 13:56 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 10:55 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:07 +0000
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 04:29 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:52 +0000
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-22 05:04 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:18 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:45 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:35 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 15:10 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 17:30 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 23:02 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 23:14 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 15:45 -0800
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:57 -0600
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 16:36 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 01:37 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 18:53 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:42 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:22 +0100
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:52 -0500
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 15:13 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 14:33 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 07:30 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:17 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:29 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 21:50 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 10:45 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 11:55 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 13:58 +0100
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-24 05:07 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:10 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:51 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 17:32 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-24 19:07 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 20:14 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 22:24 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 22:46 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 12:34 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 01:01 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 14:15 +1300
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:05 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:51 +0000
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 09:48 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:56 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 12:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:28 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 13:33 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:38 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 15:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 13:53 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:25 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:03 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:08 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:14 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 12:02 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:37 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 13:58 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:26 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 15:01 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:42 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:25 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:37 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 16:33 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:44 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 17:53 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:33 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:43 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 12:17 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 18:31 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 21:19 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-26 10:24 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 22:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:39 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:46 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:43 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:47 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:17 -0800
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 07:28 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 12:35 +0100
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 21:47 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 22:57 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-24 06:47 -0800
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 20:22 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:12 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 21:54 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 11:41 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-24 23:17 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:11 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 12:04 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:05 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:40 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 22:36 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:15 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:48 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:46 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 17:48 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:23 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 16:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:28 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 20:23 +0000
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:52 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 20:17 +0000
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-21 00:13 -0600
Page 1 of 8 [1] 2 3 4 5 6 7 8 Next page →
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-20 00:03 +0000 |
| Subject | Compiler tests |
| Message-ID | <%UO0E.109606$Ur2.51894@fx14.am4> |
I have an interest in compilers so when I download a new compiler or a
new language, I often give it a little stress test, which is just lots
of repeated lines, just to see how it copes.
I've assembled a bunch of such tests here, mostly for C compilers but
also for a few other languages that happened to be on my machine:
https://github.com/sal55/mx/blob/master/compilertest.txt
I've tested with blocks of 20K, 100K, 500K and 2M lines of code, all on
the same PC.
As can be seen, many deal with that effortlessly, while others struggle.
I found the results interesting; maybe others will too. (And my own
products aren't the fastest either..)
--
bart
[toc] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2019-01-20 16:52 +1300 |
| Message-ID | <gai9j2Fuek4U6@mid.individual.net> |
| In reply to | #140005 |
On 20/01/2019 13:03, Bart wrote: > I have an interest in compilers so when I download a new compiler or a > new language, I often give it a little stress test, which is just lots > of repeated lines, just to see how it copes. > > I've assembled a bunch of such tests here, mostly for C compilers but > also for a few other languages that happened to be on my machine: > > https://github.com/sal55/mx/blob/master/compilertest.txt > > I've tested with blocks of 20K, 100K, 500K and 2M lines of code, all on > the same PC. You would be better off compiling real large files (such as the SQLite blob) which will test an optimiser. > As can be seen, many deal with that effortlessly, while others struggle. It really only tests how fast the compiler front end parses stuff into memory, not how the compiler handles real code. -- Ian
[toc] | [prev] | [next] | [standalone]
| From | "Pascal J. Bourguignon" <pjb@informatimago.com> |
|---|---|
| Date | 2019-01-20 10:27 +0100 |
| Message-ID | <m24la3llvv.fsf@despina.home> |
| In reply to | #140008 |
Ian Collins <ian-news@hotmail.com> writes: > On 20/01/2019 13:03, Bart wrote: >> I have an interest in compilers so when I download a new compiler or a >> new language, I often give it a little stress test, which is just lots >> of repeated lines, just to see how it copes. >> >> I've assembled a bunch of such tests here, mostly for C compilers but >> also for a few other languages that happened to be on my machine: >> >> https://github.com/sal55/mx/blob/master/compilertest.txt >> >> I've tested with blocks of 20K, 100K, 500K and 2M lines of code, all on >> the same PC. > > You would be better off compiling real large files (such as the SQLite > blob) which will test an optimiser. > >> As can be seen, many deal with that effortlessly, while others struggle. > > It really only tests how fast the compiler front end parses stuff into > memory, not how the compiler handles real code. Well, obviously some compilers try to optimize duplicate expressions, and get lost in the number of them. Not a realistic test, but definitely a good try-to-break-the-compiler test. -- __Pascal J. Bourguignon__ http://www.informatimago.com
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2019-01-20 12:37 +0100 |
| Message-ID | <q21mhu$jm6$1@dont-email.me> |
| In reply to | #140009 |
On 20/01/2019 10:27, Pascal J. Bourguignon wrote: > Ian Collins <ian-news@hotmail.com> writes: > >> On 20/01/2019 13:03, Bart wrote: >>> I have an interest in compilers so when I download a new compiler or a >>> new language, I often give it a little stress test, which is just lots >>> of repeated lines, just to see how it copes. >>> >>> I've assembled a bunch of such tests here, mostly for C compilers but >>> also for a few other languages that happened to be on my machine: >>> >>> https://github.com/sal55/mx/blob/master/compilertest.txt >>> >>> I've tested with blocks of 20K, 100K, 500K and 2M lines of code, all on >>> the same PC. >> >> You would be better off compiling real large files (such as the SQLite >> blob) which will test an optimiser. >> >>> As can be seen, many deal with that effortlessly, while others struggle. >> >> It really only tests how fast the compiler front end parses stuff into >> memory, not how the compiler handles real code. > > Well, obviously some compilers try to optimize duplicate expressions, > and get lost in the number of them. Not a realistic test, but > definitely a good try-to-break-the-compiler test. > Unfortunately for Bart, compilers are usually targeted to real code. And in real code, duplicate expressions and structures /do/ turn up (especially in template-based C++ coding), and optimising them is very important to the efficiency of the code generation. But meaningless mass copying of the same code does not turn up in real code, and compiler writers don't particularly care about the results. This means that Bart's simplistic compiler can do a better job than real compilers at this kind of meaningless artificial benchmark, and he likes to trump it as "proof" that his tool is great and gcc is terrible. It's a common theme for him, and it is no more useful now than it was the past dozen times he presented this. There /are/ issues with gcc compiling very large functions (in particular, it takes a huge amount of memory), there are real use-cases (such as for simulators), and there is work being done to improve things. I don't know the current state.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-20 13:08 +0000 |
| Message-ID | <lp_0E.182687$1D1.4718@fx29.am4> |
| In reply to | #140011 |
On 20/01/2019 11:37, David Brown wrote:
> On 20/01/2019 10:27, Pascal J. Bourguignon wrote:
>> Well, obviously some compilers try to optimize duplicate expressions,
>> and get lost in the number of them. Not a realistic test, but
>> definitely a good try-to-break-the-compiler test.
>>
>
> Unfortunately for Bart, compilers are usually targeted to real code. And
> in real code, duplicate expressions and structures /do/ turn up
> (especially in template-based C++ coding), and optimising them is very
> important to the efficiency of the code generation.
Except that the problem timings were with /unoptimised/ compiles. With
optimisation enabled, they were much faster.
But meaningless
> mass copying of the same code does not turn up in real code, and
> compiler writers don't particularly care about the results.
>
> This means that Bart's simplistic compiler can do a better job than real
> compilers at this kind of meaningless artificial benchmark, and he likes
> to trump it as "proof" that his tool is great and gcc is terrible. It's
> a common theme for him, and it is no more useful now than it was the
> past dozen times he presented this.
>
>
> There /are/ issues with gcc compiling very large functions (in
> particular, it takes a huge amount of memory), there are real use-cases
> (such as for simulators), and there is work being done to improve
> things. I don't know the current state.
Here's another test where each line is in a separate function:
void f1(void) {int a,b=2,c=3,d=4; a=b+c*d;}
void f2(void) {int a,b=2,c=3,d=4; a=b+c*d;}
void f3(void) {int a,b=2,c=3,d=4; a=b+c*d;}
void f4(void) {int a,b=2,c=3,d=4; a=b+c*d;}
....
And here are some quick results:
20K 100K
Pelles C T/O ---
lccwin 13.0 T/O seconds to compile
dmc 0.4 OOM
gcc 25.0 307.0
gcc opt 39.0 ---
tiny c 0.2 0.7
msvc 3.0 34.0
clang 6.3 30.0
(bcc 0.3 1.6)
They still seem to fare very differently.
This test was useful in that it highlighted a problem with my bcc, which
suffers from using a fixed-size hashtable for the main symbol table. The
1.6 figure is after tweaking the size. I will to change it to adapt as
needed.
However, bcc generates .asm, and my assembler for that has its own hard
limit of 64K entries that can't be so easily changed, but it will have
to in order to create an exe from this.
Why does gcc's speed get slower non-linearly compared with clang? Maybe
it could also have issues that need looking at. It seemed to spend an
awfully long time running 'as'.
(I would like to see one of your test-suites where every single test
program is useful real-life code.)
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-20 15:02 +0000 |
| Message-ID | <S301E.67860$OD1.40746@fx27.am4> |
| In reply to | #140013 |
On 20/01/2019 13:08, Bart wrote: > Here's another test where each line is in a separate function: > And here are some quick results: > > 20K 100K > Pelles C T/O --- > lccwin 13.0 T/O seconds to compile > dmc 0.4 OOM > gcc 25.0 307.0 If I let lccwin run to completion (since I did that for gcc) then its 100K timing is 330 seconds. The timing for Pelles C. for 20K, is 400 seconds (1000 times longer than DMC). For 10K it was only 10 seconds so something funny going on there. I know DB is never going to be convinced such tests are anything other than a waste of time (unless they show gcc to be consistently superior) but as the developer of any of these I would definitely be interested in any odd results. I am now about to change my C compiler to eliminate the symbol table limit. (Although there will still be a few other hard limits left.)
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2019-01-20 17:31 +0100 |
| Message-ID | <q227p7$shm$1@dont-email.me> |
| In reply to | #140016 |
On 20/01/2019 16:02, Bart wrote: > On 20/01/2019 13:08, Bart wrote: > >> Here's another test where each line is in a separate function: > >> And here are some quick results: >> >> 20K 100K >> Pelles C T/O --- >> lccwin 13.0 T/O seconds to compile >> dmc 0.4 OOM >> gcc 25.0 307.0 > > If I let lccwin run to completion (since I did that for gcc) then its > 100K timing is 330 seconds. > > The timing for Pelles C. for 20K, is 400 seconds (1000 times longer than > DMC). For 10K it was only 10 seconds so something funny going on there. > > I know DB is never going to be convinced such tests are anything other > than a waste of time (unless they show gcc to be consistently superior) > but as the developer of any of these I would definitely be interested in > any odd results. They'd be a waste of time even if you felt gcc "won" - because they don't test anything remotely related to compiler usage. (I am confident, however, that if gcc /did/ turn out to rate highly in one of these tests, then you would tweak the test or use different gcc versions until you were again able to "prove" the "superiority" of your own compiler.) > > I am now about to change my C compiler to eliminate the symbol table > limit. (Although there will still be a few other hard limits left.) > Won't that be fantastic! Instead of being limited to 64K of meaningless duplicated functions, you'll be able to compile millions of meaningless functions. I am sure the C programming world is holding their breath waiting for this revolution.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-20 17:08 +0000 |
| Message-ID | <9W11E.125428$gy2.50589@fx21.am4> |
| In reply to | #140020 |
On 20/01/2019 16:31, David Brown wrote: > On 20/01/2019 16:02, Bart wrote: >> On 20/01/2019 13:08, Bart wrote: >> >>> Here's another test where each line is in a separate function: >> >>> And here are some quick results: >>> >>> 20K 100K >>> Pelles C T/O --- >>> lccwin 13.0 T/O seconds to compile >>> dmc 0.4 OOM >>> gcc 25.0 307.0 >> >> If I let lccwin run to completion (since I did that for gcc) then its >> 100K timing is 330 seconds. >> >> The timing for Pelles C. for 20K, is 400 seconds (1000 times longer >> than DMC). For 10K it was only 10 seconds so something funny going on >> there. >> >> I know DB is never going to be convinced such tests are anything other >> than a waste of time (unless they show gcc to be consistently >> superior) but as the developer of any of these I would definitely be >> interested in any odd results. > > They'd be a waste of time even if you felt gcc "won" - because they > don't test anything remotely related to compiler usage. It's not meant to. It's meant to test one aspect. Otherwise, faced with the task of speeding up a program, or reducing efficiently, or improving any part of it, you wouldn't know where to start. In the case of gcc compiling 100,000 one-line functions, it appears that over 70% of the time is spent in its assembler. (Compared with 30% with 20,000 functions, or 5% with 100,000 assignments in one function.) So that's bottleneck in a part of it that can happen with either large files, or files that have a very large number of unique identifiers. More tests would be needed to establish which, or if there is another reason. That or may not have any significant impact on real, manually written code, but it might be something worth investigating. Especially since clang works much better /on the same input/. > (I am > confident, however, that if gcc /did/ turn out to rate highly in one of > these tests, then you would tweak the test or use different gcc versions > until you were again able to "prove" the "superiority" of your own > compiler.) No, I have keep the tests fair, otherwise there's no point. I also recognise when something must be wrong enough that it can't fairly be compared. >> I am now about to change my C compiler to eliminate the symbol table >> limit. (Although there will still be a few other hard limits left.) > Won't that be fantastic! Instead of being limited to 64K of meaningless > duplicated functions, you'll be able to compile millions of meaningless > functions. I am sure the C programming world is holding their breath > waiting for this revolution. You're right, people working on fast network communications don't want to test any meaningless streams of bytes. Graphics developers don't want to work with any meaningless pixels or vectors or faces. And vehicle developers don't want to test their cars round and round the same track. Utterly meaningless.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2019-01-20 21:40 +0100 |
| Message-ID | <q22mb1$u74$1@dont-email.me> |
| In reply to | #140024 |
On 20/01/2019 18:08, Bart wrote: > On 20/01/2019 16:31, David Brown wrote: > >> Won't that be fantastic! Instead of being limited to 64K of >> meaningless duplicated functions, you'll be able to compile millions >> of meaningless functions. I am sure the C programming world is >> holding their breath waiting for this revolution. > > You're right, people working on fast network communications don't want > to test any meaningless streams of bytes. Graphics developers don't want > to work with any meaningless pixels or vectors or faces. > > And vehicle developers don't want to test their cars round and round the > same track. Utterly meaningless. > I assume it was entirely intentional that you missed my point.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-20 22:41 +0000 |
| Message-ID | <GO61E.101549$dE1.32673@fx22.am4> |
| In reply to | #140030 |
On 20/01/2019 20:40, David Brown wrote: > On 20/01/2019 18:08, Bart wrote: >> On 20/01/2019 16:31, David Brown wrote: > >> >>> Won't that be fantastic! Instead of being limited to 64K of >>> meaningless duplicated functions, you'll be able to compile millions >>> of meaningless functions. I am sure the C programming world is >>> holding their breath waiting for this revolution. >> >> You're right, people working on fast network communications don't want >> to test any meaningless streams of bytes. Graphics developers don't >> want to work with any meaningless pixels or vectors or faces. >> >> And vehicle developers don't want to test their cars round and round >> the same track. Utterly meaningless. >> > > I assume it was entirely intentional that you missed my point. No, I don't know what your point was at all except to diss everything I ever do. Mine: if you look at the 30 or so timings I posted, it seems clear that 20% of those tools could do with some attention even for the results in the first column. All by running the simplest kind of stress test. And, if you ever have several million lines of flat (no loops or functions) C code that has been generated and that needs to be run, you know which C compilers to avoid.
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2019-01-21 00:39 +0100 |
| Message-ID | <q230r4$ues$1@dont-email.me> |
| In reply to | #140032 |
On 20/01/2019 23:41, Bart wrote: > On 20/01/2019 20:40, David Brown wrote: >> On 20/01/2019 18:08, Bart wrote: >>> On 20/01/2019 16:31, David Brown wrote: >> >>> >>>> Won't that be fantastic! Instead of being limited to 64K of >>>> meaningless duplicated functions, you'll be able to compile millions >>>> of meaningless functions. I am sure the C programming world is >>>> holding their breath waiting for this revolution. >>> >>> You're right, people working on fast network communications don't >>> want to test any meaningless streams of bytes. Graphics developers >>> don't want to work with any meaningless pixels or vectors or faces. >>> >>> And vehicle developers don't want to test their cars round and round >>> the same track. Utterly meaningless. >>> >> >> I assume it was entirely intentional that you missed my point. > > No, I don't know what your point was at all except to diss everything I > ever do. I don't diss everything you do - but you quickly forget when I tell you something you wrote was a good idea. However, much of what you think of as important in compilers is irrelevant to me, and to most people who write C code for a living and not for a hobby. Much of the way you do your development is the antithesis of quality code development. Much of your complaints about C and compilers is due entirely to your misunderstandings, your misrepresentations, your refusal to learn the language and the tools, and your hard-headed bias towards certain language features. What do you expect here? Most people here have long ago given you up as a hopeless case or a troll - they reply to your posts only to be sure that innocent third parties don't get the wrong idea from what you write. And to spoon-feed you the point, your absurd "test" files bear no relation to real source files that people want to compile. If you want useful results from tests, you have to test with realistic cases. So your stream of bytes for networking will be real test data, or perhaps random data, not a million zero bytes. Your graphics test scenes will be real pictures, not a blank window. So if you want to test speed of compilation, use real code. > > Mine: if you look at the 30 or so timings I posted, it seems clear that > 20% of those tools could do with some attention even for the results in > the first column. All by running the simplest kind of stress test. > > And, if you ever have several million lines of flat (no loops or > functions) C code that has been generated and that needs to be run, you > know which C compilers to avoid. That's as useful as telling me that if I ever find myself carrying millions of eggs in a wheelbarrow, I should know which bumpy roads to avoid. I write programs, using C. The prime responsibilities of the code are to work correctly, be clearly correct, be understandable, be maintainable. Critical to that is being modular and well organised. If I have a file that is more than a couple of thousand lines long, I have failed at that job. It is rare that any of my files exceed one thousand lines. None of these files takes more than a small fraction of a second to compile - with a top-class compiler, and lots of optimisations and warning flags. Tell me, /why/ would I care about the compile speed here? Why would I care about the speed of ridiculous "stress tests" involving millions of lines without loops or functions? Why would I care how fast a toy compiler can run, when it does not do a job that I need? Now, there /are/ languages and projects for which compile speed is important. Most obviously, for C++ code with a lot of template libraries, compile speed is a big issue. Tell me, how fast does your compiler work for big C++ projects with lots of template libraries? Once you've got that working, and can show comparisons to other compilers using real-world source code, /then/ you'll have something that will be of interest.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-21 00:36 +0000 |
| Message-ID | <5u81E.116645$dp2.90631@fx18.am4> |
| In reply to | #140034 |
On 20/01/2019 23:39, David Brown wrote:
> On 20/01/2019 23:41, Bart wrote:
> And to spoon-feed you the point, your absurd "test" files bear no
> relation to real source files that people want to compile. If you want
> useful results from tests, you have to test with realistic cases. So
> your stream of bytes for networking will be real test data, or perhaps
> random data, not a million zero bytes. Your graphics test scenes will
> be real pictures, not a blank window.
I can see you've done very little performance optimisation, or you
haven't done it very well.
> So if you want to test speed of
> compilation, use real code.
That will depend on a million factors which all add up. Where do you
start? As soon as you isolate one small part, you find that not enough
real data exists to give meaningful timings. So you have to duplicate
input in a way that doesn't happen in real data.
Suppose my program is a console text editor but the refresh (say on page
down) appears sluggish.
How would you go about making it a bit faster, where would you start?
I would do it by a test program that writes a screenful of data - any
data - then repeats that N times, say 100 times.
I can measure how long that takes, and that gives me a figure to compare
against. Then I can proceed to find any bottlenecks, try different
approaches, to see what makes a difference.
You, however, would say that is not real data and that no real program
would just flash 100 screenfuls of nonsense one after the other.
So, what would YOU do?
> Now, there /are/ languages and projects for which compile speed is
> important. Most obviously, for C++ code with a lot of template
> libraries, compile speed is a big issue. Tell me, how fast does your
> compiler work for big C++ projects with lots of template libraries? Once
> you've got that working, and can show comparisons to other compilers
> using real-world source code, /then/ you'll have something that will be
> of interest.
I don't use C++. I design languages specifically to be easy to use and
to build compilers for. And for my purposes that works admirably:
C:\cx2>timeit "mm cc" # rebuild my C compiler from scratch
Compiling cc.m to cc.exe
mm cc 0.21
C:\cx2>timeit "mm -c64 cc" # translate it to a single C file
Compiling cc.m to cc.c
mm -c64 cc 0.13
C:\cx2>timeit "gcc cc.c" # Now build it using gcc -O0
gcc cc.c 3.17 # ouch! And it's not even faster
C:\cx2>timeit "gcc -O3 cc.c" # Build using gcc -O3
gcc -O3 cc.c 13.93 # Double ouch! And only 30% faster
C:\cx2>timeit "bcc cc" # Enough! try my bcc
Compiling cc.c to cc.exe
bcc cc 0.18 # Much better...
# But tcc will do too:
C:\cx2>timeit "tcc cc.c \windows\system32\user32.dll"
tcc cc.c \windows\system32\user32.dll 0.14
> using real-world source code, /then/ you'll have something that will be
> of interest.
Languages such as D and Go have been touted as being faster than C or
C++ when building large programs. But how do they fare on something a
bit simpler but still demanding?
Well have a look at my table: Go lives up to the hype, although not
spectacularly so. But there appears to be something amiss with D.
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2019-01-21 16:33 +1300 |
| Message-ID | <gakss0Fuek4U10@mid.individual.net> |
| In reply to | #140041 |
On 21/01/2019 13:36, Bart wrote: > On 20/01/2019 23:39, David Brown wrote: >> On 20/01/2019 23:41, Bart wrote: > >> And to spoon-feed you the point, your absurd "test" files bear no >> relation to real source files that people want to compile. If you want >> useful results from tests, you have to test with realistic cases. So >> your stream of bytes for networking will be real test data, or perhaps >> random data, not a million zero bytes. Your graphics test scenes will >> be real pictures, not a blank window. > > I can see you've done very little performance optimisation, or you > haven't done it very well. A strange conclusion to draw. >> So if you want to test speed of >> compilation, use real code. > > That will depend on a million factors which all add up. Where do you > start? As soon as you isolate one small part, you find that not enough > real data exists to give meaningful timings. So you have to duplicate > input in a way that doesn't happen in real data. You could, as I suggested earlier, compile suitable large real source file such as the SQLite combined file (~200K lines). This is one of the biggest "real world" source files a compiler is likely to encounter. -- Ian.
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-21 11:58 +0000 |
| Message-ID | <vti1E.113058$Ur2.71958@fx14.am4> |
| In reply to | #140043 |
On 21/01/2019 03:33, Ian Collins wrote:
> On 21/01/2019 13:36, Bart wrote:
>> On 20/01/2019 23:39, David Brown wrote:
>>> On 20/01/2019 23:41, Bart wrote:
>>
>>> And to spoon-feed you the point, your absurd "test" files bear no
>>> relation to real source files that people want to compile. If you want
>>> useful results from tests, you have to test with realistic cases. So
>>> your stream of bytes for networking will be real test data, or perhaps
>>> random data, not a million zero bytes. Your graphics test scenes will
>>> be real pictures, not a blank window.
>>
>> I can see you've done very little performance optimisation, or you
>> haven't done it very well.
>
> A strange conclusion to draw.
>
>>> So if you want to test speed of
>>> compilation, use real code.
>>
>> That will depend on a million factors which all add up. Where do you
>> start? As soon as you isolate one small part, you find that not enough
>> real data exists to give meaningful timings. So you have to duplicate
>> input in a way that doesn't happen in real data.
>
> You could, as I suggested earlier, compile suitable large real source
> file such as the SQLite combined file (~200K lines). This is one of the
> biggest "real world" source files a compiler is likely to encounter.
You're starting from scratch writing a C compiler, with performance as
one of the aims, and the first thing you do is attempt to compile a
219,000 line program?
Which happens to include (on Windows) "windows.h" which, if you use a
version from mingw or msvc, involves hundreds of #includes and hundreds
of thousands more lines of declarations.
That is totally impractical; it's only going to work at all when the
compiler is nearly finished. (Imagine NASA testing a Saturn V for the
first time on an actual trip to the moon.)
You usually start off writing a bare tokeniser (no symbol lookups), then
one with lookups, then, for C, superimposing a macro processor.
For the first, the input can be anything at all, so long as it's big.
I've obtained throughput of over 10M lines per second, so even your
sqlite program [ignoring #-directives as they are just tokens at this
point] would take only 20msec, which is not enough to properly measure
just by itself; you may want to repeat it 10 or 50 times.
On the journey, however, even that is too complex. Suppose there's a
part that the lexer is weak on, be it processing strings or identifiers
or numeric constants, how can you tell from input as chaotic as sqlite3.c?
You might construct an input of millions of the same number just so you
can hone that part of the program. And so on and so forth.
It's an incremental process and you need simple, controllable inputs.
=========
BTW here's what the finished product looks like actually working on
sqlite3 (using my own small windows.h):
C:\cx\big>timeit "..\cc -c sqlite3" # cc.exe passed thru gcc-O3
Compiling sqlite3.c to sqlite3.asm
..\cc -c sqlite3 0.30
C:\cx\big>timeit "bcc -c sqlite3" # bcc.exe used my compiler
Compiling sqlite3.c to sqlite3.asm
bcc -c sqlite3 0.38
So it still takes only 0.3 seconds for a total of 222Kloc. Internal
timing (clock() difference between program entry and exit) is about
250ms. And that includes all passes. If you isolate some passes to
concentrate on those, then timings will be much less.
Since timing resolution is only 16ms (using clock(), 4-8ms using other
means but also more variable), it's harder to measure the effect of any
small change.
Hence it is necessary to partially synthesise the input in order to
magnify the effects of any changes.
sqlite3.c itself only has 11,000 symbols; yesterday I was working with 2
million.
(Last month I was working on generated programs designed to test new
floating point processing in my C compiler tokeniser. This was flat code
of 100,000 lines of C, just the sort that David Brown says never comes
up in real code. Except each line here was different.)
[toc] | [prev] | [next] | [standalone]
| From | Malcolm McLean <malcolm.arthur.mclean@gmail.com> |
|---|---|
| Date | 2019-01-21 04:15 -0800 |
| Message-ID | <00b249c4-4b32-4781-92a9-afb6888b80f9@googlegroups.com> |
| In reply to | #140048 |
On Monday, January 21, 2019 at 11:58:27 AM UTC, Bart wrote: > > (Last month I was working on generated programs designed to test new > floating point processing in my C compiler tokeniser. This was flat code > of 100,000 lines of C, just the sort that David Brown says never comes > up in real code. Except each line here was different.) < Code isn't necessarily human-written. A program to test the compiler's floating point handling wouldn't qualify as a "real program" however. It has to be for some useful end purpose (or for time-wasting, like a video game).
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2019-01-22 07:53 +1300 |
| Message-ID | <gamiopFuek4U11@mid.individual.net> |
| In reply to | #140048 |
On 22/01/2019 00:58, Bart wrote: > On 21/01/2019 03:33, Ian Collins wrote: >> On 21/01/2019 13:36, Bart wrote: >>> On 20/01/2019 23:39, David Brown wrote: >>>> On 20/01/2019 23:41, Bart wrote: >>> >>>> And to spoon-feed you the point, your absurd "test" files bear no >>>> relation to real source files that people want to compile. If you want >>>> useful results from tests, you have to test with realistic cases. So >>>> your stream of bytes for networking will be real test data, or perhaps >>>> random data, not a million zero bytes. Your graphics test scenes will >>>> be real pictures, not a blank window. >>> >>> I can see you've done very little performance optimisation, or you >>> haven't done it very well. >> >> A strange conclusion to draw. >> >>>> So if you want to test speed of >>>> compilation, use real code. >>> >>> That will depend on a million factors which all add up. Where do you >>> start? As soon as you isolate one small part, you find that not enough >>> real data exists to give meaningful timings. So you have to duplicate >>> input in a way that doesn't happen in real data. >> >> You could, as I suggested earlier, compile suitable large real source >> file such as the SQLite combined file (~200K lines). This is one of the >> biggest "real world" source files a compiler is likely to encounter. > > You're starting from scratch writing a C compiler, with performance as > one of the aims, and the first thing you do is attempt to compile a > 219,000 line program? You weren't doing that, you were comparing different working compilers. -- Ian.
[toc] | [prev] | [next] | [standalone]
| From | BGB <cr88192@hotmail.com> |
|---|---|
| Date | 2019-01-23 01:15 -0600 |
| Message-ID | <q294b2$t07$1@news.albasani.net> |
| In reply to | #140048 |
On 1/21/2019 5:58 AM, Bart wrote: > On 21/01/2019 03:33, Ian Collins wrote: >> On 21/01/2019 13:36, Bart wrote: >>> On 20/01/2019 23:39, David Brown wrote: >>>> On 20/01/2019 23:41, Bart wrote: >>> >>>> And to spoon-feed you the point, your absurd "test" files bear no >>>> relation to real source files that people want to compile. If you want >>>> useful results from tests, you have to test with realistic cases. So >>>> your stream of bytes for networking will be real test data, or perhaps >>>> random data, not a million zero bytes. Your graphics test scenes will >>>> be real pictures, not a blank window. >>> >>> I can see you've done very little performance optimisation, or you >>> haven't done it very well. >> >> A strange conclusion to draw. >> >>>> So if you want to test speed of >>>> compilation, use real code. >>> >>> That will depend on a million factors which all add up. Where do you >>> start? As soon as you isolate one small part, you find that not enough >>> real data exists to give meaningful timings. So you have to duplicate >>> input in a way that doesn't happen in real data. >> >> You could, as I suggested earlier, compile suitable large real source >> file such as the SQLite combined file (~200K lines). This is one of >> the biggest "real world" source files a compiler is likely to encounter. > > You're starting from scratch writing a C compiler, with performance as > one of the aims, and the first thing you do is attempt to compile a > 219,000 line program? > I just went and got SQLite building with my compiler. In the course of it, ended up having to implement some things, as it stumbled on a few more "implementation holes" in my compiler, such as me needing to implement proper lexical scoping for local variables, ... Previously, I was using a flat space for local variables; but SQLite sometimes defines variables with overlapping names in different sub-blocks. Generally, my own code, as well as Doom and Quake, did not make use of these semantics. In this case, it is still using a flat space as far as the middle and backend is concerned, but logic was added in the frontend to effectively rename the local variables to include a sequence numbers for the containing blocks (ex: "foo" becomes "foo!39" and similar). This was handled in the frontend for technical reasons (the original block structure, and thus symbol visibility, is lost once the AST no longer exists). Also needed to change some how it was managing 'static' variables within functions; and to add support for struct instances held in temporaries; ... But, did eventually get it to build seemingly successfully (whether or not everything works correctly is another matter). Currently (with my compiler built with debug settings), building SQLite takes 1.2 seconds and uses approx 200MB of memory. Built with optimization enabled, it compiles SQLite in 0.7 seconds. Comparing: GCC takes 2.4 seconds; Clang takes 1.5 seconds. This is on my (new) Ryzen 7; Generally, Clang was somewhat more sluggish than GCC on my previous AMD FX. The work on my compilers seems to have created a few new bugs though (which causes occasional memory corruption in Doom and Quake), but otherwise things mostly still seem to work. May need to go try to hunt down whatever got messed up. > Which happens to include (on Windows) "windows.h" which, if you use a > version from mingw or msvc, involves hundreds of #includes and hundreds > of thousands more lines of declarations. > > That is totally impractical; it's only going to work at all when the > compiler is nearly finished. (Imagine NASA testing a Saturn V for the > first time on an actual trip to the moon.) > FWIW, "windows.h" isn't actually all that complicated in terms of its contents; it is mostly just bulky. Once one gets through the preprocessing stage, it is mostly all struct definitions, typedefs, and function prototypes. > You usually start off writing a bare tokeniser (no symbol lookups), then > one with lookups, then, for C, superimposing a macro processor. > Not sure how much I want to go into tokenization and parsing. There are several different strategies I have used, with none really being "obviously superior". My C compiler's parser uses a comparatively old design, partly as it is a surviving remnant of one of my early script-language interpreters, but it still works pretty acceptably even if "not exactly particularly elegant". > For the first, the input can be anything at all, so long as it's big. > I've obtained throughput of over 10M lines per second, so even your > sqlite program [ignoring #-directives as they are just tokens at this > point] would take only 20msec, which is not enough to properly measure > just by itself; you may want to repeat it 10 or 50 times. > > On the journey, however, even that is too complex. Suppose there's a > part that the lexer is weak on, be it processing strings or identifiers > or numeric constants, how can you tell from input as chaotic as sqlite3.c? > Some amount of it is being pedantic in how the code is structured. When writing the logic, one either it handles the case in some sensible way (or make a best effort to do so), or one of the appropriate error functions is used. If an error function is called, it points out when things "need to be looked at". While it is more effort, for compilers it does make debugging a little easier overall. The contrast being to try to pile on a bunch of "make it work" logic, which tends to make a mess and is a pain to debug. > You might construct an input of millions of the same number just so you > can hone that part of the program. And so on and so forth. > > It's an incremental process and you need simple, controllable inputs. > This sort of testing typically has a relatively low probability of finding new bugs. It is OK as a benchmark, but if the logic already works, simply increasing the size of the input wont do much to find new bugs. A better case would be tests which aim to have relatively complete coverage of the language, which attempts to test every major language feature in various ways, in an attempt to verify that the various features work as expected. If a value somewhere differs from what is expected, the program can then trigger a breakpoint at that location. Such tests might also include things that are "technically UB" if they are an expected or defined part of the behavior for the target machine. Though, I have partly ended up relying on Doom and Quake to detect when things have gotten broke, as they tend to detect compiler screw-ups a little easier than my attempts to effectively detect issues via compiler tests. But, some basic tests (performed early on during program start-up) still work OK as "baseline sanity checks", typically causing these test programs to fail early on and in a more obvious way if something major gets broken. > ========= > > BTW here's what the finished product looks like actually working on > sqlite3 (using my own small windows.h): > > C:\cx\big>timeit "..\cc -c sqlite3" # cc.exe passed thru gcc-O3 > Compiling sqlite3.c to sqlite3.asm > ..\cc -c sqlite3 0.30 > > C:\cx\big>timeit "bcc -c sqlite3" # bcc.exe used my compiler > Compiling sqlite3.c to sqlite3.asm > bcc -c sqlite3 0.38 > > So it still takes only 0.3 seconds for a total of 222Kloc. Internal > timing (clock() difference between program entry and exit) is about > 250ms. And that includes all passes. If you isolate some passes to > concentrate on those, then timings will be much less. > > Since timing resolution is only 16ms (using clock(), 4-8ms using other > means but also more variable), it's harder to measure the effect of any > small change. > > Hence it is necessary to partially synthesise the input in order to > magnify the effects of any changes. > > sqlite3.c itself only has 11,000 symbols; yesterday I was working with 2 > million. > > (Last month I was working on generated programs designed to test new > floating point processing in my C compiler tokeniser. This was flat code > of 100,000 lines of C, just the sort that David Brown says never comes > up in real code. Except each line here was different.) >
[toc] | [prev] | [next] | [standalone]
| From | Bart <bc@freeuk.com> |
|---|---|
| Date | 2019-01-23 12:00 +0000 |
| Message-ID | <9HY1E.78436$nC1.37667@fx04.am4> |
| In reply to | #140106 |
On 23/01/2019 07:15, BGB wrote:
> On 1/21/2019 5:58 AM, Bart wrote:
>> You're starting from scratch writing a C compiler, with performance as
>> one of the aims, and the first thing you do is attempt to compile a
>> 219,000 line program?
>>
>
> I just went and got SQLite building with my compiler.
>
>
> In the course of it, ended up having to implement some things, as it
> stumbled on a few more "implementation holes" in my compiler, such as me
> needing to implement proper lexical scoping for local variables, ...
It took me ages to get my compiler properly building sqlite3.c,
especially using its own windows.h rather one that was borrowed for the
purpose.
> Currently (with my compiler built with debug settings), building SQLite
> takes 1.2 seconds and uses approx 200MB of memory.
>
> Built with optimization enabled, it compiles SQLite in 0.7 seconds.
>
>
> Comparing:
> GCC takes 2.4 seconds;
> Clang takes 1.5 seconds.
>
> This is on my (new) Ryzen 7; Generally, Clang was somewhat more sluggish
> than GCC on my previous AMD FX.
Is that on Windows?
Your machine seems to be faster than my PC. I get these timings for
compiling sqlite3.c (the 219Kloc version) to either .obj or .asm using "-c":
bcc 0.3 secs (via C and gcc-O3)
bcc 0.4 (uses own compiler)
msvc 2.1
lccwin 2.2
clang 4.4
gcc 8.1.0 8.6
pelles c 17.8
tcc Fails (some missing decl in windows.h I think)
dmc Fails
(bcc is my compiler, and for a "-c" target file, it uses its own asm
format, which is fed direct to a linker. Note that it uses its own
compact windows.h which might give it an edge.)
A working executable using sqlite3 can be created with the files
sqlite3.c, sqlite3.h and shell.c (all downloaded from the same source).
Some compilers have trouble with shell.c; results building those
together to form an exe are as follows:
bcc 0.5 secs (via C and gcc-O3)
bcc 0.6 (own unoptimised compiler)
msvc 3.1
clang 6.3
gcc 11.3
tcc Fails (both files)
dmc Fails (both files)
pelles c Fails (shell.c)
lccwin Fails (shell.c)
bcc did need a bit of a tweak to compile shell.c, in the form of this macro:
#if __BCC__
#define _MSC_VER 1
#endif
This makes shell.c think MSVC is being used. This doesn't help any of
the other failing compilers except tcc, but that can't complete the .exe
because of sqlite3.c. When it works, then tcc will be the fastest.
Compiling and running this program looks like this:
C:\cx\big>bcc -run sqlite3 shell
Compiling sqlite3.c to sqlite3.asm
Compiling shell.c to sqlite3.asm
Assembling to sqlite3.exe
SQLite version 3.25.3/BCC 2018-11-05 20:37:38
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>
which is far as I went (I don't know SQL). Doubtless there are numerous
code gen bugs which will make it go wrong, but I was happy to get that
far. It's a bit further than some other compilers at least!
(Notice the /BCC in the opening message; I put that in to ensure it was
running my version as I couldn't quite believe it worked.)
> FWIW, "windows.h" isn't actually all that complicated in terms of its
> contents; it is mostly just bulky.
Trying to directly use windows.h from MSVC or mingw (which involves
90-160 header files in all) doesn't work because my compiler isn't MSVC
(the MSC_VER trick is not enough) and it isn't gcc. I have to recreate it.
Currently my windows.h is 2.6Kloc. MS' version is 200Kloc I think, and
even slimmed down versions of other compilers are 20Kloc and up.
>
> Once one gets through the preprocessing stage, it is mostly all struct
> definitions, typedefs, and function prototypes.
Yes, adding stuff as needed, that is what you end up doing. And it goes
on and on and on. Maybe there are automatic ways of doing this, but I
haven't had much luck. (I worked on something for extracting the
interface from the 300,000 lines of GTK2 headers, but that was for use
with a different language. And that project has stalled.)
[toc] | [prev] | [next] | [standalone]
| From | Ian Collins <ian-news@hotmail.com> |
|---|---|
| Date | 2019-01-24 08:13 +1300 |
| Message-ID | <garsnnFfhgbU1@mid.individual.net> |
| In reply to | #140114 |
On 24/01/2019 01:00, Bart wrote: > On 23/01/2019 07:15, BGB wrote: >> >> >> Comparing: >> GCC takes 2.4 seconds; >> Clang takes 1.5 seconds. >> >> This is on my (new) Ryzen 7; Generally, Clang was somewhat more sluggish >> than GCC on my previous AMD FX. > > Is that on Windows? > > Your machine seems to be faster than my PC. I get these timings for > compiling sqlite3.c (the 219Kloc version) to either .obj or .asm using "-c": > > bcc 0.3 secs (via C and gcc-O3) > bcc 0.4 (uses own compiler) > msvc 2.1 > lccwin 2.2 > clang 4.4 > gcc 8.1.0 8.6 > pelles c 17.8 > tcc Fails (some missing decl in windows.h I think) > dmc Fails Ubuntu on Ryzen: $ time gcc -c sqlite3.c real 0m2.762s user 0m2.599s sys 0m0.092s $ time clang -c sqlite3.c real 0m1.230s user 0m1.172s sys 0m0.037s > Compiling and running this program looks like this: > > C:\cx\big>bcc -run sqlite3 shell > Compiling sqlite3.c to sqlite3.asm > Compiling shell.c to sqlite3.asm > Assembling to sqlite3.exe > > SQLite version 3.25.3/BCC 2018-11-05 20:37:38 > Enter ".help" for usage hints. > Connected to a transient in-memory database. > Use ".open FILENAME" to reopen on a persistent database. > sqlite> > > which is far as I went (I don't know SQL). Doubtless there are numerous > code gen bugs which will make it go wrong, but I was happy to get that > far. It's a bit further than some other compilers at least! https://www.sqlite.org/th3.html -- Ian.
[toc] | [prev] | [next] | [standalone]
| From | BGB <cr88192@hotmail.com> |
|---|---|
| Date | 2019-01-23 13:30 -0600 |
| Message-ID | <q2afd1$9fi$1@news.albasani.net> |
| In reply to | #140114 |
On 1/23/2019 6:00 AM, Bart wrote: > On 23/01/2019 07:15, BGB wrote: >> On 1/21/2019 5:58 AM, Bart wrote: > >>> You're starting from scratch writing a C compiler, with performance >>> as one of the aims, and the first thing you do is attempt to compile >>> a 219,000 line program? >>> >> >> I just went and got SQLite building with my compiler. >> >> >> In the course of it, ended up having to implement some things, as it >> stumbled on a few more "implementation holes" in my compiler, such as >> me needing to implement proper lexical scoping for local variables, ... > > It took me ages to get my compiler properly building sqlite3.c, > especially using its own windows.h rather one that was borrowed for the > purpose. > This build is not targeting Windows, it is targeting "other" and then running on top of a "bare metal" C library, so no "windows.h" in this case. But, my compiler can parse "windows.h" though (if albeit doing so is kinda pointless on a "bare metal" target). My compiler does generally do a lot of things similarly to MSVC, but doesn't currently target Windows. In some of my own projects (both for my CPU project and CNC lathe), the output is for a PE/COFF variant targeting a custom ISA. On the lathe, it is just sort of emulated on top of ARM, but a VM is used (rather than plain ARM code), mostly because in this case it needs to be cut up into small time-slices. So, ARM on ARM still has the issue that ARM is comparably a bit more costly to emulate (even on top of ARM), ..., and conventional threading makes scheduler timing issues worse. This is partly as Linux is seemingly hell-bent on still periodically interrupting threads even if their priority is basically maxed out and they have their own CPU core available. At least, if not much is going on on the processor otherwise (no other active threads), the interruptions are generally smaller and less frequent. The thread that is running the motor controls and external electronic interfaces ideally needs as few interruptions as possible (needs to do things within a few microseconds of a specific time, deal with external input signals, ...); so is left to run with basically maxed out scheduler priorities. But much of the time still has little to do (at any given moment), besides sit there and spin in delay loop, but it can be made to "do something useful" if it can also run a VM (instead of just spinning in an delay loop). Main requirement though is that the VM not itself produce unpredictable delays. This is easier to pull off with a lower-level CPU emulator though than with a "high level" VM design (it was actually this issue previously which had led to my BS2 language being designed mostly the way it was, but the VM design than ended up being implemented for it still has some issues here). The VM can then run lower priority stuff, like UI and similar. >> Currently (with my compiler built with debug settings), building >> SQLite takes 1.2 seconds and uses approx 200MB of memory. >> >> Built with optimization enabled, it compiles SQLite in 0.7 seconds. >> >> >> Comparing: >> GCC takes 2.4 seconds; >> Clang takes 1.5 seconds. >> >> This is on my (new) Ryzen 7; Generally, Clang was somewhat more >> sluggish than GCC on my previous AMD FX. > > Is that on Windows? > This case was with WSL (or, MS's "Linux on top of Windows" thing). > Your machine seems to be faster than my PC. I get these timings for > compiling sqlite3.c (the 219Kloc version) to either .obj or .asm using > "-c": > > bcc 0.3 secs (via C and gcc-O3) > bcc 0.4 (uses own compiler) > msvc 2.1 > lccwin 2.2 > clang 4.4 > gcc 8.1.0 8.6 > pelles c 17.8 > tcc Fails (some missing decl in windows.h I think) > dmc Fails > > (bcc is my compiler, and for a "-c" target file, it uses its own asm > format, which is fed direct to a linker. Note that it uses its own > compact windows.h which might give it an edge.) > OK, WSL in this case, still using the "other" target. New PC is a 3.7 GHz Ryzen 7 2700X with DDR4-2933. This thing generally goes approx 3x faster for many things than my AMD FX-8350 with DDR3-1333, which I had underclocked slightly (from the stock 4.0GHz to 3.7GHz, and had disabled turbo-boost, ...) mostly so it wouldn't try to melt under high CPU load. Performance vs the FX is pretty chaotic though, as things which were slower or faster on the FX may be reversed on the Ryzen. For example, the "Zstd" compressor was kinda slowish on my FX (not that much faster than GZIP), and I had some faster LZ compressors. On the Ryzen, Zstd got a pretty significant boost, whereas my LZ compressors didn't get much of a speedup (and got surpassed by Zstd). One of my previous compressors, which beat out LZ4 by using only DWORD memory access (rather than LZ4 using byte-oriented memory access), similarly gets beaten slightly by LZ4 on the Ryzen (with both pulling off decode speeds of ~ 3.0 to 3.5 GB per second; but LZ4 generally getting better compression by not being limited to working in multiples of 4 bytes). So, it gets a bit chaotic here... But, one annoyance is, for whatever reason, a lot of my stuff, even if I create a lot of worker threads, does not exceed ~ 8% CPU (or gain much speed up vs the single threaded case). As of yet, haven't figured out what is going on here (it is like, all the threads in a given process keep end up being scheduled on the same CPU core). Though, some stuff is still going decently fast even with this limitation. I suspect it is probably some recent Windows scheduler funkiness or something... I have seen some other programs briefly go well over this 8% limit though, just mostly need to figure it out. > A working executable using sqlite3 can be created with the files > sqlite3.c, sqlite3.h and shell.c (all downloaded from the same source). > > Some compilers have trouble with shell.c; results building those > together to form an exe are as follows: > > bcc 0.5 secs (via C and gcc-O3) > bcc 0.6 (own unoptimised compiler) > msvc 3.1 > clang 6.3 > gcc 11.3 > tcc Fails (both files) > dmc Fails (both files) > pelles c Fails (shell.c) > lccwin Fails (shell.c) > > bcc did need a bit of a tweak to compile shell.c, in the form of this > macro: > > #if __BCC__ > #define _MSC_VER 1 > #endif > > This makes shell.c think MSVC is being used. This doesn't help any of > the other failing compilers except tcc, but that can't complete the .exe > because of sqlite3.c. When it works, then tcc will be the fastest. > > Compiling and running this program looks like this: > > C:\cx\big>bcc -run sqlite3 shell > Compiling sqlite3.c to sqlite3.asm > Compiling shell.c to sqlite3.asm > Assembling to sqlite3.exe > > SQLite version 3.25.3/BCC 2018-11-05 20:37:38 > Enter ".help" for usage hints. > Connected to a transient in-memory database. > Use ".open FILENAME" to reopen on a persistent database. > sqlite> > > which is far as I went (I don't know SQL). Doubtless there are numerous > code gen bugs which will make it go wrong, but I was happy to get that > far. It's a bit further than some other compilers at least! > > (Notice the /BCC in the opening message; I put that in to ensure it was > running my version as I couldn't quite believe it worked.) > OK. >> FWIW, "windows.h" isn't actually all that complicated in terms of its >> contents; it is mostly just bulky. > > Trying to directly use windows.h from MSVC or mingw (which involves > 90-160 header files in all) doesn't work because my compiler isn't MSVC > (the MSC_VER trick is not enough) and it isn't gcc. I have to recreate it. > > Currently my windows.h is 2.6Kloc. MS' version is 200Kloc I think, and > even slimmed down versions of other compilers are 20Kloc and up. > I had used both versions before in tests. >> >> Once one gets through the preprocessing stage, it is mostly all struct >> definitions, typedefs, and function prototypes. > > Yes, adding stuff as needed, that is what you end up doing. And it goes > on and on and on. Maybe there are automatic ways of doing this, but I > haven't had much luck. (I worked on something for extracting the > interface from the 300,000 lines of GTK2 headers, but that was for use > with a different language. And that project has stalled.) > > OK. Can't exactly claim to be a fan of GTK's API design...
[toc] | [prev] | [next] | [standalone]
Page 1 of 8 [1] 2 3 4 5 6 7 8 Next page →
Back to top | Article view | comp.lang.c
csiph-web