Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #140106
| From | BGB <cr88192@hotmail.com> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: Compiler tests |
| Date | 2019-01-23 01:15 -0600 |
| Organization | albasani.net |
| Message-ID | <q294b2$t07$1@news.albasani.net> (permalink) |
| References | (9 earlier) <GO61E.101549$dE1.32673@fx22.am4> <q230r4$ues$1@dont-email.me> <5u81E.116645$dp2.90631@fx18.am4> <gakss0Fuek4U10@mid.individual.net> <vti1E.113058$Ur2.71958@fx14.am4> |
On 1/21/2019 5:58 AM, Bart wrote: > On 21/01/2019 03:33, Ian Collins wrote: >> On 21/01/2019 13:36, Bart wrote: >>> On 20/01/2019 23:39, David Brown wrote: >>>> On 20/01/2019 23:41, Bart wrote: >>> >>>> And to spoon-feed you the point, your absurd "test" files bear no >>>> relation to real source files that people want to compile. If you want >>>> useful results from tests, you have to test with realistic cases. So >>>> your stream of bytes for networking will be real test data, or perhaps >>>> random data, not a million zero bytes. Your graphics test scenes will >>>> be real pictures, not a blank window. >>> >>> I can see you've done very little performance optimisation, or you >>> haven't done it very well. >> >> A strange conclusion to draw. >> >>>> So if you want to test speed of >>>> compilation, use real code. >>> >>> That will depend on a million factors which all add up. Where do you >>> start? As soon as you isolate one small part, you find that not enough >>> real data exists to give meaningful timings. So you have to duplicate >>> input in a way that doesn't happen in real data. >> >> You could, as I suggested earlier, compile suitable large real source >> file such as the SQLite combined file (~200K lines). This is one of >> the biggest "real world" source files a compiler is likely to encounter. > > You're starting from scratch writing a C compiler, with performance as > one of the aims, and the first thing you do is attempt to compile a > 219,000 line program? > I just went and got SQLite building with my compiler. In the course of it, ended up having to implement some things, as it stumbled on a few more "implementation holes" in my compiler, such as me needing to implement proper lexical scoping for local variables, ... Previously, I was using a flat space for local variables; but SQLite sometimes defines variables with overlapping names in different sub-blocks. Generally, my own code, as well as Doom and Quake, did not make use of these semantics. In this case, it is still using a flat space as far as the middle and backend is concerned, but logic was added in the frontend to effectively rename the local variables to include a sequence numbers for the containing blocks (ex: "foo" becomes "foo!39" and similar). This was handled in the frontend for technical reasons (the original block structure, and thus symbol visibility, is lost once the AST no longer exists). Also needed to change some how it was managing 'static' variables within functions; and to add support for struct instances held in temporaries; ... But, did eventually get it to build seemingly successfully (whether or not everything works correctly is another matter). Currently (with my compiler built with debug settings), building SQLite takes 1.2 seconds and uses approx 200MB of memory. Built with optimization enabled, it compiles SQLite in 0.7 seconds. Comparing: GCC takes 2.4 seconds; Clang takes 1.5 seconds. This is on my (new) Ryzen 7; Generally, Clang was somewhat more sluggish than GCC on my previous AMD FX. The work on my compilers seems to have created a few new bugs though (which causes occasional memory corruption in Doom and Quake), but otherwise things mostly still seem to work. May need to go try to hunt down whatever got messed up. > Which happens to include (on Windows) "windows.h" which, if you use a > version from mingw or msvc, involves hundreds of #includes and hundreds > of thousands more lines of declarations. > > That is totally impractical; it's only going to work at all when the > compiler is nearly finished. (Imagine NASA testing a Saturn V for the > first time on an actual trip to the moon.) > FWIW, "windows.h" isn't actually all that complicated in terms of its contents; it is mostly just bulky. Once one gets through the preprocessing stage, it is mostly all struct definitions, typedefs, and function prototypes. > You usually start off writing a bare tokeniser (no symbol lookups), then > one with lookups, then, for C, superimposing a macro processor. > Not sure how much I want to go into tokenization and parsing. There are several different strategies I have used, with none really being "obviously superior". My C compiler's parser uses a comparatively old design, partly as it is a surviving remnant of one of my early script-language interpreters, but it still works pretty acceptably even if "not exactly particularly elegant". > For the first, the input can be anything at all, so long as it's big. > I've obtained throughput of over 10M lines per second, so even your > sqlite program [ignoring #-directives as they are just tokens at this > point] would take only 20msec, which is not enough to properly measure > just by itself; you may want to repeat it 10 or 50 times. > > On the journey, however, even that is too complex. Suppose there's a > part that the lexer is weak on, be it processing strings or identifiers > or numeric constants, how can you tell from input as chaotic as sqlite3.c? > Some amount of it is being pedantic in how the code is structured. When writing the logic, one either it handles the case in some sensible way (or make a best effort to do so), or one of the appropriate error functions is used. If an error function is called, it points out when things "need to be looked at". While it is more effort, for compilers it does make debugging a little easier overall. The contrast being to try to pile on a bunch of "make it work" logic, which tends to make a mess and is a pain to debug. > You might construct an input of millions of the same number just so you > can hone that part of the program. And so on and so forth. > > It's an incremental process and you need simple, controllable inputs. > This sort of testing typically has a relatively low probability of finding new bugs. It is OK as a benchmark, but if the logic already works, simply increasing the size of the input wont do much to find new bugs. A better case would be tests which aim to have relatively complete coverage of the language, which attempts to test every major language feature in various ways, in an attempt to verify that the various features work as expected. If a value somewhere differs from what is expected, the program can then trigger a breakpoint at that location. Such tests might also include things that are "technically UB" if they are an expected or defined part of the behavior for the target machine. Though, I have partly ended up relying on Doom and Quake to detect when things have gotten broke, as they tend to detect compiler screw-ups a little easier than my attempts to effectively detect issues via compiler tests. But, some basic tests (performed early on during program start-up) still work OK as "baseline sanity checks", typically causing these test programs to fail early on and in a more obvious way if something major gets broken. > ========= > > BTW here's what the finished product looks like actually working on > sqlite3 (using my own small windows.h): > > C:\cx\big>timeit "..\cc -c sqlite3" # cc.exe passed thru gcc-O3 > Compiling sqlite3.c to sqlite3.asm > ..\cc -c sqlite3 0.30 > > C:\cx\big>timeit "bcc -c sqlite3" # bcc.exe used my compiler > Compiling sqlite3.c to sqlite3.asm > bcc -c sqlite3 0.38 > > So it still takes only 0.3 seconds for a total of 222Kloc. Internal > timing (clock() difference between program entry and exit) is about > 250ms. And that includes all passes. If you isolate some passes to > concentrate on those, then timings will be much less. > > Since timing resolution is only 16ms (using clock(), 4-8ms using other > means but also more variable), it's harder to measure the effect of any > small change. > > Hence it is necessary to partially synthesise the input in order to > magnify the effects of any changes. > > sqlite3.c itself only has 11,000 symbols; yesterday I was working with 2 > million. > > (Last month I was working on generated programs designed to test new > floating point processing in my C compiler tokeniser. This was flat code > of 100,000 lines of C, just the sort that David Brown says never comes > up in real code. Except each line here was different.) >
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Compiler tests Bart <bc@freeuk.com> - 2019-01-20 00:03 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-20 16:52 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-20 10:27 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 12:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 13:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 15:02 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:31 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 21:40 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 22:41 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 00:39 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 00:36 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-21 16:33 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 11:58 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 04:15 -0800
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-22 07:53 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 01:15 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 12:00 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-24 08:13 +1300
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 13:30 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 08:22 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:23 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 14:42 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 14:17 +0000
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:46 -0800
You guys really, really need to get a room (Was: Compiler tests) gazelle@shell.xmission.com (Kenny McCormack) - 2019-01-24 14:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-20 17:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 17:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 09:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 12:49 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:31 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 14:38 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 15:15 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-21 16:05 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 16:39 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-21 12:51 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 00:16 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 08:43 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:31 +0000
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-22 17:49 +0100
Re: Compiler tests jacobnavia <jacob@jacob.remcomp.fr> - 2019-01-23 11:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 11:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-21 15:35 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-21 15:49 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-21 17:42 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 03:18 -0600
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 11:11 +0100
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-23 12:18 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 13:56 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 10:55 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:07 +0000
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 04:29 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 12:52 +0000
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-22 05:04 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:18 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:45 -0800
Re: Compiler tests fir <profesor.fir@gmail.com> - 2019-01-22 05:35 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 15:10 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 17:30 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-22 23:02 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-22 23:14 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 15:45 -0800
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:57 -0600
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 16:36 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 01:37 +0000
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-22 18:53 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:42 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 09:22 +0100
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:52 -0500
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 15:13 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 14:33 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 07:30 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:05 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:17 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:29 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:04 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 21:50 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 10:45 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 11:55 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 13:58 +0100
Re: Compiler tests mark.bluemel@gmail.com - 2019-01-24 05:07 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:10 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 14:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:51 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 15:44 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 17:32 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-24 19:07 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 20:14 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 22:24 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-24 22:46 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 12:34 +1300
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 01:01 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-25 14:15 +1300
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:05 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:51 +0000
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 09:48 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:56 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 12:08 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:28 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 13:33 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:38 +0100
Re: Compiler tests Mateusz Viste <mateusz@wont.tell> - 2019-01-25 15:08 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 13:53 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:25 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 10:03 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 10:08 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:14 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 12:02 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 13:37 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 13:58 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:26 +0100
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 15:01 +0100
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:42 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:25 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:37 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 16:33 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:44 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 17:53 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 10:33 -0800
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:43 +0100
Re: Compiler tests Keith Thompson <kst-u@mib.org> - 2019-01-25 12:17 -0800
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 18:31 +0000
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 20:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 21:19 +0000
Re: Compiler tests Ian Collins <ian-news@hotmail.com> - 2019-01-26 10:24 +1300
Re: Compiler tests "Pascal J. Bourguignon" <pjb@informatimago.com> - 2019-01-25 22:27 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:39 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:46 +0000
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 12:43 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:37 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 14:19 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 15:47 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-25 07:17 -0800
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 07:28 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 12:35 +0100
Re: Compiler tests Jorgen Grahn <grahn+nntp@snipabacken.se> - 2019-02-03 21:47 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-02-03 22:57 +0100
Re: Compiler tests Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2019-01-24 06:47 -0800
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 20:22 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 22:12 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-23 21:54 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-24 11:41 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-24 23:17 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 11:11 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 12:04 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 14:05 +0100
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 18:40 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-25 22:36 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-25 22:15 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:48 +0000
Re: Compiler tests scott@slp53.sl.home (Scott Lurndal) - 2019-01-25 14:46 +0000
Re: Compiler tests Ben Bacarisse <ben.usenet@bsb.me.uk> - 2019-01-25 17:48 +0000
Re: Compiler tests James Kuyper <jameskuyper@alumni.caltech.edu> - 2019-01-23 08:23 -0800
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 16:41 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 16:18 +0000
Re: Compiler tests David Brown <david.brown@hesbynett.no> - 2019-01-23 17:28 +0100
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-23 20:23 +0000
Re: Compiler tests Robert Wessel <robertwessel2@yahoo.com> - 2019-01-22 17:52 -0600
Re: Compiler tests Bart <bc@freeuk.com> - 2019-01-20 20:17 +0000
Re: Compiler tests BGB <cr88192@hotmail.com> - 2019-01-21 00:13 -0600
csiph-web