Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #136704
| From | BGB <cr88192@hotmail.com> |
|---|---|
| Newsgroups | comp.lang.c |
| Subject | Re: software disenchantment |
| Date | 2018-10-11 20:15 -0500 |
| Organization | albasani.net |
| Message-ID | <pposj3$mtu$1@news.albasani.net> (permalink) |
| References | (18 earlier) <ppn5kf$clr$1@dont-email.me> <RVFvD.15651$I34.1016@fx43.am4> <ppneji$ke$1@dont-email.me> <ppo38r$jr7$1@news.albasani.net> <ppo6s0$mog$1@dont-email.me> |
On 10/11/2018 2:04 PM, David Brown wrote: > On 11/10/18 20:03, BGB wrote: >> On 10/11/2018 7:10 AM, David Brown wrote: >>> On 11/10/18 12:50, Bart wrote: >>>> On 11/10/2018 10:37, David Brown wrote: >>>>> On 11/10/18 00:28, Bart wrote: >>>> >>>>>> Would 2500 object files need to be accessed in the course of even an >>>>>> incremental build? Or is it clever enough to patch in only the >>>>>> ones that >>>>>> have changed? >>>> >>>>> So for any incremental build, usually only a small fraction (typically >>>>> just one) of the object files get re-built. >>>>> >>>>> If you have incremental linking, then much of the previous link >>>>> results >>>>> are also kept - if not, then you may have to read all the object files >>>>> for the link. >>>> >>>> I said 2500 OBJECT files. >>> >>> Yes, I know you said that. >>> >>>> So that already assumes that only some source >>>> files are re-compiled. >>> >>> No, it does not imply this. But that does not matter to my point. >>> >>> It is not clear what you mean by "would 2500 object files need to be >>> accessed" - accessed in what way? I took that to mean "read" - if you >>> had meant "written" or "generated", I think you would have said that. >>> The only reason you read object files in a typical build process is for >>> the linking. >>> >>> It is common for the link to use all the object files in one link >>> command. But if you have a bigger project, you might use incremental >>> linking - then you don't need to read all the object files for each >>> link. (Link-time optimisation is also possible, but more complex.) >>> >> >> At least with MSVC, I have often disabled incremental linking (on by >> default), as it tends to reduce performance and occasionally "fouls >> up" in ways which require manually deleting the output files (the >> produced binary is broken; or sometimes "link.exe" will just keep >> crashing until the output files are deleted, *). > > Don't judge a general technique from MS' incompetence at implementing it. > I guess things could be better if there were an "oh crap, link.exe crashed" feature which could automatically delete the output EXE and ILK files and then try again (vs the user needing to do this manually when it happens). Often, it ends up easier to just stick on the command-line option to disable this feature and not deal with it anymore. >> >> *: Though, usually if "link.exe" crashes, I have found it is easiest >> to just delete the output so it can start clean (whether or not it >> crashes again, most of the time the binary is broken). >> >> Not tried it with GCC or similar, so don't really know how it compares. >> >> >>>> >>>>> Some timings for a project with 113 object files are: >>>>> >>>>> Serial "clean" build: 51s >>>>> Parallel "clean" build: 12s >>>>> Incremental build for one C file changed: 1.8s >>>>> >>>>> Does that explain why people use incremental builds? >>>> >>>> Incremental builds and multi-processing. It's what you have to throw at >>>> a problem when the basic methods are slow and inefficient. >>>> >>> >>> No, it is what you use to make everything go much faster. It does not >>> matter how fast your compilations are - with big enough projects, >>> "compile everything, one file at a time" will be slow in the end. If >>> your super compiler takes a mere 50 ms to compile a file, then with 2500 >>> files it takes 2 minutes. A parallel clean build on an 8-core PC will >>> take 15 seconds. An incremental build will take 50 ms. >>> >>> Serial build-everything-everytime builds are fine for toy projects - but >>> not for more serious work, /regardless/ of the compiler speed. >>> >> >> IME, "always rebuild everything" works up to a certain scale. > > Yes, it does. Incremental builds make a bigger difference the bigger > the project. > > However, for anything other than a simple test or "hello, world", you > normally want some control of the build - names, compiler switches, > libraries, etc. You want control of directories so that your source > directories are not jumbled with object files and other generated files. > You need somewhere to hold this information - the build information > and "project management" bit. Some people like to do this with an IDE, > others with a build system like "make". (I like to use "make" for the > build itself, but have an IDE setup to make the development smoother.) > > And once you have that, your incremental build is automatic. I have > incremental builds for even the smallest of projects with no more than a > dozen C files - it is easier than /not/ having it. > I use Makefiles a lot, at least once it passes the threshold of what can be reasonably handled by a "one liner" batch file or shell script. Usually end up having suffixes on the Makefiles though, given I need different options for building on Linux vs on Windows or similar. > (I don't bother with incremental links - my projects are not big enough > for that to be worth the effort setting it up.) > Yep. I also have doubts much of what I am doing would benefit all that much. >> >> Coarse grained incremental works well, like rebuilding individual >> DLL's or SO's, or building code as chunks of ~ 25kLOC or so, but still >> letting make skip over stuff that doesn't change on the larger scale. >> >> So, for example, for a fairly typical ~ 100 kLOC DLL, it could be >> organized into roughly 4 TU's (rather than compiling ~ 50 individual C >> or C++ files). >> >> >> If the program is a single binary, particularly if modest size, "just >> rebuild directly" works pretty well (and the build can be done as a >> "one liner" in a BAT or SH file). >> > > <snip> > (I'm snipping a lot here. I have read it, but if I try to reply to > points in it we'll mostly re-cover old ground, or continue endlessly.) > >> >> There is the questionable aspect of my compiler that it goes through a >> stack-based IL stage (functionally similar to .NET CIL in terms of how >> it behaves; though how metadata is handled is quite different), but >> this has turned out to still be useful (I can use it for things like >> the C library and similar). > > Stack-based intermediary languages are common in compilers (with SSA > being the other common style). > OK. I have mostly just seen compilers using COFF or ELF objects for object files (or older ones using OMF or similar), typically with the AR format used for static libraries, apart from Clang which uses LLVM bitcode for this. MSVC can produce CIL, but typically only when compiling for .NET (and thus using a VM), and produces plain COFF objects otherwise. It just seemed atypical for me to be using a stack IL (externally, if generating object files or static libraries), but still producing native-code as the final output. Most of the time about the only times I have seen a stack IL used otherwise has been in the context of targeting a VM. Mine uses a 3AC/SSA style in the backend, but (unlike LLVM) doesn't support a binary serialized representation for it (though a textual representation exists for debugging reasons). Specifics differ enough to where I couldn't easily reuse LLVM bitcode or SPIR-V (absent an ugly translation shim); namely differences in things like register-numbering and in terms of how various operations are represented and similar, ... It is also possible to serialize and reload the ASTs in a format I called SBXE (sort of analogous to WBXML), but haven't found much of a use-case for doing so (whereas being able to dump as textual XML is at least sometimes useful for debugging, nevermind if these files are potentially quite large and not particularly human-readable). > <snip> > >> One drawback IMO with GCC is that it tries to do too much with a >> single compiler, leading it to be quite big and unwieldy. >> >> Several smaller and more specialized compilers could possibly better >> address some things, with the main GCC mostly focusing on PC-like >> targets; probably with other compilers toolchains for various classes >> of embedded and microcontroller targets. >> > > gcc /does/ focus on a certain class of processor - what might be termed > "C friendly" processors. This covers a large number of devices, but > misses many classes entirely - DSP's, 8-bit accumulator processors, > anything with CHAR_BIT != 8 or signed integers that are not two's > complement. > I was meaning if, say, it focused mostly on x86 and ARM and similar using ELF and maybe PE/COFF and MachO as output formats. In this case, most of the embedded cross compilers could be done via forks or by other more specialized compilers; it doesn't seem like that much would be lost as which combination of features exist and the quality (or sanity) of the generated code already varies a bit from one target to another. Just because it is gcc doesn't mean it generates good code (or, for that matter, that trying to build a cross compiler for an obscure target wont just result in a mess of build errors), if it is for a target that isn't really being maintained. It doesn't seem like one would really lose all that much apart from the "brand familiarity" of it being gcc.
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
software disenchantment Bart <bc@freeuk.com> - 2018-10-06 15:13 +0100
Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-06 09:54 -0700
Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-06 10:26 -0700
Re: software disenchantment fir <profesor.fir@gmail.com> - 2018-10-14 04:26 -0700
Re: software disenchantment antispam@math.uni.wroc.pl - 2018-10-09 18:20 +0000
Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-09 11:54 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-09 21:22 +0100
Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-09 17:47 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 11:28 +0100
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 11:38 +0100
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-10 05:49 -0700
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-10 15:47 +0200
Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 09:58 -0400
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-10 14:39 +0000
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-10 09:42 -0700
Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 13:18 -0400
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 20:31 +0100
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 17:53 -0500
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 10:06 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 02:34 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 12:03 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 03:59 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 16:56 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 09:02 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 19:39 +0200
Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 19:53 +0000
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-16 03:54 -0500
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 20:25 +0200
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:16 +0000
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 07:15 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 17:05 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 09:15 -0700
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 17:33 +0000
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 19:53 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 17:59 +0100
Re: software disenchantment nobody@example.org (Scott) - 2018-10-10 16:53 +0000
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 12:45 -0500
Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-10 11:04 -0700
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 14:22 -0500
Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 15:53 -0400
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 09:06 +1300
Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 13:21 -0700
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-10 20:27 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 22:54 +0100
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:01 +1300
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:05 +1300
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 23:16 +0100
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:22 +1300
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 23:28 +0100
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:33 +1300
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 11:37 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 11:50 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 14:10 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 05:24 -0700
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 15:12 +0200
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:13 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 14:44 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 16:38 +0200
Re: software disenchantment mark.bluemel@gmail.com - 2018-10-11 08:09 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 17:43 +0100
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 17:36 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 18:56 +0100
Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 11:54 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 20:09 +0100
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:10 +0000
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 12:57 +0200
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 14:27 -0500
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 22:03 +0200
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 15:56 -0500
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 11:30 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 23:34 +0100
Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 16:24 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 01:46 +0100
Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 18:58 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-12 08:50 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 11:12 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 12:55 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 12:10 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:29 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 13:06 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 14:30 +0200
Re: software disenchantment James Kuyper <jameskuyper@alumni.caltech.edu> - 2018-10-12 08:53 -0400
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 15:11 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 13:55 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 15:13 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 16:10 +0100
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:22 +0000
Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-12 09:12 -0700
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 13:00 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 12:52 +0100
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-14 08:44 +1300
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 21:03 +0100
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-14 10:13 +1300
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 23:02 +0100
Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-13 16:47 -0700
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-14 01:05 +0100
Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-13 18:12 -0700
Re: software disenchantment Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-10-14 03:05 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-14 20:14 +0200
Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-14 11:25 -0700
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:17 +0000
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:16 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 14:36 +0100
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 14:25 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 15:55 +0100
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:22 +0000
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-12 08:26 -0700
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:51 +0000
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 13:06 +0200
Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 19:30 +0000
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 12:39 +0200
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-12 08:44 +0200
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:01 +0200
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 20:41 +0200
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 20:02 +0100
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 22:13 +0200
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 13:03 -0500
Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-11 11:33 -0700
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 21:04 +0200
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 20:15 -0500
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:17 +0200
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:10 +0000
Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 19:22 -0500
Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 10:58 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 02:37 -0700
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 12:06 +0200
Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 04:00 -0700
Re: software disenchantment Cecil Bayona <cbayona@cbayona.com> - 2018-10-11 13:03 -0500
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 19:36 +0100
Re: software disenchantment Cecil Bayona <cbayona@cbayona.com> - 2018-10-11 15:37 -0500
Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-10 14:01 -0700
Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 09:02 +1300
Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-10 19:11 +0000
Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-10 13:08 -0700
Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-11 14:52 +0000
Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 14:58 +0000
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 17:19 +0200
Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-12 09:49 +0000
Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-13 09:30 +0200
Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 08:53 +0000
Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-15 09:44 +0000
Re: software disenchantment Jens Stuckelberger <Jens_Stuckelberger@nowhere.net> - 2018-10-10 13:36 +0000
Re: software disenchantment "Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid> - 2018-10-10 13:51 -0700
Re: software disenchantment nobody@example.org (Scott) - 2018-10-11 01:53 +0000
Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-11 10:20 +0000
Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 12:53 +0100
Re: software disenchantment "Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid> - 2018-10-11 23:18 -0700
csiph-web