Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #136688

Re: software disenchantment

From David Brown <david.brown@hesbynett.no>
Newsgroups comp.lang.c
Subject Re: software disenchantment
Date 2018-10-11 21:04 +0200
Organization A noiseless patient Spider
Message-ID <ppo6s0$mog$1@dont-email.me> (permalink)
References (17 earlier) <32vvD.21117$134.7822@fx03.am4> <ppn5kf$clr$1@dont-email.me> <RVFvD.15651$I34.1016@fx43.am4> <ppneji$ke$1@dont-email.me> <ppo38r$jr7$1@news.albasani.net>

Show all headers | View raw

On 11/10/18 20:03, BGB wrote:
> On 10/11/2018 7:10 AM, David Brown wrote:
>> On 11/10/18 12:50, Bart wrote:
>>> On 11/10/2018 10:37, David Brown wrote:
>>>> On 11/10/18 00:28, Bart wrote:
>>>
>>>>> Would 2500 object files need to be accessed in the course of even an
>>>>> incremental build? Or is it clever enough to patch in only the ones 
>>>>> that
>>>>> have changed?
>>>
>>>> So for any incremental build, usually only a small fraction (typically
>>>> just one) of the object files get re-built.
>>>>
>>>> If you have incremental linking, then much of the previous link results
>>>> are also kept - if not, then you may have to read all the object files
>>>> for the link.
>>>
>>> I said 2500 OBJECT files.
>>
>> Yes, I know you said that.
>>
>>> So that already assumes that only some source
>>> files are re-compiled.
>>
>> No, it does not imply this.  But that does not matter to my point.
>>
>> It is not clear what you mean by "would 2500 object files need to be
>> accessed" - accessed in what way?  I took that to mean "read" - if you
>> had meant "written" or "generated", I think you would have said that.
>> The only reason you read object files in a typical build process is for
>> the linking.
>>
>> It is common for the link to use all the object files in one link
>> command.  But if you have a bigger project, you might use incremental
>> linking - then you don't need to read all the object files for each
>> link.  (Link-time optimisation is also possible, but more complex.)
>>
> 
> At least with MSVC, I have often disabled incremental linking (on by 
> default), as it tends to reduce performance and occasionally "fouls up" 
> in ways which require manually deleting the output files (the produced 
> binary is broken; or sometimes "link.exe" will just keep crashing until 
> the output files are deleted, *).

Don't judge a general technique from MS' incompetence at implementing it.

> 
> *: Though, usually if "link.exe" crashes, I have found it is easiest to 
> just delete the output so it can start clean (whether or not it crashes 
> again, most of the time the binary is broken).
> 
> Not tried it with GCC or similar, so don't really know how it compares.
> 
> 
>>>
>>>> Some timings for a project with 113 object files are:
>>>>
>>>> Serial "clean" build: 51s
>>>> Parallel "clean" build: 12s
>>>> Incremental build for one C file changed: 1.8s
>>>>
>>>> Does that explain why people use incremental builds?
>>>
>>> Incremental builds and multi-processing. It's what you have to throw at
>>> a problem when the basic methods are slow and inefficient.
>>>
>>
>> No, it is what you use to make everything go much faster.  It does not
>> matter how fast your compilations are - with big enough projects,
>> "compile everything, one file at a time" will be slow in the end.  If
>> your super compiler takes a mere 50 ms to compile a file, then with 2500
>> files it takes 2 minutes.  A parallel clean build on an 8-core PC will
>> take 15 seconds.  An incremental build will take 50 ms.
>>
>> Serial build-everything-everytime builds are fine for toy projects - but
>> not for more serious work, /regardless/ of the compiler speed.
>>
> 
> IME, "always rebuild everything" works up to a certain scale.

Yes, it does.  Incremental builds make a bigger difference the bigger 
the project.

However, for anything other than a simple test or "hello, world", you 
normally want some control of the build - names, compiler switches, 
libraries, etc.  You want control of directories so that your source 
directories are not jumbled with object files and other generated files. 
  You need somewhere to hold this information - the build information 
and "project management" bit.  Some people like to do this with an IDE, 
others with a build system like "make".  (I like to use "make" for the 
build itself, but have an IDE setup to make the development smoother.)

And once you have that, your incremental build is automatic.  I have 
incremental builds for even the smallest of projects with no more than a 
dozen C files - it is easier than /not/ having it.

(I don't bother with incremental links - my projects are not big enough 
for that to be worth the effort setting it up.)

> 
> Coarse grained incremental works well, like rebuilding individual DLL's 
> or SO's, or building code as chunks of ~ 25kLOC or so, but still letting 
> make skip over stuff that doesn't change on the larger scale.
> 
> So, for example, for a fairly typical ~ 100 kLOC DLL, it could be 
> organized into roughly 4 TU's (rather than compiling ~ 50 individual C 
> or C++ files).
> 
> 
> If the program is a single binary, particularly if modest size, "just 
> rebuild directly" works pretty well (and the build can be done as a "one 
> liner" in a BAT or SH file).
> 

<snip>
(I'm snipping a lot here.  I have read it, but if I try to reply to 
points in it we'll mostly re-cover old ground, or continue endlessly.)

> 
> There is the questionable aspect of my compiler that it goes through a 
> stack-based IL stage (functionally similar to .NET CIL in terms of how 
> it behaves; though how metadata is handled is quite different), but this 
> has turned out to still be useful (I can use it for things like the C 
> library and similar).

Stack-based intermediary languages are common in compilers (with SSA 
being the other common style).

<snip>

> One drawback IMO with GCC is that it tries to do too much with a single 
> compiler, leading it to be quite big and unwieldy.
> 
> Several smaller and more specialized compilers could possibly better 
> address some things, with the main GCC mostly focusing on PC-like 
> targets; probably with other compilers toolchains for various classes of 
> embedded and microcontroller targets.
> 

gcc /does/ focus on a certain class of processor - what might be termed 
"C friendly" processors.  This covers a large number of devices, but 
misses many classes entirely - DSP's, 8-bit accumulator processors, 
anything with CHAR_BIT != 8 or signed integers that are not two's 
complement.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

software disenchantment Bart <bc@freeuk.com> - 2018-10-06 15:13 +0100
  Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-06 09:54 -0700
  Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-06 10:26 -0700
    Re: software disenchantment fir <profesor.fir@gmail.com> - 2018-10-14 04:26 -0700
  Re: software disenchantment antispam@math.uni.wroc.pl - 2018-10-09 18:20 +0000
    Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-09 11:54 -0700
      Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-09 21:22 +0100
        Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-09 17:47 -0700
          Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 11:28 +0100
            Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 11:38 +0100
              Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-10 05:49 -0700
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-10 15:47 +0200
                Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 09:58 -0400
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-10 14:39 +0000
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-10 09:42 -0700
                Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 13:18 -0400
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 20:31 +0100
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 17:53 -0500
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 10:06 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 02:34 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 12:03 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 03:59 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 16:56 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 09:02 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 19:39 +0200
                Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 19:53 +0000
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-16 03:54 -0500
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 20:25 +0200
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:16 +0000
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 07:15 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 17:05 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 09:15 -0700
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 17:33 +0000
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 19:53 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 17:59 +0100
                Re: software disenchantment nobody@example.org (Scott) - 2018-10-10 16:53 +0000
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 12:45 -0500
                Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-10 11:04 -0700
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 14:22 -0500
                Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 15:53 -0400
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 09:06 +1300
                Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-10 13:21 -0700
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-10 20:27 +0000
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 22:54 +0100
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:01 +1300
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:05 +1300
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 23:16 +0100
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:22 +1300
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-10 23:28 +0100
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 11:33 +1300
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 11:37 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 11:50 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 14:10 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 05:24 -0700
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 15:12 +0200
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:13 +0000
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 14:44 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 16:38 +0200
                Re: software disenchantment mark.bluemel@gmail.com - 2018-10-11 08:09 -0700
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 17:43 +0100
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 17:36 +0000
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 18:56 +0100
                Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 11:54 -0700
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 20:09 +0100
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:10 +0000
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 12:57 +0200
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 14:27 -0500
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 22:03 +0200
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 15:56 -0500
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 11:30 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 23:34 +0100
                Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 16:24 -0700
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 01:46 +0100
                Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-11 18:58 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-12 08:50 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 11:12 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 12:55 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 12:10 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:29 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 13:06 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 14:30 +0200
                Re: software disenchantment James Kuyper <jameskuyper@alumni.caltech.edu> - 2018-10-12 08:53 -0400
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 15:11 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 13:55 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 15:13 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 16:10 +0100
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:22 +0000
                Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-12 09:12 -0700
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 13:00 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 12:52 +0100
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-14 08:44 +1300
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 21:03 +0100
                Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-14 10:13 +1300
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-13 23:02 +0100
                Re: software disenchantment Keith Thompson <kst-u@mib.org> - 2018-10-13 16:47 -0700
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-14 01:05 +0100
                Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-13 18:12 -0700
                Re: software disenchantment Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-10-14 03:05 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-14 20:14 +0200
                Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-14 11:25 -0700
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:17 +0000
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 13:16 +0000
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 14:36 +0100
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 14:25 +0000
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-12 15:55 +0100
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:22 +0000
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-12 08:26 -0700
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-12 15:51 +0000
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-13 13:06 +0200
                Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 19:30 +0000
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 12:39 +0200
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-12 08:44 +0200
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:01 +0200
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 20:41 +0200
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 20:02 +0100
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 22:13 +0200
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 13:03 -0500
                Re: software disenchantment "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2018-10-11 11:33 -0700
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 21:04 +0200
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-11 20:15 -0500
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-12 13:17 +0200
                Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 13:10 +0000
                Re: software disenchantment BGB <cr88192@hotmail.com> - 2018-10-10 19:22 -0500
                Re: software disenchantment David Brown <david.brown@hesbynett.no> - 2018-10-11 10:58 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 02:37 -0700
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 12:06 +0200
                Re: software disenchantment Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-10-11 04:00 -0700
                Re: software disenchantment Cecil Bayona <cbayona@cbayona.com> - 2018-10-11 13:03 -0500
                Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 19:36 +0100
                Re: software disenchantment Cecil Bayona <cbayona@cbayona.com> - 2018-10-11 15:37 -0500
                Re: software disenchantment Thiago Adams <thiago.adams@gmail.com> - 2018-10-10 14:01 -0700
        Re: software disenchantment Ian Collins <ian-news@hotmail.com> - 2018-10-11 09:02 +1300
      Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-10 19:11 +0000
        Re: software disenchantment David Kleinecke <dkleinecke@gmail.com> - 2018-10-10 13:08 -0700
          Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-11 14:52 +0000
            Re: software disenchantment scott@slp53.sl.home (Scott Lurndal) - 2018-10-11 14:58 +0000
            Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-11 17:19 +0200
              Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-12 09:49 +0000
                Re: software disenchantment Wouter Verhelst <w@uter.be> - 2018-10-13 09:30 +0200
                Re: software disenchantment Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-10-15 08:53 +0000
                Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-15 09:44 +0000
    Re: software disenchantment Jens Stuckelberger <Jens_Stuckelberger@nowhere.net> - 2018-10-10 13:36 +0000
  Re: software disenchantment "Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid> - 2018-10-10 13:51 -0700
    Re: software disenchantment nobody@example.org (Scott) - 2018-10-11 01:53 +0000
    Re: software disenchantment Dan Purgert <dan@djph.net> - 2018-10-11 10:20 +0000
    Re: software disenchantment Bart <bc@freeuk.com> - 2018-10-11 12:53 +0100
      Re: software disenchantment "Chris M. Thomasson" <invalid_chris_thomasson@invalid.invalid> - 2018-10-11 23:18 -0700

csiph-web