Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #8177

Re: Incremental build systems, infamake

Cross-posted to 2 groups.

Show all headers | View raw

On 07/25/11 09:11 PM, Joshua Maurice wrote:
> On Jul 25, 2:00 am, Ian Collins<ian-n...@hotmail.com>  wrote:
>> On 07/25/11 07:25 PM, Joshua Maurice wrote:
>>
>>> An incrementally correct build system - is a build system which can
>>> only produce incrementally correct builds. That is, there is no source
>>> code changes which a developer may do, inadvertently or purposefully,
>>> which can result in the build system doing an incrementally incorrect
>>> build.
>>
>>> The key part to realize about this requirement is that the build
>>> system scripts, such as makefiles, are themselves source code under
>>> this model. At least, I claim that they ought to be considered as
>>> source code for the purposes of determining if your incremental build
>>> system is incrementally correct.
>>
>>> Now, at this point, there is a blurring of lines between build system
>>> tool and build system. When the makefiles are both source code and
>>> build system, it becomes hard to distinguish between them. I'm merely
>>> trying to say that I want a tool in which it is exceptionally hard, if
>>> not impossible short of maliciousness, to "break" the build system
>>> such that it might produce an incrementally incorrect build.
>>
>> Makefiles aren't source code and build system, the make utility is the
>> build system.
>
> Makefiles are source code insofaras incremental correctness is
> concerned. make, the executable, in a sense, doesn't distinguish
> between cpp source files and makefiles. They're both input to the
> executable. Both are regularly modified by developers. Both get
> checked into source control. In terms of build correctness, I see no
> reason to distinguish between them.

Sorry I wasn't clear: makefiles are source, the make utility is the 
build system.

>>> Now, why do I want this? I hate waiting for hours each day doing a
>>> build. Yes, I know that the source code base in question has other
>>> problems if it takes that long to build, but unfortunately I am not
>>> the king of the company, and so I have to make due. Besides, even
>>> spending 10 minutes on a full clean rebuild is 10 minutes
>>> (potentially) wasted, if another better solution exists. And even more
>>> importantly, 4 hours spent tracking down a bug which was simply the
>>> result of a broken incremental build is 4 hours wasted.
>>
>> Wouldn't the time spent working on a build tool be better spent cleaning
>> up the code and the existing system?
>
> Perhaps. If the cost is amortized across a bunch of people, then no.
> If someone can just take a tool, plug it in with minimal effort, and
> get a lot of bang for it, then it would be worth it.

But the someone could still spend their time fixing the current mess.

>>> I claim that it's not that hard. However, as far as I can tell, very
>>> few people have attempted such a thing. In the C++ world at least,
>>> most ended with something that was "good enough" that caught 95% of
>>> the cases. A good example AFAIK is the Recursive Make Considered
>>> Harmful solution, found here:
>>>       http://miller.emu.id.au/pmiller/books/rmch/
>>
>> Flat makefiles rule!
>
> Heh. Are you saying only 1 makefile? 1 makefile vs a bunch of
> makefiles with GNU make include isn't that big of a difference, if at
> all.

So long as the make utility can schedule the building of all the file in 
the tree, then yes.

>>> It's good, but it still misses cases. Depending on the exact details
>>> of implementation, it won't do a rebuild if you change compiler or
>>> linker options, such as a new preprocessor define. It won't relink an
>>> executable if you delete one of its associated cpp source files. It
>>> won't recompile an object file if you introduce a new header file
>>> which hides a previously included header file on the include path.
>>
>> Good old make will cover all of those cases.  All you need is a make
>> that maintains state.
>
> It will? I admit - if you hack the bejesus out of it as compared to
> traditional solutions you find online, such as the Recursive Make
> Considered Harmful solution, then yes. I would be greatly surprised if
> anyone but a remote fraction of actual build systems in use would
> actually do a recompile of an object file if you added a new header
> file which hid a previously included header file on the include path.

Well Sun (d)make certainly does.

If I have a rule

x.o: x.cc
	CC x.cc -o x -I b -I a

and a header in a, adding a header with the same name to b will trigger 
a rebuild.

>> I work for myself, so wasted time is my money wasted.  I've found the
>> most effective way to minimise build times is to design the code and
>> makefile(s) to build fast.  That normally means more, smaller files and
>> a single flat makefile.  These combine to get the best performance out
>> of a distributed or parallel make.
>>
>> If the build still isn't fast enough, I just throw more hardware at it.
>
> So, do you always do a full clean rebuild on every change?

No, I trust dmake.

> Presumably
> no. Are you a linux guy, or windows guy, e.g. visual studios devenv?

Mainly Solaris.

> If you're a Linux guy, I presume you use a text editor of some kind,
> possibly with some cool integration with the build tools, so that you
> can trigger of some sort of incremental build after doing some
> changes. That is, you don't do a full clean rebuild every time. I bet
> further in fact that when working on a cpp file, you just recompile
> that cpp file until you get it working and done, and only then do you
> do the full build - maybe from clean, maybe an incremental.

No, I run an incremental build after every change in order to run the 
unit tests (make for me is "make test").  That's the main reason I put 
so much effort into optimising my builds.

> Here's the
> point where I want an incrementally correct build. I want to know if I
> broke the other team's code. Sure - it would be nice if I had well
> documented, well defined stable interfaces in order to minimize the
> impact of a change and hence the time of a full clean rebuild, but
> well I don't control the coding process of my company unfortunately.

Trust your tools.  I use the same tools as the OS developers.  If they 
are good enough for them (and more to the point, their QA!), they are 
good enough for me...

-- 
Ian Collins

Back to comp.lang.c++ | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 00:25 -0700
  Re: Incremental build systems, infamake Ian Collins <ian-news@hotmail.com> - 2011-07-25 21:00 +1200
    Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 02:11 -0700
      Re: Incremental build systems, infamake Ian Collins <ian-news@hotmail.com> - 2011-07-25 21:33 +1200
        Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 02:38 -0700
          Re: Incremental build systems, infamake Ian Collins <ian-news@hotmail.com> - 2011-07-25 21:55 +1200
          Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 03:04 -0700
            Re: Incremental build systems, infamake Ian Collins <ian-news@hotmail.com> - 2011-07-25 22:22 +1200
              Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 03:25 -0700
                Re: Incremental build systems, infamake Ian Collins <ian-news@hotmail.com> - 2011-07-25 23:41 +1200
            Re: Incremental build systems, infamake Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-07-26 16:11 +0000
              Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-26 15:19 -0700
                Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-26 15:28 -0700
                Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-26 15:49 -0700
  Re: Incremental build systems, infamake Nobody <nobody@nowhere.com> - 2011-07-25 12:52 +0100
    Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-25 13:34 -0700
      Re: Incremental build systems, infamake Nobody <nobody@nowhere.com> - 2011-07-27 19:49 +0100
        Re: Incremental build systems, infamake Joshua Maurice <joshuamaurice@gmail.com> - 2011-07-29 03:58 -0700
  Re: Incremental build systems, infamake Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-07-26 15:04 +0000

csiph-web