Re: Compiling and linking code from different places in the file system

From	James Harris <james.harris.1@gmail.com>
Newsgroups	comp.lang.misc
Subject	Re: Compiling and linking code from different places in the file system
Date	2018-03-24 16:48 +0000
Organization	A noiseless patient Spider
Message-ID	<p95vhs$veb$1@dont-email.me> (permalink)
References	(25 earlier) <p8u9k3$1pil$1@gioia.aioe.org> <p8udbe$tne$1@dont-email.me> <p93fj3$fki$1@gioia.aioe.org> <p93n4n$ac8$1@dont-email.me> <p949o5$1oa7$1@gioia.aioe.org>

Show all headers | View raw

On 24/03/2018 01:30, Andy Walker wrote:
> On 23/03/18 20:13, James Harris wrote:
>>>      You have not explained why you want to re-invent that particular
>>> wheel.  It seems to stem from misunderstandings about "make".
>> Some problems with make.
>> 1. It is more than awkward at handling dependencies which are produced
>> on the fly. That leads to many makefiles having to split the operations
>> so you often see
>>      make depend
>>      make
>> The second command uses the dependency files produced by the first. It
>> would be better to have that stuff handled automatically.
> 
> 	Arguably;  but that would require "make" to re-determine all the
> dependencies. 

Sure. Is that bad?

> If it really bothers you, there are ways around it, esp
> in Gnu "make". 

Yes, some versions of make allow it. But such techniques are not portable.

> Personally, I prefer the simpler versions of "make", and
> to keep my own "makefile"s very simple too.  But of course people like
> to add "features" to everything they use.  The same would happen to your
> re-invention if ever it became widespread.
> 
>> 2. I normally prefer commands run to be printed only when they fail in
>> some way, [...].
> 
> 	"make -s"?

Close, but it doesn't print the commands even when they fail.

> 
>> 3. The make command is, if anything, too flexible for building
>> programs, and that leads to some people writing crazy makefiles.
> 
> 	I asked earlier what you meant by "flexible", and I'm still not
> sure I understand your point.  "Crazy" "makefile"s are mostly written
> by other tools, and you are not expected to understand them, any more
> than you are expected to understand the parser tables produced by tools
> such as Yacc and Lex.

A makefile can do almost anything you can do under your id, and some 
people who write makefiles should enter obfuscation contests. They could 
be even worse when written by a program.

Rather than that, sometimes, restrictions which enforce a dependable 
structure can be very welcome.

> 
>> 4. Finally, make is a security risk. When you run make on someone else's
>> makefile you don't always know which commands it will run under your userid.
> 
> 	"make -n";  &/or use one of the many sandboxing techniques.

I've found that -n isn't reliable for this. And rather than use 
sandboxing it would surely be easier and safer to control the commands 
which can be run.

Basically, IMO there are enough problems with make to, er, make it 
worthwhile considering a custom solution.

> 
>> That's not to have a go at make. I think it's great. But that list maybe
>> explains some of the reasons for a custom build system.
> 
> 	Well, it partially explains why you want to write your own version
> of "make".  But that seems to be a lot of work to avoid some very minor
> inconveniences [to you] of your own "makefile"s;  and it's a whole lot
> more work if you want to distribute your re-invention and expect others
> to use it [while avoiding those same security problems].

Yes.

> 
>>>> The idea is that one config file would say what restrictions or
>>>> limitations source files had (e.g. word size, endianness assumptions,
>>>> required hardware, required libraries etc)
>>>      If your code really has limitations because of such features
>>> [eg, won't work on 16-bit or big-endian machines], then there's not
>>> much more to be said, except "caveat emptor".
>> Even assembly code...?
>> Take another example, an IRQ handler. Even if written in a HLL it
>> would be specific to a certain CPU. That doesn't mean it lacks
>> portability it should have.
> 
> 	Perhaps I wasn't clear.  If your code is [eg] 32-bit specific
> [for perhaps excellent reasons], then what are *you* going to do when
> someone with a 16-bit machine wants to import your software?  OK, your
> config file says "32-bit only";  is that not "end of story"?  The only
> way out is to provide either two separate sources or one source with
> conditional 16/32-bit sections;  but in that case, "32-bit" is no longer
> a *limitation* but an option, to be selected by "configure" or near
> equivalent.  Similarly for all other "limitations";  either they are
> show-stoppers, or else they require the distributor to supply some way
> of getting both [or more] effects by selection at the importing end.

Well put. Yes, there would normally be other pieces of code for other 
target environments.

To be clear of how I see this all fitting together, the user running the 
builds would have a config file (call it ~/.pbuild.cfg) which said 
things like

* to build 16-bit x86 code from .nasm sources ...
* to build 32-bit x86 code from .nasm sources ...
* to build 64-but x86 code from .nasm sources ...

* to build 16-bit x86 code from .c sources ...
* to build 32-bit x86 code from .c sources ...
* to build 64-but x86 code from .c sources ...

where ... indicates the commands to run.

Given that, lets assume we want to compile a program which is made from 
two modules, boot and body, where boot starts up in a way to suit a 
particular environment and then calls body. So when run

   boot --> body   (i.e. boot calls body)

Say we need the program to work in three environments: x86-16, x86-32 
and x86-64.

That could be done with the following four pieces of source code

   x16/boot.nasm
   x32/boot.nasm
   x64/boot.nasm
   body.c

The idea would be that body.c would be portable but that

   xN/boot.nasm

would be N-bit assembly code.

The config file that describes the code (call it Pbuild.cfg) would say 
things like

   x16/*  bits=16 cpu=x86
   x32/*  bits=32 cpu=x86
   x64/*  bits=64 cpu=x86

If body.c were to be completely portable it would need no such 
description because it had no restrictions.

A build of that lot would produce six object files which could be named

   x86-16/x16/boot.nasm.o
   x86_32/x32/boot.nasm.o
   x86_64/x64/boot.nasm.o

   x86_16/body.c.o
   x86_32/body.c.o
   x86_64/body.c.o

To form a complete app the relevant parts would be linked together but 
linking is another topic so I won't go in to it just now.

The idea is that all such object files would be built or rebuilt 
automatically, using the info in the config files to decide what to do, 
and using the dates and dependencies to limit the work done to the 
minimum needed.

That's how I see this working. Does it all make a bit more sense now?


-- 
James Harris

Thread

Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-01 14:40 +0000
  Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-01 14:55 +0000
    Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-02 21:05 +0000
      Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 01:18 +0000
        Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:03 +0000
      Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:40 -0500
        Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:10 +0000
          Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:58 -0500
            Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:23 +0000
              Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-09 02:28 -0500
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-09 02:44 -0500
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 23:26 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-11 07:44 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-12 21:59 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-14 01:45 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-14 10:59 +0000
      Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:40 -0500
        Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:20 +0000
          Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:58 -0500
            Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 10:30 +0000
  Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 11:11 -0500
    Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 13:51 -0500
    Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 09:48 +0000
  Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 12:25 +0000
    Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 15:32 +0000
      Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-04 07:52 +0000
      Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-04 08:43 +0000
        Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-05 07:58 +0000
          Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:37 +0000
            Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-07 21:02 +0000
        Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-05 15:29 +0000
          Re: Compiling and linking code from different places in the file system rugxulo@gmail.com - 2018-03-06 18:23 -0800
          Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 04:54 +0000
            Re: Compiling and linking code from different places in the file system Robert Wessel <robertwessel2@yahoo.com> - 2018-03-07 00:33 -0600
              Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 09:55 +0000
              Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-07 17:29 +0000
  Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:39 -0500
    Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 10:44 +0000
      Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:57 -0500
        Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 10:28 +0000
          Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-11 07:44 -0400
            Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-12 22:44 +0000
              Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-14 01:47 -0400
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 10:23 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 11:30 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 11:30 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:30 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 14:32 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 16:01 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 19:28 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-15 08:59 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-15 19:21 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 08:08 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-16 11:12 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 16:45 +0100
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-16 16:09 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 18:42 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-17 18:32 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-17 21:37 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-17 21:00 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-18 11:39 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-18 12:29 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-19 13:38 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-20 05:40 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-20 11:05 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 06:03 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-21 18:51 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 19:55 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-23 18:03 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-23 20:13 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-24 01:30 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-24 16:48 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-26 19:42 +0100
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-26 22:35 +0100
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 12:01 +0100
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 20:58 +0100
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 20:49 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-20 18:37 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-21 19:33 +0000
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-21 20:35 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 21:10 +0100
                Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 06:47 +0000
                Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-27 00:58 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-30 19:55 +0100
                Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 06:49 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 21:04 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-23 21:54 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 11:22 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-24 12:12 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 13:15 +0100
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 13:36 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-24 14:52 +0000
                Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 11:06 +0000
                Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-14 16:52 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 19:38 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-15 09:00 +0100
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-15 12:54 +0000
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-14 11:18 +0000
                Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:52 +0100
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-15 06:29 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-15 20:37 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-17 11:51 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-20 06:31 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-20 17:14 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 07:10 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-24 01:38 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-24 16:10 +0000
                Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-24 20:05 -0400
                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-25 16:55 +0100
  Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:39 +0100

csiph-web