Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.misc > #3913
| From | James Harris <james.harris.1@gmail.com> |
|---|---|
| Newsgroups | comp.lang.misc |
| Subject | Re: Compiling and linking code from different places in the file system |
| Date | 2018-03-15 20:37 +0000 |
| Organization | A noiseless patient Spider |
| Message-ID | <p8eli7$2c4$1@dont-email.me> (permalink) |
| References | (5 earlier) <p8388p$bn8$2@gioia.aioe.org> <p86vs7$iq$1@dont-email.me> <p8act5$aj4$2@gioia.aioe.org> <p8b0df$n3d$1@dont-email.me> <p8dhql$1c8t$1@gioia.aioe.org> |
On 15/03/2018 10:29, Rod Pemberton wrote:
> On Wed, 14 Mar 2018 11:18:02 +0000
> James Harris <james.harris.1@gmail.com> wrote:
>
>> On 14/03/2018 05:47, Rod Pemberton wrote:
>>> On Mon, 12 Mar 2018 22:44:22 +0000
>>> James Harris <james.harris.1@gmail.com> wrote:
>
> <snip>
>
>> What I have in mind to deal with this problem:
>>
>> Imagine a project were to be distributed as source files. It would
>> come with configuration files which tell the "build system" what it
>> needs to know about the source files, things like
>>
>> * what word size or sizes each requires
>
> Can't these be generated from parsing the code?
No, because the code could be in any language and the build system
wouldn't know how to interpret all languages.
>
> E.g., many years ago I created a mainframe program that would read a
> database file of any format, and allow you to display the records
> appropriate to the specific database. The structure for the database
> records was needed by the display program, but that couldn't be
> statically compiled in via an include file, because the app needed to
> work for multiple databases with different formats and different
> includes. So, I used an AWK script to parse each include file, and emit
> a file with the structure in a format that display program could use,
> dynamically, for each database, respectively. Then, you just started
> the display app, passed it the name of the database to display, and
> also the name of the file with the reformatted database structure for
> the respective database.
>
>> * what CPU modes or architectures each requires
>
> Should the code (or coder) decide this? What if the mode or
> architecture isn't available?
Yes, I have in mind that the coder of the source files would write the
configs to describe those source files. I haven't worked out the
specifics yet but to make an example, a config file could look like this
drivers/*.nasm: bits=32 cpu=386-pmode
system/*: bits=32..
The first line says that all the .nasm files in the drivers/ folder are
written for 32-bit object code and for a certain CPU mode.
The second line would say that the files in system/ can work with 32-bit
or greater word sizes. (The .. here means "or greater".)
The idea is as follows.
1. If the build system were told to make 32-bit PMode objects then it
would build files matching both of these.
drivers/*.nasm
system/*
2. But if it were told to make 32-bit Arm objects then it would create
them only from
system/*
(There would presumably be another folder which supplied the relevant
Arm-system drivers.)
>
>> * any other limitations of each source file
>
> Wouldn't limitations go against the portable code concept? ;-)
You put a smiley so I guess you are joking, but one limitation might be
that portable source code would need a certain minimum integer size such
as ints which have to be at least 32 bits wide (but could be larger). An
alternative limitation could be that the code would work on a certain
architecture of machine - such as an IBM AT compatible.
>
>> They would come with the project source files.
>>
>> By contrast, the user would have a configuration file which says
>> which compilers or assemblers to use for the kinds of attributes
>> which can be specified about the source files.
>
> Wouldn't each compiler or assembler already have their own
> required format? E.g., I can't envision a Linux makefile working with
> Turbo C, nor vice-versa. I.e., how do you intend to emit a
> compiler native makefile from your build script? ...
I don't intend to emit a makefile. :-)
The idea is that the build system would invoke compilers and other
programs as needed.
>
>> Therefore, the only
>> commands which would be runnable would be those the user chooses.
>
> Well, total user control is good.
>
> So, I'm guessing that you're attempting to separate the architecture
> specifics from the code specifics.
>
> Is the compiling distributed across multiple platforms too? If so,
> then in theory, the appropriate architecture can always be used to
> compile the code for it.
I was thinking more about cross compiling: one platform having compilers
for different targets.
>
>> Yes, I am looking to be able to pre-build object files for /many/
>> targets.
>
> Initial thoughts are:
>
> a) multiple directories, each unique to a specific platform
> b) cloud computing with distributed compiling upon the correct
> architecture for the code
The great thing about having config files is the person who writes the
source can organise it as he or she chooses. For example, a config file
could specify each source file individually.
drivers/kb_16.nasm: bits=16
drivers/kb_32.nasm: bits=32
drivers/kb_64.nasm: bits=64
drivers/kbc_16.nasm: bits=16
drivers/kbc_32.nasm: bits=32
drivers/kbc_64.nasm: bits=64
But that would be a pain so the programmer could choose to organise the
source tree in a better way, making the config file simpler.
drivers_x16/*: bits=16
drivers_x32/*: bits=32
drivers_x64/*: bits=64
Then drivers_x16 would include all the 16-bit drivers. And so on. The
programmer could choose how to lay out the source tree.
>
>> Yes, I am looking to be able to pre-build object files for /many/
>> targets. Imagine a folder T containing some assembly source files
>> such as Nasm sources for x86-32 code. There would still be many types
>> of object files which could be generated therefrom.
>>
>> If a certain source file were named
>>
>> src/T/S.nasm
>>
>> it could be assembled to become these object files:
>>
>> obj/T/i1/S.nasm.o
>> obj/T/i2/S.nasm.o
>> obj/T/i3/S.nasm.o
>> obj/T/i4/S.nasm.o
>> obj/T/i5/S.nasm.o
>
> Is this one-command, multiple build? ...
The user would run one command but that command would run multiple
compile commands...!
>
> Wouldn't it be easier to have a room with multiple computers, each
> running a daemon or other process awaiting commands. You could
> multicast out many of the same commands to all machines:
Goodness! That's all a bit elaborate. Tomorrow the world, eh! In fact,
the mechanics of how it would be run aren't important just now. I'm only
looking at a way to control what compile commands are run.
>
> 1) ftp the source packages from control machine
> 2) unzip/untar the packages
> 3) configure and make
>
> Obviously, some machines might have different configure and make
> processes. So, some machines would need to translate locally.
>
>> and so on. Each of those object files would have been derived from
>> the same original file, S.nasm.
>>
>> Naturally, there could be multiple source files in the folder src/T.
>> Each of them would be assembled to the subfolders in obj/T. Each of
>> the iN subfolders would contain the output of the assembler relating
>> to a package of measures such as
>>
>> * object file type (ELF32, COFF, win32)
>> * calling convention (many possibilities)
>> * optimisation type (basically for space or speed)
>>
>> In other words,
>>
>> i1 could be ELF32, cdecl, unoptimised
>> i2 could be ELF32, cdecl, optimised for speed
>> i3 could be ELF32, stdcall, unoptimised
>> i4 could be ELF32, stdcall, optimised for speed
>>
>> etc.
>>
>> At the moment I cannot think of any meaningful way to specify the
>> object-type attributes individually, hence the idea to combine them
>> into a single numbered folder such as i3.
>
> What is wrong with multiple cross-compile builds on the same machine,
> but passing different command line options for each build? I.e., the
> current way this is done.
You maybe have something more elaborate in mind. I am thinking only that
the user will run one command, call it CCC, and the CCC program will
work out which other commands to run. For example, if "sym" were to be a
folder containing source code, the command might be
CCC sym
What CCC would do is
1. Look for a config file in sym which tells it what each of the source
files in sym is capable of.
2. Iterate over the source files in sym. For each one:
2a. Work out all the different object files we can derive from it[*].
2b. For each one, if it exists and is up to date then do nothing.
2c. For each which is missing or is not up to date ... build it.
[*] Working out what object files we can derive from each source file is
something I still have to come up with a mechanism for but it will
probably be down to per-user config files something like the following.
Imagine the following is (a simplified version of) a config file in the
user's directory
target x32: bits=32
cc -m32 <src> -o <obj>
target x64: bits=64
cc -m64 <src> -o <obj>
target arm_32: bits=32
gcc-arm-linux-gnueabihf -m32 <src> -o <obj>
The idea is that the CCC program mentioned earlier would see that a
certain source file can be compiled for the x64 target and then run the
target x64 command on that source file, and similar for other options.
To be clear, there are two types of config files. One is to be supplied
with the source files and describes their characteristics (such as "this
file will work with word sizes of 32-bits or more"). The other is
written by the person running the builds and says "to compile for x86-64
this is the recipe". (As with make, the indented commands are recipes
and there can be multiple commands there.)
Does that make more sense now?
>
>> Each object file could be seen as an "instance" of the source file.
>> If a source file were to change then all the instances which were
>> made from it would have to be remade.
>
> Wouldn't it be easier to generate /ONE/ "object" file which could be
> used for all machines, e.g., interpretable byte code or JIT compilation,
> etc., then distribute it to each machine to do final compilation or
> interpretation?
That could be an option. But I wouldn't want to compile *all* C and
assembly code to byte code (!) so other options are also needed.
>
>> That was rather a long way to answer your question but, yes, the idea
>> is to cross compile for multiple targets, and variants of those
>> targets.
>
> I like these ideas best:
>
> a) create one "object" file for all machines, e.g., JIT of Java byte
> code or LLVM IR or ANDF etc
> b) distributed cloud-computing compile on each specific architecture
You have in mind some things which are different from what I intend.
--
James Harris
Back to comp.lang.misc | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-01 14:40 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-01 14:55 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-02 21:05 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 01:18 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:03 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:40 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:10 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:58 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:23 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-09 02:28 -0500
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-09 02:44 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 23:26 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-11 07:44 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-12 21:59 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-14 01:45 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-14 10:59 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:40 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:20 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:58 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 10:30 +0000
Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 11:11 -0500
Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 13:51 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 09:48 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 12:25 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 15:32 +0000
Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-04 07:52 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-04 08:43 +0000
Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-05 07:58 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:37 +0000
Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-07 21:02 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-05 15:29 +0000
Re: Compiling and linking code from different places in the file system rugxulo@gmail.com - 2018-03-06 18:23 -0800
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 04:54 +0000
Re: Compiling and linking code from different places in the file system Robert Wessel <robertwessel2@yahoo.com> - 2018-03-07 00:33 -0600
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 09:55 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-07 17:29 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-06 06:39 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 10:44 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-07 06:57 -0500
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 10:28 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-11 07:44 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-12 22:44 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-14 01:47 -0400
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 10:23 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 11:30 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 11:30 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:30 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 14:32 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 16:01 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 19:28 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-15 08:59 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-15 19:21 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 08:08 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-16 11:12 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 16:45 +0100
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-16 16:09 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-16 18:42 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-17 18:32 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-17 21:37 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-17 21:00 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-18 11:39 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-18 12:29 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-19 13:38 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-20 05:40 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-20 11:05 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 06:03 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-21 18:51 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 19:55 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-23 18:03 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-23 20:13 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-24 01:30 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-24 16:48 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-26 19:42 +0100
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-26 22:35 +0100
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 12:01 +0100
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 20:58 +0100
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 20:49 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-20 18:37 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-21 19:33 +0000
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-21 20:35 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 21:10 +0100
Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 06:47 +0000
Re: Compiling and linking code from different places in the file system Andy Walker <anw@cuboid.co.uk> - 2018-03-27 00:58 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-30 19:55 +0100
Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 06:49 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-23 21:04 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-23 21:54 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 11:22 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-24 12:12 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 13:15 +0100
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-24 13:36 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-24 14:52 +0000
Re: Compiling and linking code from different places in the file system Marco van de Voort <marcov@toad.stack.nl> - 2018-04-19 11:06 +0000
Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-14 16:52 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-14 19:38 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-15 09:00 +0100
Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-15 12:54 +0000
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-14 11:18 +0000
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:52 +0100
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-15 06:29 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-15 20:37 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-17 11:51 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-20 06:31 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-20 17:14 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-21 07:10 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-24 01:38 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-24 16:10 +0000
Re: Compiling and linking code from different places in the file system Rod Pemberton <NoEmail@trraxvfeqa.prg> - 2018-03-24 20:05 -0400
Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-25 16:55 +0100
Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:39 +0100
csiph-web