Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #381741 > unrolled thread

Build system proposal

Back to article view | Back to comp.lang.c

Contents

  Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-04 15:39 +0000
    Re: Build system proposal bart <bc@freeuk.com> - 2024-02-04 15:58 +0000
      Re: Build system proposal scott@slp53.sl.home (Scott Lurndal) - 2024-02-04 16:58 +0000
        Re: Build system proposal bart <bc@freeuk.com> - 2024-02-04 17:15 +0000
          Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-04 17:40 +0000
        Re: Build system proposal Ben Bacarisse <ben.usenet@bsb.me.uk> - 2024-02-04 23:32 +0000
          Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-04 23:58 +0000
            Re: Build system proposal Ben Bacarisse <ben.usenet@bsb.me.uk> - 2024-02-05 14:42 +0000
              Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-05 15:17 +0000
                Re: Build system proposal bart <bc@freeuk.com> - 2024-02-05 16:00 +0000
                  Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-05 16:51 +0000
                    Re: Build system proposal bart <bc@freeuk.com> - 2024-02-05 17:39 +0000
                      Re: Build system proposal "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2024-02-05 11:13 -0800
                      Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-05 23:41 +0000
                  Re: Build system proposal Michael S <already5chosen@yahoo.com> - 2024-02-05 19:14 +0200
                Re: Build system proposal David Brown <david.brown@hesbynett.no> - 2024-02-05 20:47 +0100
                  Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-05 21:19 +0000
                    Re: Build system proposal Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-02-05 13:38 -0800
                    Re: Build system proposal David Brown <david.brown@hesbynett.no> - 2024-02-06 11:10 +0100
                  Re: Build system proposal Michael S <already5chosen@yahoo.com> - 2024-02-05 23:26 +0200
                  Re: Build system proposal bart <bc@freeuk.com> - 2024-02-05 21:28 +0000
                    Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-05 23:38 +0000
                      Re: Build system proposal David Brown <david.brown@hesbynett.no> - 2024-02-06 12:56 +0100
                    Re: Build system proposal David Brown <david.brown@hesbynett.no> - 2024-02-06 12:47 +0100
                Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-05 23:34 +0000
                  Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-06 01:41 +0000
                    Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-06 02:27 +0000
                      Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-06 02:59 +0000
                        Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-06 03:09 +0000
                          Re: Build system proposal "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2024-02-05 22:33 -0800
                      Re: Build system proposal "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2024-02-05 22:32 -0800
                        Re: Build system proposal Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-02-06 07:25 +0000
                          Re: Build system proposal "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2024-02-05 23:33 -0800
                            Re: Build system proposal Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-02-06 08:20 +0000
              Re: Build system proposal Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-02-05 08:19 -0800
          Re: Build system proposal Tim Rentsch <tr.17687@z991.linuxsc.com> - 2024-02-04 23:24 -0800
    Re: Build system proposal scott@slp53.sl.home (Scott Lurndal) - 2024-02-04 16:32 +0000

Page 1 of 2  [1] 2  Next page →

#381741 — Build system proposal

How shall we write a better build system?

Current systems use scripts with ad hoc, weird syntax. They don't work 
consistently across platforms, they break and are hard to fix, and are 
complicated to set up. You even get get one scripting system as a front 
end to another, a sort of admission of failure.

But can we do better?

A scripting language provides loops, conditional branches, and 
subroutine calls. And we've already got a perfecty good language for 
describing that. It is C itself.

So here's the proposal. We implement a C interpreter. Then we provide a 
library of facilities - obviously globbing and system calls, maybe also 
installed program detection, as C functions called in the normal way. 
And the resulting C program is the build script. Because the C languague 
is well known and standardised, there's no unfamiliar syntax or 
unexpected behaviour.

We've got the skills to do this. Bart writes compilers, and so can 
easily modify one to run as an interpreter. And we can develop the 
library using normal C compilers. When we're happy with it, it is folded 
into the interpreter. A clean, closed, achieveable, but very flexible 
design.

It will need to run on Linux, Mac and Windows at a minimum. And 
hopefully build on all three with the same script and no platform 
conditional code.

-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [next] | [standalone]

#381745

On 04/02/2024 15:39, Malcolm McLean wrote:
> How shall we write a better build system?
> 
> Current systems use scripts with ad hoc, weird syntax. They don't work 
> consistently across platforms, they break and are hard to fix, and are 
> complicated to set up. You even get get one scripting system as a front 
> end to another, a sort of admission of failure.
> 
> But can we do better?
> 
> A scripting language provides loops, conditional branches, and 
> subroutine calls. And we've already got a perfecty good language for 
> describing that. It is C itself.
> 
> So here's the proposal. We implement a C interpreter.

Do we need an interpreter? We need to assume there is a C compiler 
otherwise we won't get very far with the build!

If the concern is the time it takes to compile the script, it should be 
a small proportion of that needed to build the app. I assume the script 
is only built once.

C for scripting builds has been done before. I think Thiago Adams did so.

The Seed7 build process had its own version of an auto-config script 
which was written in C, not Bash (but building the app still uses 
makefiles, nearly 20 of them for assorted compilers).

[toc] | [prev] | [next] | [standalone]

#381748

bart <bc@freeuk.com> writes:
>On 04/02/2024 15:39, Malcolm McLean wrote:
>> How shall we write a better build system?
>> 
>> Current systems use scripts with ad hoc, weird syntax. They don't work 
>> consistently across platforms, they break and are hard to fix, and are 
>> complicated to set up. You even get get one scripting system as a front 
>> end to another, a sort of admission of failure.
>> 
>> But can we do better?
>> 
>> A scripting language provides loops, conditional branches, and 
>> subroutine calls. And we've already got a perfecty good language for 
>> describing that. It is C itself.
>> 
>> So here's the proposal. We implement a C interpreter.

This was tried.  It's called the c-shell.

[toc] | [prev] | [next] | [standalone]

#381749

On 04/02/2024 16:58, Scott Lurndal wrote:
> bart <bc@freeuk.com> writes:
>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>> How shall we write a better build system?
>>>
>>> Current systems use scripts with ad hoc, weird syntax. They don't work
>>> consistently across platforms, they break and are hard to fix, and are
>>> complicated to set up. You even get get one scripting system as a front
>>> end to another, a sort of admission of failure.
>>>
>>> But can we do better?
>>>
>>> A scripting language provides loops, conditional branches, and
>>> subroutine calls. And we've already got a perfecty good language for
>>> describing that. It is C itself.
>>>
>>> So here's the proposal. We implement a C interpreter.
> 
> This was tried.  It's called the c-shell.

This is not a shell program. And it's not proposing using a language 
that vaguely looks like C, but /is/ C. I don't think the proposal is to 
ever type it live via a REPL, but always from a file.

Although as I understand it, what is proposed could be done right now: 
anyone can provide a small C program to orchestrate the building of the 
main program.

So perhaps it will be more of a library.

But it will take some care that it doesn't end up doing exactly what 
makefiles do (with lots of assumptions about C compilers and the same 
failure points), but written as a sequence of C function calls.

I think the main objection will be that it is a two step process. This 
may have been the motivation to use an interpreter, but that is too huge 
of a task, and too big a dependency, to solve a problem that could be 
achieved with shebang lines.

[toc] | [prev] | [next] | [standalone]

#381753

On 04/02/2024 17:15, bart wrote:
> On 04/02/2024 16:58, Scott Lurndal wrote:
>> bart <bc@freeuk.com> writes:
>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>> How shall we write a better build system?
>>>>
>>>> Current systems use scripts with ad hoc, weird syntax. They don't work
>>>> consistently across platforms, they break and are hard to fix, and are
>>>> complicated to set up. You even get get one scripting system as a front
>>>> end to another, a sort of admission of failure.
>>>>
>>>> But can we do better?
>>>>
>>>> A scripting language provides loops, conditional branches, and
>>>> subroutine calls. And we've already got a perfecty good language for
>>>> describing that. It is C itself.
>>>>
>>>> So here's the proposal. We implement a C interpreter.
>>
>> This was tried.  It's called the c-shell.
> 
> This is not a shell program. And it's not proposing using a language 
> that vaguely looks like C, but /is/ C. I don't think the proposal is to 
> ever type it live via a REPL, but always from a file.
> 
> Although as I understand it, what is proposed could be done right now: 
> anyone can provide a small C program to orchestrate the building of the 
> main program.
> 
> So perhaps it will be more of a library.
> 
> But it will take some care that it doesn't end up doing exactly what 
> makefiles do (with lots of assumptions about C compilers and the same 
> failure points), but written as a sequence of C function calls.
> 
> I think the main objection will be that it is a two step process. This 
> may have been the motivation to use an interpreter, but that is too huge 
> of a task, and too big a dependency, to solve a problem that could be 
> achieved with shebang lines.
 >
tcc has an interpeter mode. So yes, you could put a tcc shebang and 
achieve the same thing. But then the library wouldn't be integrated.
-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [prev] | [next] | [standalone]

#381774

scott@slp53.sl.home (Scott Lurndal) writes:

> bart <bc@freeuk.com> writes:
>>On 04/02/2024 15:39, Malcolm McLean wrote:
>>> How shall we write a better build system?
>>> 
>>> Current systems use scripts with ad hoc, weird syntax. They don't work 
>>> consistently across platforms, they break and are hard to fix, and are 
>>> complicated to set up. You even get get one scripting system as a front 
>>> end to another, a sort of admission of failure.
>>> 
>>> But can we do better?
>>> 
>>> A scripting language provides loops, conditional branches, and 
>>> subroutine calls. And we've already got a perfecty good language for 
>>> describing that. It is C itself.
>>> 
>>> So here's the proposal. We implement a C interpreter.
>
> This was tried.  It's called the c-shell.

csh isn't a C interpreter by any stretch of the imagination!

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#381776

On 04/02/2024 23:32, Ben Bacarisse wrote:
> scott@slp53.sl.home (Scott Lurndal) writes:
> 
>> bart <bc@freeuk.com> writes:
>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>> How shall we write a better build system?
>>>>
>>>> Current systems use scripts with ad hoc, weird syntax. They don't work
>>>> consistently across platforms, they break and are hard to fix, and are
>>>> complicated to set up. You even get get one scripting system as a front
>>>> end to another, a sort of admission of failure.
>>>>
>>>> But can we do better?
>>>>
>>>> A scripting language provides loops, conditional branches, and
>>>> subroutine calls. And we've already got a perfecty good language for
>>>> describing that. It is C itself.
>>>>
>>>> So here's the proposal. We implement a C interpreter.
>>
>> This was tried.  It's called the c-shell.
> 
> csh isn't a C interpreter by any stretch of the imagination!
> 

No. But the concept of a C interpreter isn't new. There's one called 
pico C which is open source and available to be incorporated. And a 
scripting language is an obvious application for an intepreter.
-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [prev] | [next] | [standalone]

#381813

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On 04/02/2024 23:32, Ben Bacarisse wrote:
>> scott@slp53.sl.home (Scott Lurndal) writes:
>> 
>>> bart <bc@freeuk.com> writes:
>>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>>> How shall we write a better build system?
>>>>>
>>>>> Current systems use scripts with ad hoc, weird syntax. They don't work
>>>>> consistently across platforms, they break and are hard to fix, and are
>>>>> complicated to set up. You even get get one scripting system as a front
>>>>> end to another, a sort of admission of failure.
>>>>>
>>>>> But can we do better?
>>>>>
>>>>> A scripting language provides loops, conditional branches, and
>>>>> subroutine calls. And we've already got a perfecty good language for
>>>>> describing that. It is C itself.
>>>>>
>>>>> So here's the proposal. We implement a C interpreter.
>>>
>>> This was tried.  It's called the c-shell.
>>
>> csh isn't a C interpreter by any stretch of the imagination! 
>
> No. But the concept of a C interpreter isn't new.

Indeed.  But C is not a good fit for interpretation, so there are often
compromises.

> There's one called pico C

Yes, and there's tcc's -run option.

> which is open source and available to be incorporated.

Likewise tcc.  So what's wrong (in your opinion) with these two that
caused you to suggest implementing another C interpreter?

> And a scripting
> language is an obvious application for an intepreter.

The term "scripting language" is so vague as to be almost useless, but
in this context the suggestion seems to be to use C as a language to
invoke the commands required to compile and link a C program.  But C is
very poorly suited to this task.  One would end up writing a library of
functions to do string manipulation and (unless 'system' was deemed
sufficient) program execution.  C itself would bring very little to the
party and the resulting scripts would be hard to read.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#381819

On 05/02/2024 14:42, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> 
>> On 04/02/2024 23:32, Ben Bacarisse wrote:
>>> scott@slp53.sl.home (Scott Lurndal) writes:
>>>
>>>> bart <bc@freeuk.com> writes:
>>>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>>>> How shall we write a better build system?
>>>>>>
>>>>>> Current systems use scripts with ad hoc, weird syntax. They don't work
>>>>>> consistently across platforms, they break and are hard to fix, and are
>>>>>> complicated to set up. You even get get one scripting system as a front
>>>>>> end to another, a sort of admission of failure.
>>>>>>
>>>>>> But can we do better?
>>>>>>
>>>>>> A scripting language provides loops, conditional branches, and
>>>>>> subroutine calls. And we've already got a perfecty good language for
>>>>>> describing that. It is C itself.
>>>>>>
>>>>>> So here's the proposal. We implement a C interpreter.
>>>>
>>>> This was tried.  It's called the c-shell.
>>>
>>> csh isn't a C interpreter by any stretch of the imagination!
>>
>> No. But the concept of a C interpreter isn't new.
> 
> Indeed.  But C is not a good fit for interpretation, so there are often
> compromises.
> 
It's less of a problem now. The interpreter can just spin up a virtual 
machine and then create a memory space. That was less of a practical 
approach when machines were more limited.
>> There's one called pico C
> 
> Yes, and there's tcc's -run option.
> 
>> which is open source and available to be incorporated.
> 
> Likewise tcc.  So what's wrong (in your opinion) with these two that
> caused you to suggest implementing another C interpreter?
> 
Bart writes compilers. So I assumed, wrongly as it turned out, that it 
would be very simple and attractive to him to modify one to be an 
interpreter. The system would be core C language, plus a library 
callable as C functions with facilities such as "glob". Now the obvious 
way to implement that is to write the library in C or some other 
language, the fold it into the interpreter. tcc might provide an 
alternative way of achieving the same thing, but I haven't looked into 
it. To fold the library into the interpreter you need the interpreter 
source and to be comfortable with it.
>> And a scripting
>> language is an obvious application for an intepreter.
> 
> The term "scripting language" is so vague as to be almost useless, but
> in this context the suggestion seems to be to use C as a language to
> invoke the commands required to compile and link a C program.  But C is
> very poorly suited to this task.  One would end up writing a library of
> functions to do string manipulation and (unless 'system' was deemed
> sufficient) program execution.  C itself would bring very little to the
> party and the resulting scripts would be hard to read.
> 
I'd dispute "very poorly". Of course you can devise a language which is 
better suited to building programs specifically. CMake takes exactly 
that approach. So what happens? It's effectively another programming 
language to learn. Someone else wrote elaborate CMakes scripts which I 
use at work. Sometimes thing go wrong. And then I'm messing about with a 
language I hardly ever use and does things in ways I am unfamilar with, 
trying to troubleshoot. It would be easier if the CMake scripts were in 
C. It might be bit less convenient. But it's just looping and branching 
and calling subroutines at the end of the day. As you know full well, 
that's all computers can do.

The library would have facilities for working with lists of strings, of 
course. But C can do that perfectly well, and I don't see it as big problem.

-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [prev] | [next] | [standalone]

#381822

On 05/02/2024 15:17, Malcolm McLean wrote:
> On 05/02/2024 14:42, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>
>>> On 04/02/2024 23:32, Ben Bacarisse wrote:
>>>> scott@slp53.sl.home (Scott Lurndal) writes:
>>>>
>>>>> bart <bc@freeuk.com> writes:
>>>>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>>>>> How shall we write a better build system?
>>>>>>>
>>>>>>> Current systems use scripts with ad hoc, weird syntax. They don't 
>>>>>>> work
>>>>>>> consistently across platforms, they break and are hard to fix, 
>>>>>>> and are
>>>>>>> complicated to set up. You even get get one scripting system as a 
>>>>>>> front
>>>>>>> end to another, a sort of admission of failure.
>>>>>>>
>>>>>>> But can we do better?
>>>>>>>
>>>>>>> A scripting language provides loops, conditional branches, and
>>>>>>> subroutine calls. And we've already got a perfecty good language for
>>>>>>> describing that. It is C itself.
>>>>>>>
>>>>>>> So here's the proposal. We implement a C interpreter.
>>>>>
>>>>> This was tried.  It's called the c-shell.
>>>>
>>>> csh isn't a C interpreter by any stretch of the imagination!
>>>
>>> No. But the concept of a C interpreter isn't new.
>>
>> Indeed.  But C is not a good fit for interpretation, so there are often
>> compromises.
>>
> It's less of a problem now. The interpreter can just spin up a virtual 
> machine and then create a memory space. That was less of a practical 
> approach when machines were more limited.
>>> There's one called pico C
>>
>> Yes, and there's tcc's -run option.
>>
>>> which is open source and available to be incorporated.
>>
>> Likewise tcc.  So what's wrong (in your opinion) with these two that
>> caused you to suggest implementing another C interpreter?
>>
> Bart writes compilers. So I assumed, wrongly as it turned out, that it 
> would be very simple and attractive to him to modify one to be an 
> interpreter.

It's actually quite hard to write an interpreter for a statically typed 
language, which has to call FFI routines written in native code, which 
include call-back functions that will call back into your program; they 
will expect a native code function!

It's extra hard when the language is C, because then it is impossible to 
draw a line between routines in language X which is being interpreted, 
and language Y which is native code on the other side of the FFI.

Since X and Y are both C.

However, I do have an interpreter for a statically typed IL. To 
interpret C code, I need to change the C compiler backend to generate 
that IL. (And also tweak that IL because it uses a 64-bit execution 
core, but C requires a mixed 32/64-bit core.)

So it's still quite a bit of work.

But I'm still unclear as to why you need a C interpreter, rather than 
just using whatever C compiler is provided.

If you provide an interpreter, what form would it take; a binary? If it 
can be binary, then why not a binary of the app?

And if you provide source code for the interpreter, theb building the 
interpreter might be as much of a job as building the main app!

[toc] | [prev] | [next] | [standalone]

#381828

On 05/02/2024 16:00, bart wrote:
> On 05/02/2024 15:17, Malcolm McLean wrote:
>> On 05/02/2024 14:42, Ben Bacarisse wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>
>>>> On 04/02/2024 23:32, Ben Bacarisse wrote:
>>>>> scott@slp53.sl.home (Scott Lurndal) writes:
>>>>>
>>>>>> bart <bc@freeuk.com> writes:
>>>>>>> On 04/02/2024 15:39, Malcolm McLean wrote:
>>>>>>>> How shall we write a better build system?
>>>>>>>>
>>>>>>>> Current systems use scripts with ad hoc, weird syntax. They 
>>>>>>>> don't work
>>>>>>>> consistently across platforms, they break and are hard to fix, 
>>>>>>>> and are
>>>>>>>> complicated to set up. You even get get one scripting system as 
>>>>>>>> a front
>>>>>>>> end to another, a sort of admission of failure.
>>>>>>>>
>>>>>>>> But can we do better?
>>>>>>>>
>>>>>>>> A scripting language provides loops, conditional branches, and
>>>>>>>> subroutine calls. And we've already got a perfecty good language 
>>>>>>>> for
>>>>>>>> describing that. It is C itself.
>>>>>>>>
>>>>>>>> So here's the proposal. We implement a C interpreter.
>>>>>>
>>>>>> This was tried.  It's called the c-shell.
>>>>>
>>>>> csh isn't a C interpreter by any stretch of the imagination!
>>>>
>>>> No. But the concept of a C interpreter isn't new.
>>>
>>> Indeed.  But C is not a good fit for interpretation, so there are often
>>> compromises.
>>>
>> It's less of a problem now. The interpreter can just spin up a virtual 
>> machine and then create a memory space. That was less of a practical 
>> approach when machines were more limited.
>>>> There's one called pico C
>>>
>>> Yes, and there's tcc's -run option.
>>>
>>>> which is open source and available to be incorporated.
>>>
>>> Likewise tcc.  So what's wrong (in your opinion) with these two that
>>> caused you to suggest implementing another C interpreter?
>>>
>> Bart writes compilers. So I assumed, wrongly as it turned out, that it 
>> would be very simple and attractive to him to modify one to be an 
>> interpreter.
> 
> It's actually quite hard to write an interpreter for a statically typed 
> language, which has to call FFI routines written in native code, which 
> include call-back functions that will call back into your program; they 
> will expect a native code function!
> 
> It's extra hard when the language is C, because then it is impossible to 
> draw a line between routines in language X which is being interpreted, 
> and language Y which is native code on the other side of the FFI.
> 
> Since X and Y are both C.
> 
> However, I do have an interpreter for a statically typed IL. To 
> interpret C code, I need to change the C compiler backend to generate 
> that IL. (And also tweak that IL because it uses a 64-bit execution 
> core, but C requires a mixed 32/64-bit core.)
> 
> So it's still quite a bit of work.
> 
> But I'm still unclear as to why you need a C interpreter, rather than 
> just using whatever C compiler is provided.
> 
> If you provide an interpreter, what form would it take; a binary? If it 
> can be binary, then why not a binary of the app?
> 
> And if you provide source code for the interpreter, theb building the 
> interpreter might be as much of a job as building the main app!
> 
> 
The way if would work is that there would be a program called 
"fixedmake", or whatever, which was distributed as an executable exactly 
like make. The someone writes program source, packages it up in a zip 
file, and instead of a makefile, he puts a "fixedmake" file in the 
source directory. However the fixedmake file is formally a C program. 
Maybe we could have the convention that it is called buildme.c.

The user types fixedmake buildme.c. fixedmake is effectively a C 
intepreter. buildme.c executes, calls C compilers and other things as 
externals, and the program is built.

Now because buildme.c is a C program, we could use other approaches. You 
could go gcc buildme.c -lfixedmake.lib, prodcue an a.out, and run a.out. 
But the snag is that it's meant to be a build system. And launching gcc 
is in and of itself a build. So it's not really automated any more.

Or we could say that since fixedmake is really just a C intepreter, we 
don't have to be ones to write that. Just use tcc. But the problem is 
that whilst buildme.c will uusally be fairly short and simple, to keep 
it short and simple there will need to a rich library with facilites for 
getting lisrs of files, launching compilers, and so on. So if we use 
tcc, we've got to package this library as well as buildme.c and the 
actual sources in the distribution.

However tcc is open source, so we might be able to modify tcc.

Before the system has any sort of traction, you can't assume that the 
fixedmake executable will be available, however. So people are going to 
have to provide it with the distribution. And if you provide as binary, 
why not just provide the program as binary? And if you provide as source 
and build with make, why not build the program with make? But I don't 
see any way round that for any new build system.

-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [prev] | [next] | [standalone]

#381836

On 05/02/2024 16:51, Malcolm McLean wrote:
> On 05/02/2024 16:00, bart wrote:

>> And if you provide source code for the interpreter, theb building the 
>> interpreter might be as much of a job as building the main app!
>>
>>
> The way if would work is that there would be a program called 
> "fixedmake", or whatever, which was distributed as an executable exactly 
> like make. The someone writes program source, packages it up in a zip 
> file, and instead of a makefile, he puts a "fixedmake" file in the 
> source directory. However the fixedmake file is formally a C program. 
> Maybe we could have the convention that it is called buildme.c.
> 
> The user types fixedmake buildme.c. fixedmake is effectively a C 
> intepreter. buildme.c executes, calls C compilers and other things as 
> externals, and the program is built.
> 
> Now because buildme.c is a C program, we could use other approaches. You 
> could go gcc buildme.c -lfixedmake.lib, prodcue an a.out, and run a.out. 
> But the snag is that it's meant to be a build system. And launching gcc 
> is in and of itself a build. So it's not really automated any more.
> 
> Or we could say that since fixedmake is really just a C intepreter, we 
> don't have to be ones to write that. Just use tcc. But the problem is 
> that whilst buildme.c will uusally be fairly short and simple, to keep 
> it short and simple there will need to a rich library with facilites for 
> getting lisrs of files, launching compilers, and so on. So if we use 
> tcc, we've got to package this library as well as buildme.c and the 
> actual sources in the distribution.
> 
> However tcc is open source, so we might be able to modify tcc.
> 
> Before the system has any sort of traction, you can't assume that the 
> fixedmake executable will be available, however. So people are going to 
> have to provide it with the distribution. And if you provide as binary, 
> why not just provide the program as binary? And if you provide as source 
> and build with make, why not build the program with make? But I don't 
> see any way round that for any new build system.


One problem as I see is that the build system doesn't know which C 
compiler is available, or which one the client wishes to use.

The client will know that, so can simply supply that information. 
However it is needed in two places:

(1) The compiler used to build build.c (as it's called in the demo
     below).

(2) The compiler used by build.exe to build the app.

(1) is easy, the client is just told to use their prefered compiler here:

    tcc build.c

But the next step is harder: build.exe won't know what was typed in step 
(1). Even if they do 'tcc -run build.c', 'tcc' does not appear as 
'args[0]', it will be 'build.c' (not even build.exe).


Here is a mock-up anyway of a build system:

1) The demo app used has three C files cipher.c, hmac.c,sha2.c

2) There is a program called build.c

3) The data describing the project is in a file called here
    filelist.txt, although it contains C source code.

Files (2) and (3) are shown below. It runs like this, using tcc's run 
option:

     c:\c>tcc -run build.c
     Compiler = tcc
     Invoking compiler:tcc -o cipher.exe cipher.c hmac.c sha2.c
     Finished building: cipher.exe

build.c defaults to using tcc. For use with another compiler, the 
process looks like this:

     c:\c>gcc build.c -o build

     c:\c>build gcc
     Compiler = gcc
     Invoking compiler:gcc -o cipher.exe cipher.c hmac.c sha2.c
     Finished building: cipher.exe

It's a litle untidier. However you can always provide a makefile to shut 
people up. The difference here is:

* The makefile only contains the handful of lines needed to implement
   the above

* The process can be easily be done manually if needed

* The critical build info, the list of files, is inside the file
   filelist.txt. While it is C, it can still be presented in a readable
   manner. It can even be incorporated into the user's own programs.

The info in filelist.txt is simplistic. It also needs the locations of 
any special headers, the libraries to use and and so on. This is all 
info added to the final invocation.

Ideally build.c, knowing the name of the compiler, can select the right 
forms of any options needed.



--------------------------------------------------------------
filelist.txt
--------------------------------------------------------------
//#define exeext = ".exe"

char* files[] = {
  "cipher.c",
  "hmac.c",
  "sha2.c"
};

int nfiles = sizeof(files)/sizeof(files[0]);

char* projectname = "cipher";
char* exefilename = "cipher.exe";
--------------------------------------------------------------

--------------------------------------------------------------
build.c
--------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "filelist.txt"

char* compiler;

void error(char* mess, char* param){
     printf("Build error: %s %s\n",mess, param);
     exit(1);
}

void compileproject(void) {
     int n, length, i;
     char* cmdstr;

     length=0;
     for (i=0; i<nfiles; ++i) length+=strlen(files[i])+1;
     cmdstr=malloc(length+100);

     strcpy(cmdstr, compiler);
     strcat(cmdstr, " ");

     strcat(cmdstr, "-o ");
     strcat(cmdstr, exefilename);
     strcat(cmdstr, " ");

     for (i=0; i<nfiles; ++i) {
         strcat(cmdstr, files[i]);
         strcat(cmdstr, " ");
     }

     printf("Invoking compiler:");
     puts(cmdstr);
     if (system(cmdstr)!=0)
         error("Error building:", exefilename);
     else
         printf("Finished building: %s\n", exefilename);
}

int main(int n, char** args) {

     if (n>=2)
         compiler=strdup(args[1]);
     else
         compiler="tcc";

     printf("Compiler = %s\n", compiler);

     compileproject();
}
--------------------------------------------------------------

[toc] | [prev] | [next] | [standalone]

#381843

On 2/5/2024 9:39 AM, bart wrote:
> On 05/02/2024 16:51, Malcolm McLean wrote:
>> On 05/02/2024 16:00, bart wrote:
[...]
>      printf("Invoking compiler:");
>      puts(cmdstr);
>      if (system(cmdstr)!=0)
>          error("Error building:", exefilename);
>      else
>          printf("Finished building: %s\n", exefilename);
> }
[...]

For some reason this is reminding me of how one compiles and links GLSL 
shader programs on the fly wrt OpenGL.

[toc] | [prev] | [next] | [standalone]

#381872

On Mon, 5 Feb 2024 17:39:25 +0000, bart wrote:

>      strcat(cmdstr, "-o ");
>      strcat(cmdstr, exefilename);
>      strcat(cmdstr, " ");
> 
>      for (i=0; i<nfiles; ++i) {
>          strcat(cmdstr, files[i]);
>          strcat(cmdstr, " ");
>      }
> 
>      ...
>      if (system(cmdstr)!=0)

What is wrong with this code?

[toc] | [prev] | [next] | [standalone]

#381831

On Mon, 5 Feb 2024 16:00:43 +0000
bart <bc@freeuk.com> wrote:
> 
> But I'm still unclear as to why you need a C interpreter, rather than 
> just using whatever C compiler is provided.
>

One reason could be that when build system's "meta" code (as opposed to
bulk of project's code) has a bug, especially out of bound array
access, I'd rather want it to fail gracefully, with meaningful error
indication, something that typical compiled C environment unable to
provide.

[toc] | [prev] | [next] | [standalone]

#381847

On 05/02/2024 16:17, Malcolm McLean wrote:
> On 05/02/2024 14:42, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>

>> The term "scripting language" is so vague as to be almost useless, but
>> in this context the suggestion seems to be to use C as a language to
>> invoke the commands required to compile and link a C program.  But C is
>> very poorly suited to this task.  One would end up writing a library of
>> functions to do string manipulation and (unless 'system' was deemed
>> sufficient) program execution.  C itself would bring very little to the
>> party and the resulting scripts would be hard to read.
>>
> I'd dispute "very poorly". Of course you can devise a language which is 
> better suited to building programs specifically. CMake takes exactly 
> that approach. So what happens? It's effectively another programming 
> language to learn. Someone else wrote elaborate CMakes scripts which I 
> use at work. Sometimes thing go wrong. And then I'm messing about with a 
> language I hardly ever use and does things in ways I am unfamilar with, 
> trying to troubleshoot. It would be easier if the CMake scripts were in 
> C. It might be bit less convenient. But it's just looping and branching 
> and calling subroutines at the end of the day. As you know full well, 
> that's all computers can do.
> 
> The library would have facilities for working with lists of strings, of 
> course. But C can do that perfectly well, and I don't see it as big 
> problem.
> 

Have you ever tried /any/ other programming language?  Apart from 
assembly and Forth, I have not seen a language that is more cumbersome 
for string handling than C.  Yes, you /can/ do string handling in C, but 
no one would /choose/ C for that.

It's not unreasonable to suggest that you'd rather base your build 
system on an existing mainstream language than a domain-specific 
language (though DSL's have the big advantage of having a syntax and 
semantics tuned to the task in question).  But why C?

Here's a quick challenge for you.

I have a directory "src".  I want to find all the .c files in "src".  I 
want to make a list of all these files, and a list of matching object 
files to make in the "build/obj/src" directory.  For each file, I want 
to call "gcc -c src/file.c -o build/obj/src/file.o".  I want to do so in 
parallel, up to 8 commands at a time.  (Ignore any possible runtime errors.)


With make, that would be (this is untested) :

	default : all
	.PHONY all
	srcdir = "src"
	objdir = "build/obj"

	srcfiles = $(wildcard $(srcdir)/*.c)
	objfiles_src = $(srcfiles:.c=.o)
	objfiles = $(addprefix $(objdir)/, \
		$(patsubst ../%,%,$(objfiles_src)))

	all : objfiles
	
	$(objdir)/src :
         	mkdir -p $@
	
	$(objdir)/%.o : %.c | $(objdir)/src
	        gcc -c $< -o $@

Run with "make -j 8".


With Python, you could have (untested) :

	import glob
	import os
	import multiprocessing
	import subprocess

	srcdir = "src"
	objdir = "build/obj"

	srcfiles = glob.glob(srcdir + "/*.c")
	objfiles_src = [fn[:-2] + ".o" for fn in srcfiles]
	objfiles = [objdir + "/" + fn for fn in srcfiles]
	
	os.mkdirs(objdir + "/" + srcdir, exist_ok = True)

	with multiprocessing.Pool(5) as pool :
		pool.map(lambda src, obj : subprocess.call("gcc -c " +
				 src + " -o " + obj, shell = True),
				zip(srcfiles, objfiles))

Obviously much of this could be put in a reusable library, so that end 
users don't have to know about process pools.  (And there are many other 
ways to structure such code.)


Your task is to duplicate this in C.

[toc] | [prev] | [next] | [standalone]

#381850

On 05/02/2024 19:47, David Brown wrote:
> On 05/02/2024 16:17, Malcolm McLean wrote:
>> On 05/02/2024 14:42, Ben Bacarisse wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>
> 
>>> The term "scripting language" is so vague as to be almost useless, but
>>> in this context the suggestion seems to be to use C as a language to
>>> invoke the commands required to compile and link a C program.  But C is
>>> very poorly suited to this task.  One would end up writing a library of
>>> functions to do string manipulation and (unless 'system' was deemed
>>> sufficient) program execution.  C itself would bring very little to the
>>> party and the resulting scripts would be hard to read.
>>>
>> I'd dispute "very poorly". Of course you can devise a language which 
>> is better suited to building programs specifically. CMake takes 
>> exactly that approach. So what happens? It's effectively another 
>> programming language to learn. Someone else wrote elaborate CMakes 
>> scripts which I use at work. Sometimes thing go wrong. And then I'm 
>> messing about with a language I hardly ever use and does things in 
>> ways I am unfamilar with, trying to troubleshoot. It would be easier 
>> if the CMake scripts were in C. It might be bit less convenient. But 
>> it's just looping and branching and calling subroutines at the end of 
>> the day. As you know full well, that's all computers can do.
>>
>> The library would have facilities for working with lists of strings, 
>> of course. But C can do that perfectly well, and I don't see it as big 
>> problem.
>>
> 
> Have you ever tried /any/ other programming language?  Apart from 
> assembly and Forth, I have not seen a language that is more cumbersome 
> for string handling than C.  Yes, you /can/ do string handling in C, but 
> no one would /choose/ C for that.
> 
> It's not unreasonable to suggest that you'd rather base your build 
> system on an existing mainstream language than a domain-specific 
> language (though DSL's have the big advantage of having a syntax and 
> semantics tuned to the task in question).  But why C?
> 
The newgroup is comp.lang.c. So if we propose using a generally accepted 
and widely understood programming language as our build scripting 
language, the choice of language has to be C.

> Here's a quick challenge for you.
> 
> I have a directory "src".  I want to find all the .c files in "src".  I 
> want to make a list of all these files, and a list of matching object 
> files to make in the "build/obj/src" directory.  For each file, I want 
> to call "gcc -c src/file.c -o build/obj/src/file.o".  I want to do so in 
> parallel, up to 8 commands at a time.  (Ignore any possible runtime 
> errors.)
> 
> 
> With make, that would be (this is untested) :
> 
>      default : all
>      .PHONY all
>      srcdir = "src"
>      objdir = "build/obj"
> 
>      srcfiles = $(wildcard $(srcdir)/*.c)
>      objfiles_src = $(srcfiles:.c=.o)
>      objfiles = $(addprefix $(objdir)/, \
>          $(patsubst ../%,%,$(objfiles_src)))
> 
>      all : objfiles
> 
>      $(objdir)/src :
>              mkdir -p $@
> 
>      $(objdir)/%.o : %.c | $(objdir)/src
>              gcc -c $< -o $@
> 
> Run with "make -j 8".
> 
> 
> With Python, you could have (untested) :
> 
>      import glob
>      import os
>      import multiprocessing
>      import subprocess
> 
>      srcdir = "src"
>      objdir = "build/obj"
> 
>      srcfiles = glob.glob(srcdir + "/*.c")
>      objfiles_src = [fn[:-2] + ".o" for fn in srcfiles]
>      objfiles = [objdir + "/" + fn for fn in srcfiles]
> 
>      os.mkdirs(objdir + "/" + srcdir, exist_ok = True)
> 
>      with multiprocessing.Pool(5) as pool :
>          pool.map(lambda src, obj : subprocess.call("gcc -c " +
>                   src + " -o " + obj, shell = True),
>                  zip(srcfiles, objfiles))
> 
> Obviously much of this could be put in a reusable library, so that end 
> users don't have to know about process pools.  (And there are many other 
> ways to structure such code.)
> 
> 
> Your task is to duplicate this in C.
> 
> 

#include "builduils.h"

int main(void)
{
    STRINGLIST *sources;
    STRINGLIST *objfiles;
    int i;


    sources = glob("src/*.c");
    objfiles = stringlist();
    for (i = 0; i < stringlist_Nstrings(sources); i++)
    {
       char objname[1024];
       snprintf(objname, 1024, "%s/%s", objdir, 
replaceextension(stringlist_get(sources, i), ".o"));
       stringlist_add(objfiles, objname);
    }
    char outputdir[1024];
    snprinf("%s/%s", 1024, objdir, srcdir);
    mkdir(outputdir);

    callparallel("gcc -c %s %s", srcfiles, objfiles);
}

OK now I've cheated a little bit by inventing buildutils library 
functions ad hoc. But I haven't done so outrageously. And we're leaking 
memory. But likely we have many gigabytres, so who are about maybe 2K 
for a few strings. And yes, maybe we could reoplace snprintf() with 
something that creates an arbitrary length output.

The Python is a bit shorter and more concise. But there's not that much 
in it.
-- 
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

[toc] | [prev] | [next] | [standalone]

#381856

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> On 05/02/2024 19:47, David Brown wrote:
[...]
>> Have you ever tried /any/ other programming language?  Apart from 
>> assembly and Forth, I have not seen a language that is more
>> cumbersome for string handling than C.  Yes, you /can/ do string
>> handling in C, but no one would /choose/ C for that.
>> It's not unreasonable to suggest that you'd rather base your build 
>> system on an existing mainstream language than a domain-specific
>> language (though DSL's have the big advantage of having a syntax and 
>> semantics tuned to the task in question).  But why C?
>> 
> The newgroup is comp.lang.c. So if we propose using a generally
> accepted and widely understood programming language as our build
> scripting language, the choice of language has to be C.

Right -- *if* we want to discuss it here in comp.lang.c.

If C is not a suitable language for a new build systems, the solution is
to discuss it somewhere else, not to discuss it here and pretend C is
the best answer.

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#381910

On 05/02/2024 22:19, Malcolm McLean wrote:
> On 05/02/2024 19:47, David Brown wrote:
>> On 05/02/2024 16:17, Malcolm McLean wrote:
>>> On 05/02/2024 14:42, Ben Bacarisse wrote:
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>
>>
>>>> The term "scripting language" is so vague as to be almost useless, but
>>>> in this context the suggestion seems to be to use C as a language to
>>>> invoke the commands required to compile and link a C program.  But C is
>>>> very poorly suited to this task.  One would end up writing a library of
>>>> functions to do string manipulation and (unless 'system' was deemed
>>>> sufficient) program execution.  C itself would bring very little to the
>>>> party and the resulting scripts would be hard to read.
>>>>
>>> I'd dispute "very poorly". Of course you can devise a language which 
>>> is better suited to building programs specifically. CMake takes 
>>> exactly that approach. So what happens? It's effectively another 
>>> programming language to learn. Someone else wrote elaborate CMakes 
>>> scripts which I use at work. Sometimes thing go wrong. And then I'm 
>>> messing about with a language I hardly ever use and does things in 
>>> ways I am unfamilar with, trying to troubleshoot. It would be easier 
>>> if the CMake scripts were in C. It might be bit less convenient. But 
>>> it's just looping and branching and calling subroutines at the end of 
>>> the day. As you know full well, that's all computers can do.
>>>
>>> The library would have facilities for working with lists of strings, 
>>> of course. But C can do that perfectly well, and I don't see it as 
>>> big problem.
>>>
>>
>> Have you ever tried /any/ other programming language?  Apart from 
>> assembly and Forth, I have not seen a language that is more cumbersome 
>> for string handling than C.  Yes, you /can/ do string handling in C, 
>> but no one would /choose/ C for that.
>>
>> It's not unreasonable to suggest that you'd rather base your build 
>> system on an existing mainstream language than a domain-specific 
>> language (though DSL's have the big advantage of having a syntax and 
>> semantics tuned to the task in question).  But why C?
>>
> The newgroup is comp.lang.c. So if we propose using a generally accepted 
> and widely understood programming language as our build scripting 
> language, the choice of language has to be C.

No, it does not.  It is perfectly reasonable to say "not C".

> 
>> Here's a quick challenge for you.
>>
>> I have a directory "src".  I want to find all the .c files in "src".  
>> I want to make a list of all these files, and a list of matching 
>> object files to make in the "build/obj/src" directory.  For each file, 
>> I want to call "gcc -c src/file.c -o build/obj/src/file.o".  I want to 
>> do so in parallel, up to 8 commands at a time.  (Ignore any possible 
>> runtime errors.)
>>
>>
>> With make, that would be (this is untested) :
>>
>>      default : all
>>      .PHONY all
>>      srcdir = "src"
>>      objdir = "build/obj"
>>
>>      srcfiles = $(wildcard $(srcdir)/*.c)
>>      objfiles_src = $(srcfiles:.c=.o)
>>      objfiles = $(addprefix $(objdir)/, \
>>          $(patsubst ../%,%,$(objfiles_src)))
>>
>>      all : objfiles
>>
>>      $(objdir)/src :
>>              mkdir -p $@
>>
>>      $(objdir)/%.o : %.c | $(objdir)/src
>>              gcc -c $< -o $@
>>
>> Run with "make -j 8".
>>
>>
>> With Python, you could have (untested) :
>>
>>      import glob
>>      import os
>>      import multiprocessing
>>      import subprocess
>>
>>      srcdir = "src"
>>      objdir = "build/obj"
>>
>>      srcfiles = glob.glob(srcdir + "/*.c")
>>      objfiles_src = [fn[:-2] + ".o" for fn in srcfiles]
>>      objfiles = [objdir + "/" + fn for fn in srcfiles]
>>
>>      os.mkdirs(objdir + "/" + srcdir, exist_ok = True)
>>
>>      with multiprocessing.Pool(5) as pool :
>>          pool.map(lambda src, obj : subprocess.call("gcc -c " +
>>                   src + " -o " + obj, shell = True),
>>                  zip(srcfiles, objfiles))
>>
>> Obviously much of this could be put in a reusable library, so that end 
>> users don't have to know about process pools.  (And there are many 
>> other ways to structure such code.)
>>
>>
>> Your task is to duplicate this in C.
>>
>>
> 
> #include "builduils.h"
> 
> int main(void)
> {
>     STRINGLIST *sources;
>     STRINGLIST *objfiles;
>     int i;
> 
> 
>     sources = glob("src/*.c");
>     objfiles = stringlist();
>     for (i = 0; i < stringlist_Nstrings(sources); i++)
>     {
>        char objname[1024];
>        snprintf(objname, 1024, "%s/%s", objdir, 
> replaceextension(stringlist_get(sources, i), ".o"));
>        stringlist_add(objfiles, objname);
>     }
>     char outputdir[1024];
>     snprinf("%s/%s", 1024, objdir, srcdir);
>     mkdir(outputdir);
> 
>     callparallel("gcc -c %s %s", srcfiles, objfiles);
> }
> 
> OK now I've cheated a little bit by inventing buildutils library 
> functions ad hoc.

Yes.  Of course a practical solution would split the common code into a 
library, and not expect people to re-write that all the time.  But a key 
point here is that in make and in Python - both well-established 
languages and tools - this is already in place.  But you are skipping 
the hard bits.  (Good luck implementing "callparallel" with a sane 
interface that includes the call you have made here!)

We know it is /possible/ to do this kind of thing in C.  We also know it 
is ugly and fiddly compared to other languages that handle strings and 
lists as first-class types (or at least have them as classes).  Even 
when you assume the existence of a library with every type and function 
you want, you still need a lot of extra boilerplate code, you have 
artificial limits, memory leaks, etc.


> But I haven't done so outrageously. And we're leaking 
> memory. But likely we have many gigabytres, so who are about maybe 2K 
> for a few strings. And yes, maybe we could reoplace snprintf() with 
> something that creates an arbitrary length output.
> 
> The Python is a bit shorter and more concise. But there's not that much 
> in it.

[toc] | [prev] | [next] | [standalone]

#381852

On Mon, 5 Feb 2024 20:47:53 +0100
David Brown <david.brown@hesbynett.no> wrote:
> 
> Have you ever tried /any/ other programming language?  Apart from 
> assembly and Forth, I have not seen a language that is more
> cumbersome for string handling than C.
>

In order of my exposure to them: Pascal, Fortran, Ada.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.c

csiph-web