Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #127173 > unrolled thread

Compiling and linking code from different places in the file system

Back to article view | Back to comp.lang.c

Contents

  Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-01 14:40 +0000
    Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-01 14:55 +0000
      Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-02 21:05 +0000
        Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 01:18 +0000
          Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:03 +0000
        Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-02 22:37 -0800
        Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-09 01:23 +0000
    Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 11:11 -0500
      Re: Compiling and linking code from different places in the file system "James R. Kuyper" <jameskuyper@verizon.net> - 2018-03-01 13:51 -0500
        Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-01 11:02 -0800
      Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 09:48 +0000
        Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-03 04:06 -0800
          Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-03 05:24 -0800
            Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-03 06:43 -0800
      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-06 13:51 -0800
    Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-01 22:46 +0000
      Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-01 15:27 -0800
      Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-03 12:25 +0000
        Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-03 04:53 -0800
        Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-03 15:32 +0000
          Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-03 08:02 -0800
          Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-04 07:52 +0000
          Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-04 08:43 +0000
            Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-05 07:58 +0000
              Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-06 21:37 +0000
                Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-07 21:02 +0000
            Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-05 15:29 +0000
              Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 04:54 +0000
                Re: Compiling and linking code from different places in the file system Robert Wessel <robertwessel2@yahoo.com> - 2018-03-07 00:33 -0600
                  Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-07 09:55 +0000
        Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-04 07:39 +0000
          Re: Compiling and linking code from different places in the file system Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-03-07 05:00 -0800
            Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-07 14:09 +0000
              Re: Compiling and linking code from different places in the file system Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-03-07 14:26 +0000
                Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-07 16:35 +0000
                  Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-07 09:32 -0800
                    Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-07 10:11 -0800
                      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-07 11:12 -0800
                        Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-07 12:01 -0800
                          Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-07 15:43 -0800
                            Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-07 19:18 -0800
                              Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-08 09:59 -0800
                                Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 10:28 -0800
                                  Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-08 13:36 -0800
                                    Re: Compiling and linking code from different places in the file system bartc <bc@freeuk.com> - 2018-03-08 21:47 +0000
                                      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-08 15:03 -0800
                                    Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-09 09:59 +0100
                                      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-09 10:33 -0800
                                        Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-09 12:27 -0800
                                          Re: Compiling and linking code from different places in the file system scott@slp53.sl.home (Scott Lurndal) - 2018-03-09 20:48 +0000
                                            Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-09 13:31 -0800
                                              Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-10 18:31 +0100
                                                Re: Compiling and linking code from different places in the file system Robert Wessel <robertwessel2@yahoo.com> - 2018-03-10 13:22 -0600
                                              Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-10 19:15 +0000
                                          Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-09 13:28 -0800
                                            Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-09 14:13 -0800
                                              Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-09 14:45 -0800
                                        Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-10 17:39 +0100
                                          Re: Compiling and linking code from different places in the file system Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-03-10 19:00 +0000
                                          Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 08:26 -0700
                                            Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-12 17:06 +0100
                                              Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 10:23 -0700
                                                Re: Compiling and linking code from different places in the file system scott@slp53.sl.home (Scott Lurndal) - 2018-03-12 18:42 +0000
                                                  Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 12:20 -0700
                                                    Re: Compiling and linking code from different places in the file system scott@slp53.sl.home (Scott Lurndal) - 2018-03-12 20:27 +0000
                                                      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 14:12 -0700
                                                    Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-13 11:12 +0100
                                            Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-12 09:23 -0700
                                              Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 10:38 -0700
                                              Re: Compiling and linking code from different places in the file system David Kleinecke <dkleinecke@gmail.com> - 2018-03-12 11:41 -0700
                                                Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 12:10 -0700
                                      Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-09 12:09 -0800
                                        Re: Compiling and linking code from different places in the file system scott@slp53.sl.home (Scott Lurndal) - 2018-03-09 20:46 +0000
                                        Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-10 18:35 +0100
                                          Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 08:54 -0700
                                            Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-13 08:39 +0100
                                  Re: Compiling and linking code from different places in the file system Tim Rentsch <txr@alumni.caltech.edu> - 2018-03-17 11:03 -0700
                                Re: Compiling and linking code from different places in the file system James Harris <james.harris.1@gmail.com> - 2018-03-11 07:37 +0000
                                  Re: Compiling and linking code from different places in the file system David Kleinecke <dkleinecke@gmail.com> - 2018-03-11 14:22 -0700
                                    Re: Compiling and linking code from different places in the file system Spiros Bousbouras <spibou@gmail.com> - 2018-03-11 21:48 +0000
                                      Re: Compiling and linking code from different places in the file system David Kleinecke <dkleinecke@gmail.com> - 2018-03-11 17:50 -0700
                                    Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-11 15:54 -0700
                                  Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-12 09:38 -0700
                            Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-08 10:48 +0100
                              Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 09:58 -0800
                                Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-08 22:03 +0100
                                  Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 13:55 -0800
                                    Re: Compiling and linking code from different places in the file system Spiros Bousbouras <spibou@gmail.com> - 2018-03-08 22:38 +0000
                                      Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 15:03 -0800
                                        Re: Compiling and linking code from different places in the file system Spiros Bousbouras <spibou@gmail.com> - 2018-03-08 23:15 +0000
                                        Re: Compiling and linking code from different places in the file system Tim Rentsch <txr@alumni.caltech.edu> - 2018-03-17 10:44 -0700
                                          Re: Compiling and linking code from different places in the file system Steven Petruzzellis <frelwizzen@gmail.com> - 2018-03-18 03:47 -0700
                                    Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-09 10:10 +0100
                                      Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-09 10:10 -0800
                                  Re: Compiling and linking code from different places in the file system jameskuyper@verizon.net - 2018-03-08 14:21 -0800
                                    Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 15:10 -0800
                                      Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-09 10:13 +0100
                                        Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-09 10:10 -0800
                        Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-08 10:28 +0100
                          Re: Compiling and linking code from different places in the file system supercat@casperkitty.com - 2018-03-08 10:28 -0800
                            Re: Compiling and linking code from different places in the file system David Brown <david.brown@hesbynett.no> - 2018-03-08 22:17 +0100
                            Re: Compiling and linking code from different places in the file system Keith Thompson <kst-u@mib.org> - 2018-03-08 13:26 -0800
    Re: Compiling and linking code from different places in the file system "Pascal J. Bourguignon" <pjb@informatimago.com> - 2018-03-14 13:39 +0100

Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6  Next page →

#127333

On Saturday, March 3, 2018 at 8:32:42 AM UTC-7, Bart wrote:
> On 03/03/2018 12:25, James Harris wrote:
> > On 01/03/2018 22:46, Jorgen Grahn wrote:
> 
> >> No sane real project is scattered around the file system: it lives
> >> under a well-defined top-level directory.
> 
> (Aren't Linux files disseminated like that anyway?)
> 
> 
> > To be clear, the idea is to have the compile commands generated by a 
> > program.
> 
> That seems to be just moving the problem elsewhere. And having compile 
> commands, in the form of generated makefiles, seems common.
> 
> (Which is not satisfactory: what to make of all this: 
> https://pastebin.com/raw/McLRQgzS? I know there are hundreds of .c files 
> involved, but where are they in this file?)
> 
> What is the input to that program? Somewhere, something or someone needs 
> to know exactly what files are to be compiled, exactly where the include 
> files are, and exactly how to compose the various outputs into the one 
> or more executables, libraries etc that comprise the result.
> 
> > When you link to form a load module do you similarly use a project-wide 
> > set of library folders (the -L folders to a typical linker)?
> 
> These days I use a linker where /everything/ has to be specified 
> explicitly. Otherwise, various compiler systems will all do there own 
> thing, and you may or may not need to specify the language runtime, or 
> the OS libraries such as gdi32, or it might or might not need .lib or .a 
> or .dll or .so.
> 
> It seems more effort but then you always know exactly what is happening. 
> If a library can't be found, then you will know why because either you 
> forget to specify it, you gave the wrong path, or it doesn't exist. It 
> will not be a mystery.
> 
> I would not use a command like -L, which in my experience never worked 
> properly anyway.
> 
> -- 
> bartc



Audra Moore must realize that Autumn Nissen can go get whatever software he has found, right, gluey? And, of course, anyone can filter his posts, which renders _his_ flooding impotent, just like Audra Moore ;) 

But Audra Moore feels the need to mock the herd. How is wasting existing brain matter tied to love of Audra Moore in any way going to lead to enlightened usenet writing? 

--
My Snoring Solution
https://www.facebook.com/JonasEklundh
https://groups.google.com/forum/#!topic/comp.os.linux.advocacy/tzMH39QmAmU
http://www.5z8.info/warez_w2t4ge_girlsgonewildpart1.wmv
Jonas Eklundh

[toc] | [prev] | [next] | [standalone]

#127346

["Followup-To:" header set to comp.lang.c.]
On Sat, 2018-03-03, bartc wrote:
> On 03/03/2018 12:25, James Harris wrote:
>> On 01/03/2018 22:46, Jorgen Grahn wrote:
>
>>> No sane real project is scattered around the file system: it lives
>>> under a well-defined top-level directory.
>
> (Aren't Linux files disseminated like that anyway?)

We're talking about the source code for some project, so no.

When you clone a git repo, do you get one top-level directory or
several?

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#127347

On 03/03/2018 15:32, bartc wrote:
> On 03/03/2018 12:25, James Harris wrote:
>> On 01/03/2018 22:46, Jorgen Grahn wrote:
> 
>>> No sane real project is scattered around the file system: it lives
>>> under a well-defined top-level directory.
> 
> (Aren't Linux files disseminated like that anyway?)

That was Jorgen's comment rather than mine but AISI there is a 
difference. When you say "disseminated" I presume you are thinking about 
how finished apps are distributed and installed whereas here the query 
is about how source code is laid out.

> 
> 
>> To be clear, the idea is to have the compile commands generated by a
>> program.
> 
> That seems to be just moving the problem elsewhere. And having compile
> commands, in the form of generated makefiles, seems common.

The whole make infrastructure is brilliant, IMO, but it does have some 
weaknesses:

1. It is more than awkward at handling dependencies which are produced 
on the fly. That leads to many makefiles having to split the operations 
so you might see

   make depend
   make

The second command uses the dependency files produced by the first. It 
would be better to have that stuff handled automatically.

2. I would prefer the commands run to be printed only when they fail in 
some way, i.e. output like this would be handy

   command <errcode>
   stderr output

but only when the command returned a non-zero error code or wrote 
something to stderr.

3. The make command is, if anything, too flexible for building programs, 
and that leads to some crazy makefiles. Rather than have that 
flexibility in each makefile it may be better to have some universal 
rules to which limited but sufficient tweaks can be applied.

4. Finally, make is a security risk. When you run make on someone else's 
makefile you often don't know which commands it will run under your userid.

That's not to have a go at make. I think it's great. But that list maybe 
explains some of the reasons why I wanted to write a custom program 
builder.

> 
> (Which is not satisfactory: what to make of all this:
> https://pastebin.com/raw/McLRQgzS? I know there are hundreds of .c files
> involved, but where are they in this file?)
> 
> What is the input to that program? Somewhere, something or someone needs
> to know exactly what files are to be compiled, exactly where the include
> files are, and exactly how to compose the various outputs into the one
> or more executables, libraries etc that comprise the result.

That file well illustrates my points about make being too flexible and a 
security risk. As to where the .c files are mentioned, perhaps some of 
the included makefiles would give a clue. The mkrules.mk file may do 
that by scanning the directory tree based on variables which have been 
defined at the time that it is included.

> 
>> When you link to form a load module do you similarly use a project-wide
>> set of library folders (the -L folders to a typical linker)?
> 
> These days I use a linker where /everything/ has to be specified
> explicitly. 

Me too. Again, don't you think that much of that should be generatable 
by a program?

> Otherwise, various compiler systems will all do there own
> thing, and you may or may not need to specify the language runtime, or
> the OS libraries such as gdi32, or it might or might not need .lib or .a
> or .dll or .so.
> 
> It seems more effort but then you always know exactly what is happening.
> If a library can't be found, then you will know why because either you
> forget to specify it, you gave the wrong path, or it doesn't exist. It
> will not be a mystery.
> 
> I would not use a command like -L, which in my experience never worked
> properly anyway.
> 


-- 
James Harris

[toc] | [prev] | [next] | [standalone]

#127355

(Crossposting since everyone else in the thread insists on it.)

On Sun, 2018-03-04, James Harris wrote:
...
> The whole make infrastructure is brilliant, IMO, but it does have some 
> weaknesses:
>
> 1. It is more than awkward at handling dependencies which are produced 
> on the fly. That leads to many makefiles having to split the operations 
> so you might see
>
>    make depend
>    make
>
> The second command uses the dependency files produced by the first. It 
> would be better to have that stuff handled automatically.

Others thought so too, so there's a way to do it (at least if you use
gcc or clang).  See e.g. this Makefile of mine:
https://github.com/kjgrahn/groblad/blob/master/Makefile

Or

* Paul D. Smith: Auto-Dependency Generation
  http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/

> 2. I would prefer the commands run to be printed only when they fail in 
> some way, i.e. output like this would be handy
>
>    command <errcode>
>    stderr output
>
> but only when the command returned a non-zero error code or wrote 
> something to stderr.

I think you can get that, but you have to write it yourself, in the
same way that e.g. the Linux kernel Makefile just writes

  CC foo.c
  LD libfoo.a

instead of the full commands by default.

Personally I want to see what's happening, especially during
incremental builds ("what needs to rebuild after the changes I
made?").

> 3. The make command is, if anything, too flexible for building programs, 
> and that leads to some crazy makefiles.

No, incompetence and indifference leads to crazy makefiles.

> Rather than have that flexibility in each makefile it may be better
> to have some universal rules to which limited but sufficient tweaks
> can be applied.

That's not an original idea: I think that's the reason cmake, qmake
and so on exist.  You may want to investigate them.

Those also (try to) let you split your projects into independent
sub-projects.  For cmake that works reasonably well, IME.

> 4. Finally, make is a security risk. When you run make on someone else's 
> makefile you often don't know which commands it will run under your userid.

What about the thing you build -- isn't that a security risk, too?
Because I assume you want to run it eventually.

I see no way around this with any build tool.  Although you can build
in some sandbox, e.g. a Docker container on Linux.

> That's not to have a go at make. I think it's great. But that list maybe 
> explains some of the reasons why I wanted to write a custom program 
> builder.

Your goals are a bit unclear to me, so I won't comment on that.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#127407

On 05/03/2018 07:58, Jorgen Grahn wrote:
> (Crossposting since everyone else in the thread insists on it.)
> 
> On Sun, 2018-03-04, James Harris wrote:
> ...
>> The whole make infrastructure is brilliant, IMO, but it does have some
>> weaknesses:
>>
>> 1. It is more than awkward at handling dependencies which are produced
>> on the fly. That leads to many makefiles having to split the operations
>> so you might see
>>
>>     make depend
>>     make
>>
>> The second command uses the dependency files produced by the first. It
>> would be better to have that stuff handled automatically.
> 
> Others thought so too, so there's a way to do it (at least if you use
> gcc or clang).  See e.g. this Makefile of mine:
> https://github.com/kjgrahn/groblad/blob/master/Makefile
> 
> Or
> 
> * Paul D. Smith: Auto-Dependency Generation
>    http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/

Thanks. Yes, I know. I spent some hours working on this a few years ago. 
It was nightmarish. I think I tried two published solutions and one of 
my own. All had failings of one sort or another. Can't remember the 
details now but IIRC I concluded the make system is not designed to 
process dependencies after it has started executing commands. As for 
gcc, I didn't want to depend on a particular version of make or a 
particular compiler, and this was to work with an assembler as well as a 
compiler.

> 
>> 2. I would prefer the commands run to be printed only when they fail in
>> some way, i.e. output like this would be handy
>>
>>     command <errcode>
>>     stderr output
>>
>> but only when the command returned a non-zero error code or wrote
>> something to stderr.
> 
> I think you can get that, but you have to write it yourself, in the
> same way that e.g. the Linux kernel Makefile just writes
> 
>    CC foo.c
>    LD libfoo.a
> 
> instead of the full commands by default.
> 
> Personally I want to see what's happening, especially during
> incremental builds ("what needs to rebuild after the changes I
> made?").

Noted. A command-line switch could allow a choice.

> 
>> 3. The make command is, if anything, too flexible for building programs,
>> and that leads to some crazy makefiles.
> 
> No, incompetence and indifference leads to crazy makefiles.

Sure, but sensible people don't get to write all makefiles. Software can 
be distributed with makefiles which are dangerously Gordian!

> 
>> Rather than have that flexibility in each makefile it may be better
>> to have some universal rules to which limited but sufficient tweaks
>> can be applied.
> 
> That's not an original idea: I think that's the reason cmake, qmake
> and so on exist.  You may want to investigate them.
> 
> Those also (try to) let you split your projects into independent
> sub-projects.  For cmake that works reasonably well, IME.

Thanks. I'll take a look.

> 
>> 4. Finally, make is a security risk. When you run make on someone else's
>> makefile you often don't know which commands it will run under your userid.
> 
> What about the thing you build -- isn't that a security risk, too?
> Because I assume you want to run it eventually.

On Windows or Unix, yes, but they are not all OSes. A sane OS would 
execute binaries in a cocoon.

> 
> I see no way around this with any build tool.  Although you can build
> in some sandbox, e.g. a Docker container on Linux.
> 
>> That's not to have a go at make. I think it's great. But that list maybe
>> explains some of the reasons why I wanted to write a custom program
>> builder.
> 
> Your goals are a bit unclear to me, so I won't comment on that.
> 
> /Jorgen
> 


-- 
James Harris

[toc] | [prev] | [next] | [standalone]

#127453

On Tue, 2018-03-06, James Harris wrote:
> On 05/03/2018 07:58, Jorgen Grahn wrote:
>> (Crossposting since everyone else in the thread insists on it.)
>> 
>> On Sun, 2018-03-04, James Harris wrote:
>> ...
>>> The whole make infrastructure is brilliant, IMO, but it does have some
>>> weaknesses:
>>>
>>> 1. It is more than awkward at handling dependencies which are produced
>>> on the fly. That leads to many makefiles having to split the operations
>>> so you might see
>>>
>>>     make depend
>>>     make
>>>
>>> The second command uses the dependency files produced by the first. It
>>> would be better to have that stuff handled automatically.
>> 
>> Others thought so too, so there's a way to do it (at least if you use
>> gcc or clang).  See e.g. this Makefile of mine:
>> https://github.com/kjgrahn/groblad/blob/master/Makefile
>> 
>> Or
>> 
>> * Paul D. Smith: Auto-Dependency Generation
>>    http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
>
> Thanks. Yes, I know. I spent some hours working on this a few years ago. 
> It was nightmarish. I think I tried two published solutions and one of 
> my own. All had failings of one sort or another.

Ok.  Well, it works very nicely for me (see the Github link above).

> Can't remember the 
> details now but IIRC I concluded the make system is not designed to 
> process dependencies after it has started executing commands.

I'm under the impression that Gnu Make /does/ support this, and the
Smith paper says so too.

What I /have/ had problems with has been recursive dependencies via
generated header files. I cannot remember the details, but sent a memo
to myself never to let generated code into the middle of the
dependency graph.

> As for gcc, I didn't want to depend on a particular version of make
> or a particular compiler, and this was to work with an assembler as
> well as a compiler.

Oh.  Well, that would certainly explain your problems.  Not the
assembly part I think, but the lack of gcc/clang and more importantly
a full-featured Make.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#127362

On 04/03/2018 08:43, James Harris wrote:
> On 03/03/2018 15:32, bartc wrote:

>> These days I use a linker where /everything/ has to be specified
>> explicitly. 
> 
> Me too. Again, don't you think that much of that should be generatable 
> by a program?

There still needs to be some input to the program; where does that 
exist, and in what format?

I'd thought my own projects were small and simple compared with some of 
the stuff I come across. But they present some challenges of their own.

Normally my IDE takes care of the details (it is rather crude but it 
does the job), but if I wanted someone else to build my project, I 
wouldn't want to inflict my crummy home-made tools on them.

So what I've done in the past, is gone to some effort to bundle the 
project into a single C source file, with no headers or include files. 
Then someone can build it using:

    gcc -m64 program.c -oprogram.exe

or equivalent using their C compiler.

But this is not satisfactory either if someone wants to do anything else 
than just create the executable (for Windows, I can just supply the 
executable anyway). And it is not the true source code (which is not in C).

So I can envisage an intermediate representation of the project, which 
can be just a script (even a Windows batch file would do), showing each 
invocation of each program that is needed. Output from a log file might 
work as well.

Then someone will get an idea of what is involved, and can adapt as needed.

Or the representation can just be in English, which lists the files (in 
the correct if that is important), where they might be located, the 
programs needed to be run on them, and the variations used in different 
configurations.

Actually, here is attempt at such a batch file: 
https://pastebin.com/raw/UeTNgxkz.

This worked, but as it is will only build one configuration. But enough 
info is present to give a picture of what the build process looks like, 
what files are involved, and how they are structured.

(It's challenging also as the source files are not C, and it doesn't use 
a mainstream compiler. Another complication is that the files must be 
compiled in a certain order because of how the module system works.)

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#127411

On 05/03/2018 15:29, bartc wrote:
> On 04/03/2018 08:43, James Harris wrote:
>> On 03/03/2018 15:32, bartc wrote:
> 
>>> These days I use a linker where /everything/ has to be specified
>>> explicitly.
>>
>> Me too. Again, don't you think that much of that should be generatable
>> by a program?
> 
> There still needs to be some input to the program; where does that
> exist, and in what format?

Funny you should ask that. Assuming we are talking about the same thing 
I've been wrestling with that exact issue. And I think I am getting 
somewhere - as follows.

The basic requirement is that the program which runs compilers and 
linkers (i.e. the replacement for make, which I'll call the 'compiler 
driver') knows exactly what command to run each time. For security, the 
programs it is allowed to invoke cannot not be part of the project as 
distributed but have to be specified locally. But there are also some 
settings which will need to be supplied with the project itself.

Therefore there seems to be a need for config files to exist in three 
places:

1. System-wide config in /etc on Unix (and %AppData% or whatever it's 
called on Windows).

2. User-chosen config in ~ on Unix (and %Home% or whatever the home 
directory is called on Windows).

(At least one of (1) or (2) would need to exist but probably both would, 
with any user-chosen config taking precedence over any system-wide 
config when the compiler driver were run by a specific user.)

3. Project-specific config in the project itself. This latter config 
could all be in one file at the root of the project file tree or it 
could conceivably be split so that some is in the root and some is lower 
down the tree.

For security, the programs which the compiler driver would invoke would 
all be in places (1) and (2). By contrast, the description of what files 
need what treatment would all be in (3).

Basically, the type (3) config files would describe the source files in 
the project and the other two config files would tell the compiler 
driver how to convert those descriptions into compile and link commands.

As to what format the files would take, I am not sure of the details but 
I am thinking that they would be text files which contained simple 
textual mappings and/or allowed expressions which updated variables, 
somewhat like make does.

> 
> I'd thought my own projects were small and simple compared with some of
> the stuff I come across. But they present some challenges of their own.
> 
> Normally my IDE takes care of the details (it is rather crude but it
> does the job), but if I wanted someone else to build my project, I
> wouldn't want to inflict my crummy home-made tools on them.
> 
> So what I've done in the past, is gone to some effort to bundle the
> project into a single C source file, with no headers or include files.
> Then someone can build it using:
> 
>      gcc -m64 program.c -oprogram.exe
> 
> or equivalent using their C compiler.
> 
> But this is not satisfactory either if someone wants to do anything else
> than just create the executable (for Windows, I can just supply the
> executable anyway). And it is not the true source code (which is not in C).
> 
> So I can envisage an intermediate representation of the project, which
> can be just a script (even a Windows batch file would do), showing each
> invocation of each program that is needed. Output from a log file might
> work as well.

You and I are looking at the same problem, i.e. how to distribute 
projects. There's no simple answer, is there!

You may already know but some of the Unix projects use very well 
developed package-distribution mechanisms where one command can download 
and install a given package. Such mechanisms handle all dependencies 
automatically. AIUI they work with pre-compiled object files rather than 
sources.

> 
> Then someone will get an idea of what is involved, and can adapt as needed.
> 
> Or the representation can just be in English, which lists the files (in
> the correct if that is important), where they might be located, the
> programs needed to be run on them, and the variations used in different
> configurations.
> 
> Actually, here is attempt at such a batch file:
> https://pastebin.com/raw/UeTNgxkz.

It's very easy to read - which makes a nice change from most makefiles - 
but it is, of course, very specific and very manual.

> 
> This worked, but as it is will only build one configuration. But enough
> info is present to give a picture of what the build process looks like,
> what files are involved, and how they are structured.
> 
> (It's challenging also as the source files are not C, and it doesn't use
> a mainstream compiler. Another complication is that the files must be
> compiled in a certain order because of how the module system works.)
> 


-- 
James Harris

[toc] | [prev] | [next] | [standalone]

#127412

On Wed, 7 Mar 2018 04:54:59 +0000, James Harris
<james.harris.1@gmail.com> wrote:

>On 05/03/2018 15:29, bartc wrote:
>> On 04/03/2018 08:43, James Harris wrote:
>>> On 03/03/2018 15:32, bartc wrote:
>> 
>>>> These days I use a linker where /everything/ has to be specified
>>>> explicitly.
>>>
>>> Me too. Again, don't you think that much of that should be generatable
>>> by a program?
>> 
>> There still needs to be some input to the program; where does that
>> exist, and in what format?
>
>Funny you should ask that. Assuming we are talking about the same thing 
>I've been wrestling with that exact issue. And I think I am getting 
>somewhere - as follows.
>
>The basic requirement is that the program which runs compilers and 
>linkers (i.e. the replacement for make, which I'll call the 'compiler 
>driver') knows exactly what command to run each time. For security, the 
>programs it is allowed to invoke cannot not be part of the project as 
>distributed but have to be specified locally. But there are also some 
>settings which will need to be supplied with the project itself.


It's not that uncommon to have a program that's built as part of the
build needing to be run to support a later part of a build.  For
example, there's a parser for the grammar definition for an
interpreter in one of our products.  It's a C++ program.  It gets
built, then run against the grammar definition file(s) for the
interpreter, and outputs some .cpp and .h files, which are then
compiled into things later in the build.

You probably need to make at least some sort of way to define an
exception to that rule.

[toc] | [prev] | [next] | [standalone]

#127415

On 07/03/2018 06:33, Robert Wessel wrote:
> On Wed, 7 Mar 2018 04:54:59 +0000, James Harris
> <james.harris.1@gmail.com> wrote:
> 
>> On 05/03/2018 15:29, bartc wrote:
>>> On 04/03/2018 08:43, James Harris wrote:
>>>> On 03/03/2018 15:32, bartc wrote:
>>>
>>>>> These days I use a linker where /everything/ has to be specified
>>>>> explicitly.
>>>>
>>>> Me too. Again, don't you think that much of that should be generatable
>>>> by a program?
>>>
>>> There still needs to be some input to the program; where does that
>>> exist, and in what format?
>>
>> Funny you should ask that. Assuming we are talking about the same thing
>> I've been wrestling with that exact issue. And I think I am getting
>> somewhere - as follows.
>>
>> The basic requirement is that the program which runs compilers and
>> linkers (i.e. the replacement for make, which I'll call the 'compiler
>> driver') knows exactly what command to run each time. For security, the
>> programs it is allowed to invoke cannot not be part of the project as
>> distributed but have to be specified locally. But there are also some
>> settings which will need to be supplied with the project itself.
> 
> 
> It's not that uncommon to have a program that's built as part of the
> build needing to be run to support a later part of a build.  For
> example, there's a parser for the grammar definition for an
> interpreter in one of our products.  It's a C++ program.  It gets
> built, then run against the grammar definition file(s) for the
> interpreter, and outputs some .cpp and .h files, which are then
> compiled into things later in the build.
> 
> You probably need to make at least some sort of way to define an
> exception to that rule.

Good point.


-- 
James Harris

[toc] | [prev] | [next] | [standalone]

#127345

On Sat, 2018-03-03, James Harris wrote:
> On 01/03/2018 22:46, Jorgen Grahn wrote:
>> On Thu, 2018-03-01, James Harris wrote:
>>>
>>> It is normal in C programming for any source file which exports
>>> identifiers to have an associated header file which /describes/ the
>>> identifiers the source exports. For example, in a folder called PP then
>>> the two files might be named
>>>
>>>     PP/SS.c  (the source)
>>>     PP/SS.h  (the header describing SS.c's exports)
>>>
>>> When another source file, say UU.c, imports an identifier that SS.c
>>> exports it would naturally contain a statement which brings in the
>>> header at compile time.
>>>
>>>     UU.c
>>>       #include "SS.h"
>>>
>>> Because C has no concept of a hierarchical namespace here, i.e. the
>>> source file cannot say "include PP/SS.h" or "include SS.h from folder
>>> PP", the compiler has to be told about folder PP so it looks in it for
>>> the header. And that leads to my first query. (Yes, there is one!) If a
>>> given source file SS.c includes many headers, and the headers could be
>>> scattered to the four winds - i.e. distributed around the file system -
>> 
>> No sane real project is scattered around the file system: it lives
>> under a well-defined top-level directory.
>
> I agree. But a program should cope with input insanity!

Yes, by reporting an error and doing nothing.

> In other words, the problem in this case is how to generate compile 
> commands, making no assumptions about how a programmer would have laid 
> out the source tree.
>
> To be clear, the idea is to have the compile commands generated by a 
> program. It would have to work with however the programmer chose to 
> arrange the source code.

I don't believe in that approach.  The key to writing a Makefile is
(IMO) to accept that you know the exact layout of the code, and encode
that knowledge into the Makefile.

(I think that goes for other build systems too.)

>>> in a really big project what's a good way of specifying where the
>>> compiler should look for included headers?
>>>
>>> One options is, of course, to use a separate compile command for each
>>> source file. But that could end up being a bit cumbersome. And it would
>>> - at least ostensibly - prevent automatic generation of the compile
>>> commands - which is what I am essentially intending to do.
>> 
>> I don't see why that would be needed, but agreed: it would be
>> terrible.
>> 
>>> Another option is, rather than associate a search path with each
>>> individual source file, to say that a certain path should be used for
>>> all .c files in a given folder. But then that might require the source
>>> code be grouped in a certain way that doesn't suit the project at hand.
>>> (Maybe.)
>>>
>>> At the moment I normally take the easy route and keep all the source
>>> code for a given project in just one one folder or in a few folders. But
>>> as projects get larger that approach does not scale at all well,
>> 
>> Why not?
>> 
>> Here's what I do, and it does scale.
>> 
>>    pp/     # project directory
>>    pp/1.c  # some source file
>>    pp/2.h  # some header
>> 
>>    pp/sub/     # some more or less well-defined subsystem
>>    pp/sub/1.c  # some source file
>>    pp/sub/2.h  # some header
>> 
>>    pp/test/    # unit tests
>>    pp/test/1.c
>> 
>> Files refer to headers on their own level with #include "2.h".
>> Files in pp/ refer to files in pp/sub/ with #include "sub/2.h".
>> Files in pp/sub/ don't refer to files in pp/ at all.
>> 
>> The unit tests treat pp/ as if it was an installed library,
>> and do #include <2.h>, #include <sub/2.h>.
>> 
>> It's also easy to write a single Makefile (pp/Makefile) for this
>> scheme.  The only thing you need to do is provide one -I option when
>> building the unit tests.
>> 
>>      It may be more difficult if you need to support really exotic
>>      platforms (which perhaps don't have directories) or if you need to
>>      do cross-compilation (so everything but the unit tests is build
>>      twice, using different compilers).
>
> Thanks for explaining your setup. I take it that all sources under pp/ 
> can be compiled with the same set of -I folders. That may not be the 
> case, in general, of course.

See above: I provide a single -I flag, so the unit tests can pretend
they're testing an installed library.

I don't see why I would need any other -I flags.  Note that I'm on
Linux, and any libraries I depend on are installed into the system
using the package manager.  The compiler already knows where to find
these.

> When you link to form a load module do you similarly use a project-wide 
> set of library folders (the -L folders to a typical linker)?

Same story there, except I tend to collect all the pp object files
into a library because it makes the Makefile easier to write.  Then
I'll need -L. to build the executables, because the linker doesn't
search the current directory by default.

>>> and I need to improve both how I lay out project code - and
>>> especially what I do when one project needs to import files from
>>> another.
>> 
>> That's the first time you brought up interdependencies between
>> multiple projects, but I sense that that was what you were thinking
>> about all along.
>> 
>> My advice is: /don't do it/.  I've never seen it being done
>> successfully, without a hefty cost in terms of lost productivity.  If
>> you need project A to depend on project B, make it depend on library B
>> being installed onto the system instead, the way you depend on the
>> standard library or third-party libraries.
>
> Understood, though I have to say I am more inclined to split existing 
> packages into smaller ones than to make each one more independent.

People seem to be inclined to do that, so I'm not surprised.  But I've
been in several projects where managing this has taken up 50% (!) or
more of everyone's time, which is why I'm against it.

I suppose it works when you can split things into libraries which
almost never have to change, and when new features can be implemented
in only one of them.  But splitting things exactly right seems to be
very difficult.

> Besides, this compiler-driver idea is meant to be part of a bigger 
> system - which would probably go a bit off topic just now. But a compile 
> should, basically, use the most recent updates to any source file.

I don't remember the compiler-driver thing, so I cannot comment.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#127420

On Sunday, March 4, 2018 at 7:39:59 AM UTC, Jorgen Grahn wrote:
> On Sat, 2018-03-03, James Harris wrote:
> 
> > To be clear, the idea is to have the compile commands generated by a 
> > program. It would have to work with however the programmer chose to 
> > arrange the source code.
> 
> I don't believe in that approach.  The key to writing a Makefile is
> (IMO) to accept that you know the exact layout of the code, and encode
> that knowledge into the Makefile.
> 
With CMake, you point the CMake system at the components of your program,
often using an IDE, then it builds the makefile script for you.
The problem with the whole approach is that you are adding yet another
layer of complexity, but CMake has been pretty much accepted as a
mainstream tool.

[toc] | [prev] | [next] | [standalone]

#127422

On 07/03/2018 13:00, Malcolm McLean wrote:
> On Sunday, March 4, 2018 at 7:39:59 AM UTC, Jorgen Grahn wrote:
>> On Sat, 2018-03-03, James Harris wrote:
>>
>>> To be clear, the idea is to have the compile commands generated by a
>>> program. It would have to work with however the programmer chose to
>>> arrange the source code.
>>
>> I don't believe in that approach.  The key to writing a Makefile is
>> (IMO) to accept that you know the exact layout of the code, and encode
>> that knowledge into the Makefile.
>>
> With CMake, you point the CMake system at the components of your program,
> often using an IDE, then it builds the makefile script for you.
> The problem with the whole approach is that you are adding yet another
> layer of complexity, but CMake has been pretty much accepted as a
> mainstream tool.

And the complexity is there even when the application you want to apply 
it to doesn't merit it.

There's a compiler project called Smaller C on Github. It uses 
configure, gnumakefile, makefile and common.mk files, only totalling 100 
lines or so, but consisting of indecipherable rubbish.

But guess what, to built the main compiler you can just do this:

   gcc smlrc.c -osmlrc.exe

(This uses an include file like cgx86.c depending on target. There are 
also separate programs for a linker and a compiler driver.)

Tiny C can also be built with a single compile command, but it needs 900 
lines, iirc, of this configure/makefile nonsense.

What you really need to know is what macros to set up that correspond to 
your required target; why not just say that in the readme file?

It's a disease I think. And then some people what to add Cmake on top ...

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#127424

bartc <bc@freeuk.com> writes:

> And the complexity is there even when the application you want to
> apply it to doesn't merit it.

By which you mean even when you can't see the point.

> There's a compiler project called Smaller C on Github. It uses
> configure, gnumakefile, makefile and common.mk files, only totalling
> 100 lines or so, but consisting of indecipherable rubbish.

I can decipher it.  You meant indecipherable and not useful to you.  The
configure script and makefile are probably there to provide the usual
"./configure; make" method for people who don't even want to read the
documents.  It simplifies the process.

> But guess what, to built the main compiler you can just do this:
>
>   gcc smlrc.c -osmlrc.exe

There is no need to guess.  The documentation says so quite clearly.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#127435

On 07/03/2018 14:26, Ben Bacarisse wrote:
> bartc <bc@freeuk.com> writes:
> 
>> And the complexity is there even when the application you want to
>> apply it to doesn't merit it.
> 
> By which you mean even when you can't see the point.
> 
>> There's a compiler project called Smaller C on Github. It uses
>> configure, gnumakefile, makefile and common.mk files, only totalling
>> 100 lines or so, but consisting of indecipherable rubbish.
> 
> I can decipher it.  You meant indecipherable and not useful to you.  The
> configure script and makefile are probably there to provide the usual
> "./configure; make" method for people who don't even want to read the
> documents.  It simplifies the process.


Fine; have both. An overview in English, and an automatic script.

The problems with automatic script are the plethora of extra tools that 
might be needed (and which are likely to introduce a dozen failure 
opportunities - try building CPython on Windows, even using VS), and 
also, despite your comment, a cryptic syntax (but I know that you have a 
penchant for such languages).

That cryptic syntax and all those dependences mean it is harder to build 
independently (so I can compile Tiny C, but don't know what macros to 
set up so that it knows where to find its include files, or to tell it 
to target x86).

>> But guess what, to built the main compiler you can just do this:
>>
>>    gcc smlrc.c -osmlrc.exe
> 
> There is no need to guess.  The documentation says so quite clearly.

Ha, so it does (when you delve into the .md files). However the download 
I've used from 2015 didn't have those files.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#127442

On Wednesday, March 7, 2018 at 10:35:39 AM UTC-6, Bart wrote:
> The problems with automatic script are the plethora of extra tools that 
> might be needed (and which are likely to introduce a dozen failure 
> opportunities - try building CPython on Windows, even using VS), and 
> also, despite your comment, a cryptic syntax (but I know that you have a 
> penchant for such languages).

A bigger issue is that someone who wants to write code that is usable on
multiple development systems would need to have access to every system on
which the code is supposed to be usable in order to have a high probability
of actually making it work on all of them.

If instead a program can be designed so that it will work with any common
microprocessor platform where char, short, and long are 8, 16, and 32 bits,
respectively, are consistently big-endian or little-endian, and allow such
representation to be accessed using lvalues of any integer type which are
visibly derived in the context where they are used, and if such code
requires that either BIG_ENDIAN or LITTLE_ENDIAN be pre-declared, then
the code could be expected to be usable on any development system meeting
those requirements without having to be tested on every single one of them.

[toc] | [prev] | [next] | [standalone]

#127444

supercat@casperkitty.com writes:
[...]
> If instead a program can be designed so that it will work with any common
> microprocessor platform where char, short, and long are 8, 16, and 32 bits,
> respectively, are consistently big-endian or little-endian, and allow such
> representation to be accessed using lvalues of any integer type which are
> visibly derived in the context where they are used, and if such code
> requires that either BIG_ENDIAN or LITTLE_ENDIAN be pre-declared, then
> the code could be expected to be usable on any development system meeting
> those requirements without having to be tested on every single one of them.

Consider.  Writing.  Shorter.  Sentences.

Any program that assumes 32-bit long will fail on many of the systems I
use, where long is 64 bits.  How about writing code that doesn't assume
how wide long is?

No conforming implementation can predefine BIG_ENDIAN or LITTLE_ENDIAN.
And most code doesn't need to care about endianness.  I wouldn't mind
having a way to determine the current system's endianness, (maybe
something in <limits.h>), but it's not a huge deal.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#127447

On Wednesday, March 7, 2018 at 12:11:15 PM UTC-6, Keith Thompson wrote:
> supercat@casperkitty.com writes:
> [...]
> > If instead a program can be designed so that it will work with any common
> > microprocessor platform where char, short, and long are 8, 16, and 32 bits,
> > respectively, are consistently big-endian or little-endian, and allow such
> > representation to be accessed using lvalues of any integer type which are
> > visibly derived in the context where they are used, and if such code
> > requires that either BIG_ENDIAN or LITTLE_ENDIAN be pre-declared, then
> > the code could be expected to be usable on any development system meeting
> > those requirements without having to be tested on every single one of them.
> 
> Consider.  Writing.  Shorter.  Sentences.
> 
> Any program that assumes 32-bit long will fail on many of the systems I
> use, where long is 64 bits.  How about writing code that doesn't assume
> how wide long is?

A lot of code was written before C99 added int32_t and uint32_t as
standards.  The closest things to standard 32-bit type were "long" and
"unsigned long", and an assumption that "long" would be 32 bits was
about as portable as an assumption that "char" would be eight.  A
machine with 36-bit words wouldn't have a 32-bit long, but it wouldn't
have an 8-bit "char" either.

Further, a program could provide users of platforms where e.g. "int" is
32 bits but "long" is bigger with instructions for how to deal with that,
e.g. by starting with

#ifndef I32_TYPE
#define I32_TYPE long
#endif
#ifndef U32_TYPE
#define U32_TYPE unsigned long
#endif

and instructing users of such platforms to predefine "I32_TYPE" as a macro
which describes a signed 32-bit type, and U32_TYPE as a macro which describes
an unsigned 32-bit type.  Users of more common implementations wouldn't have
to worry about defining such macros.

> No conforming implementation can predefine BIG_ENDIAN or LITTLE_ENDIAN.

Most quality implementations can easily be configured for a non-conforming
mode in which such symbols are defined, using a command-line option like
"-dBIG_ENDIAN".  Alternatively, code could be shipped with an essentially-
empty "opts.h" file which is the first thing included by each other file,
and which could be edited if necessary to pre-define any macros needed
elsewhere.

> And most code doesn't need to care about endianness.  I wouldn't mind
> having a way to determine the current system's endianness, (maybe
> something in <limits.h>), but it's not a huge deal.

It can be important for code that handles things like encryption, big-number
math, CRC calculations, etc.  Not the most common usages, but not ones that
should be totally ignored either.

On the other hand, before anything like that could be added to limits.h in
any standard way, there would first have to be agreement that there are
four common ways of associated the bits of a "uint32_t" with the bits of
the underlying bytes, and that the other (32!-4) mappings are sufficiently
unusual as to justify lumping them into an "other" category.

[toc] | [prev] | [next] | [standalone]

#127449

supercat@casperkitty.com writes:
> On Wednesday, March 7, 2018 at 12:11:15 PM UTC-6, Keith Thompson wrote:
>> supercat@casperkitty.com writes:
>> [...]
>> > If instead a program can be designed so that it will work with any common
>> > microprocessor platform where char, short, and long are 8, 16, and 32 bits,
>> > respectively, are consistently big-endian or little-endian, and allow such
>> > representation to be accessed using lvalues of any integer type which are
>> > visibly derived in the context where they are used, and if such code
>> > requires that either BIG_ENDIAN or LITTLE_ENDIAN be pre-declared, then
>> > the code could be expected to be usable on any development system meeting
>> > those requirements without having to be tested on every single one of them.
>> 
>> Consider.  Writing.  Shorter.  Sentences.
>> 
>> Any program that assumes 32-bit long will fail on many of the systems I
>> use, where long is 64 bits.  How about writing code that doesn't assume
>> how wide long is?
>
> A lot of code was written before C99 added int32_t and uint32_t as
> standards.  The closest things to standard 32-bit type were "long" and
> "unsigned long", and an assumption that "long" would be 32 bits was
> about as portable as an assumption that "char" would be eight.  A
> machine with 36-bit words wouldn't have a 32-bit long, but it wouldn't
> have an 8-bit "char" either.

Yes, but int32_t and uint32_t do exist now.  Are you looking for new and
improved ways to update the C90 standard?  They did that, and it's
called C99 (and C11).

Type long is very commonly 64 bits these days.  If you want to write
code that assumes it's 32, you can always write:

    #if LONG_MAX != 0x7fffffff
    #error "long is not 32 bits"
    #fi

Or, you can use int32_t and uint32_t.

> Further, a program could provide users of platforms where e.g. "int" is
> 32 bits but "long" is bigger with instructions for how to deal with that,
> e.g. by starting with
>
> #ifndef I32_TYPE
> #define I32_TYPE long
> #endif
> #ifndef U32_TYPE
> #define U32_TYPE unsigned long
> #endif
>
> and instructing users of such platforms to predefine "I32_TYPE" as a macro
> which describes a signed 32-bit type, and U32_TYPE as a macro which describes
> an unsigned 32-bit type.  Users of more common implementations wouldn't have
> to worry about defining such macros.

Are you trying to reinvent int32_t and uint32_t?  Why??

>> No conforming implementation can predefine BIG_ENDIAN or LITTLE_ENDIAN.
>
> Most quality implementations can easily be configured for a non-conforming
> mode in which such symbols are defined, using a command-line option like
> "-dBIG_ENDIAN".  Alternatively, code could be shipped with an essentially-
> empty "opts.h" file which is the first thing included by each other file,
> and which could be edited if necessary to pre-define any macros needed
> elsewhere.

My point is simply that if such macros are to be added in a future
standard, either they'll have to be in the implementation-reserved
namespace (_BIG_ENDIAN, _LITTLE_ENDIAN), or they'll have to be in a new
standard header, or they'll risk breaking existing code.  The latter
isn't necessarily a show-stopper.  C99 broke C90 code that included
<limits.h> and defined LLONG_MAX, and I don't think that was a problem.

>> And most code doesn't need to care about endianness.  I wouldn't mind
>> having a way to determine the current system's endianness, (maybe
>> something in <limits.h>), but it's not a huge deal.
>
> It can be important for code that handles things like encryption, big-number
> math, CRC calculations, etc.  Not the most common usages, but not ones that
> should be totally ignored either.

Is the lack of predefined BIG_ENDIAN and LITTLE_ENDIAN macros (however
they're spelled) a real problem for people writing such code?  It's not
hard to incorporate that into a configuration script.

> On the other hand, before anything like that could be added to limits.h in
> any standard way, there would first have to be agreement that there are
> four common ways of associated the bits of a "uint32_t" with the bits of
> the underlying bytes, and that the other (32!-4) mappings are sufficiently
> unusual as to justify lumping them into an "other" category.

Four common ways?  I'd say two (big-endian and little-endian) should be
sufficient.  Are there middle-endian platforms other than the PDP-11?
Are there likely to be conforming C202Y implementations for such
platforms?

I have no particular objection to adding something to, say,
<limits.h> that would let an implementation specify whether it's
big-endian, little-endian, or something else, though I'm skeptical
that it's worthwhile.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#127454

On Wednesday, March 7, 2018 at 2:01:32 PM UTC-6, Keith Thompson wrote:
> supercat@casperkitty.com writes:
> > A lot of code was written before C99 added int32_t and uint32_t as
> > standards.  The closest things to standard 32-bit type were "long" and
> > "unsigned long", and an assumption that "long" would be 32 bits was
> > about as portable as an assumption that "char" would be eight.  A
> > machine with 36-bit words wouldn't have a 32-bit long, but it wouldn't
> > have an 8-bit "char" either.
> 
> Yes, but int32_t and uint32_t do exist now.  Are you looking for new and
> improved ways to update the C90 standard?  They did that, and it's
> called C99 (and C11).

Some embedded platforms have a limited range of compilers available.  If
one needs to update the code for a piece of existing hardware which uses
an obscure platform, it may not be possible to use a C99 compiler.  Use
of uint32_t probably wouldn't be a problem even on old compilers (since
pre-defining a macro with that name would likely work) but I'm curious what
justification there is for code that would require "long" to be 64 bits.
In what situations would it be appropriate for code that would require
"long" to be 64 bits to use that type rather than uint64_t, size_t, or
uintptr_t?  I know many Linux users regard the LLP64 set of types used
by Windows as inferior to the LP64 set used by Linux, but I fail to see
any basis for that.  The only difference I see between them is that one
is compatible with code written for C89 compilers while the other is
compatible with code whose programmers opted to use "long" in cases where
some other 64-bit type would have been more appropriate.

> >> No conforming implementation can predefine BIG_ENDIAN or LITTLE_ENDIAN.
> >
> > Most quality implementations can easily be configured for a non-conforming
> > mode in which such symbols are defined, using a command-line option like
> > "-dBIG_ENDIAN".  Alternatively, code could be shipped with an essentially-
> > empty "opts.h" file which is the first thing included by each other file,
> > and which could be edited if necessary to pre-define any macros needed
> > elsewhere.
> 
> My point is simply that if such macros are to be added in a future
> standard, either they'll have to be in the implementation-reserved
> namespace (_BIG_ENDIAN, _LITTLE_ENDIAN), or they'll have to be in a new
> standard header, or they'll risk breaking existing code.  The latter
> isn't necessarily a show-stopper.  C99 broke C90 code that included
> <limits.h> and defined LLONG_MAX, and I don't think that was a problem.

If a program is written to require that such macros be pre-declared in user
namespace, then it won't run the risk of having an implementation or a
later version of the Standard pre-define them in a fashion that would
contradict a program's requirements.

> Is the lack of predefined BIG_ENDIAN and LITTLE_ENDIAN macros (however
> they're spelled) a real problem for people writing such code?  It's not
> hard to incorporate that into a configuration script.

Not every build system uses Unix.

A configuration script will only be usable on build systems that support the
tools it's designed to work with.  A text file that says to invoke the
compiler in a way that pre-defines either BIG_ENDIAN or LITTLE_ENDIAN will
work on any compiler that allows macros to be pre-defined at the time of
invocation.

> > On the other hand, before anything like that could be added to limits.h in
> > any standard way, there would first have to be agreement that there are
> > four common ways of associated the bits of a "uint32_t" with the bits of
> > the underlying bytes, and that the other (32!-4) mappings are sufficiently
> > unusual as to justify lumping them into an "other" category.
> 
> Four common ways?  I'd say two (big-endian and little-endian) should be
> sufficient.  Are there middle-endian platforms other than the PDP-11?
> Are there likely to be conforming C202Y implementations for such
> platforms?
> 
> I have no particular objection to adding something to, say,
> <limits.h> that would let an implementation specify whether it's
> big-endian, little-endian, or something else, though I'm skeptical
> that it's worthwhile.

I don't think it's particularly likely that any C implementations in the
last 20 years have used either mixed-endian format, but I can certainly
imagine that such things could exist.  My point is whether or not those
should be treated as "other ordering", the other 2.63E+35 orderings
certainly should.

[toc] | [prev] | [next] | [standalone]

Page 2 of 6 — ← Prev page 1 [2] 3 4 5 6  Next page →

Back to top | Article view | comp.lang.c

csiph-web