Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compilers > #3403 > unrolled thread

fledgling assembler programmer

Back to article view | Back to comp.compilers

Contents

  fledgling assembler programmer Alan.Beck@darkrealms.ca (Alan Beck) - 2023-03-21 17:40 -0400
    Re: fledgling assembler programmer gah4 <gah4@u.washington.edu> - 2023-03-21 17:23 -0700
      Re: fledgling assembler programmer Thomas Koenig <tkoenig@netcologne.de> - 2023-03-22 06:49 +0000
        Re: fledgling assembler programmer gah4 <gah4@u.washington.edu> - 2023-03-22 13:31 -0700
          Re: fledgling assembler programmer Thomas Koenig <tkoenig@netcologne.de> - 2023-03-23 11:26 +0000
            Re: fledgling assembler programmer gah4 <gah4@u.washington.edu> - 2023-03-24 14:17 -0700
              Re: ancient PL/I, was fledgling assembler programmer drb@ihatespam.msu.edu (Dennis Boone) - 2023-03-24 22:51 +0000
                Re: ancient PL/I, was fledgling assembler programmer gah4 <gah4@u.washington.edu> - 2023-03-24 22:44 -0700
                  Re: ancient PL/I, was fledgling assembler programmer gah4 <gah4@u.washington.edu> - 2023-03-25 01:27 -0700
              Re: fledgling assembler programmer Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2023-03-25 13:07 +0100
                Re: fledgling assembler programmer George Neuner <gneuner2@comcast.net> - 2023-03-25 20:54 -0400
                  Portable Software (was: fledgling assembler programmer) Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2023-03-28 09:21 +0200
                    Re: Portable Software (was: fledgling assembler programmer) arnold@freefriends.org (Aharon Robbins) - 2023-03-28 14:42 +0000
                      Re: configuguration tools, Portable Software (was: fledgling assembler programmer) Kaz Kylheku <864-117-4973@kylheku.com> - 2023-03-29 18:33 +0000
                        Re: configuguration tools, Portable Software (was: fledgling assembler programmer) arnold@skeeve.com (Aharon Robbins) - 2023-03-31 07:10 +0000
                        Re: configuguration tools, Portable Software (was: fledgling assembler programmer) anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2023-04-02 08:56 +0000
                      Re: Portable Software Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2023-03-31 07:49 +0200
                        Re: Portable Software anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2023-04-02 10:04 +0000
                          Re: Portable Software Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2023-04-05 11:23 +0200
                            Re: Portable Software anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2023-04-05 16:30 +0000
                              Re: Portable Software Kaz Kylheku <864-117-4973@kylheku.com> - 2023-04-06 08:35 +0000
                              Re: Portable Software Hans-Peter Diettrich <DrDiettrich1@netscape.net> - 2023-04-07 15:35 +0200
                              Re: Portable Software Thomas Koenig <tkoenig@netcologne.de> - 2023-04-08 18:25 +0000
                    Re: Portable Software (was: fledgling assembler programmer) gah4 <gah4@u.washington.edu> - 2023-03-28 14:21 -0700
                      Re: Portable Software (was: fledgling assembler programmer) Kaz Kylheku <864-117-4973@kylheku.com> - 2023-03-29 18:34 +0000
                    Re: Portable Software (was: fledgling assembler programmer) George Neuner <gneuner2@comcast.net> - 2023-03-28 17:26 -0400
                      Re: Portable python Software (was: fledgling assembler programmer) George Neuner <gneuner2@comcast.net> - 2023-03-29 13:50 -0400
                      Re: Portable Software (was: fledgling assembler programmer) gah4 <gah4@u.washington.edu> - 2023-03-29 11:27 -0700
                        Re: Portable Software (was: fledgling assembler programmer) Thomas Koenig <tkoenig@netcologne.de> - 2023-03-31 05:19 +0000
                          Re: Portable Software (was: fledgling assembler programmer) gah4 <gah4@u.washington.edu> - 2023-03-31 12:41 -0700
                    Re: Portable Software (was: fledgling assembler programmer) anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2023-03-31 16:34 +0000
        Re: fledgling assembler programmer arnold@skeeve.com (Aharon Robbins) - 2023-03-23 13:56 +0000
      Re: fledgling assembler programmer anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2023-03-22 10:02 +0000
    Re: fledgling assembler programmer David Brown <david.brown@hesbynett.no> - 2023-03-22 14:39 +0100
    Re: fledgling assembler programmer George Neuner <gneuner2@comcast.net> - 2023-03-22 14:54 -0400

Page 1 of 2  [1] 2  Next page →

#3403 — fledgling assembler programmer

//Hello all,//

Hi,

I have started to learn Assembler out of an old book.

It is ancient (2003) but I don't think 8086 programming has changed
much. But the tools have.

I took assembly language in school but dropped out. Now I want another
go at it.

Would someone be my Mentor and answer a ton of questions that would
dwindle out as time went on?

If it's OK, we could do it here. Or netmail

Books are from a bookstore.


Book 1
Assembly Language for the PC 3rd edition, John Socha and Peter Norton.

Book 2
Assembly Language (step by step) Jeff Duntemann. Too Chatty.

I cannot afford a modern book at this time.

Thats what I picked up from the thrift store.

These books are dated around the time I was taking machine code in
school and I find it interesting now.

I hope someone picks me up.

I am running linux and using DOSemu

Also a modern DEBUG and and a modern Vi

I am also a Ham Radio Operator (45 years)

1:229/426.36

Regards,
Alan Beck
VY2XU
[Please reply directly unless the response is relatted to compilers. -John]

[toc] | [next] | [standalone]

#3404

On Tuesday, March 21, 2023 at 2:40:22 PM UTC-7, Alan Beck wrote:

> I have started to learn Assembler out of an old book.

(Hopefully enough related to compilers.)

Not so long after I started learning OS/360 Fortran and PL/I, I found
the compiler option for printing out the generated code in sort-of
assembly language.  (Not actually assembleable, though.)

About that time, I also had source listings on microfilm of
the OS/360 Fortran library, and some other Fortran callable
assembly programs.  And also, the IBM S/370 Principles
of Operation.

With those, and no actual book meant to teach assembly
programming, I figured it out, and started writing my own
programs, though mostly callable from Fortran or PL/I.

Compilers today don't write out the generated code in the same way,
and there aren't so many libraries around to read. And, personally,
8086 is my least favorite to write assembly code in.

Learning C, and thinking about pointers and addresses, is a good start
toward assembly programming.

In any case, I don't think I have any idea how others learn
programming for any language, and especially not for assembly
programming. I used to read IBM reference manuals, cover to cover.
That was mostly high school years. After that, I figured out how to
use them as reference manuals.

Most of my 80x86 assembly programming in the last
20 years is (re)writing this one program:

rdtsc:  rdtsc
   ret

When called from C, and returning a 64 bit integer, it return the Time
Stamp Counter. (Works for 32 bit code, returning in EDX:EAX. 64 bit is
different.)

C programming works so well, that there are only a few
things you can't do in C, and so need assembly programs.

[toc] | [prev] | [next] | [standalone]

#3405

gah4 <gah4@u.washington.edu> schrieb:
> On Tuesday, March 21, 2023 at 2:40:22 PM UTC-7, Alan Beck wrote:
>
>> I have started to learn Assembler out of an old book.

At the risk of stating the blindingly obvious:  There is more
than one assembler language, each computer architecture has its
own (with extensions over time, too).  There are also sometimes
different syntax variant, for example AT&T vs. Intel.

[...]

> Compilers today don't write out the generated code in the same way,

Quite the opposite.

The standard on UNIXy systems is to write out assemblly language to
a file, which is then further processed with the actual assembler.
Use "-S" to just generate the foo.s file from foo.c.

Plus, you can disassemble object files and programs with "objdump -d".

> and there aren't so many libraries around to read.

Not ones written in assembler.  But it is possible to download
the source code to many libraries, for example glibc, and then
examine what it is compiled to.

Another possibility would be to use http://godbolt.org, which shows
you assembler generated for different systems with differnt options.
To really make sense of it for different architectures you are
not familiar with may be difficult, though).  Or build clang/LLVM
yourself and set different options for the architecture.

>And, personally,
> 8086 is my least favorite to write assembly code in.

I like 6502 even less :-)

> Learning C, and thinking about pointers and addresses, is a good start
> toward assembly programming.

That, I agree with.  And it helps a lot to also look at the
generated code.

[...]

> C programming works so well, that there are only a few
> things you can't do in C, and so need assembly programs.

Bringing it back a bit towards compilers:  Reading assembler code is
a good way to see where they generate inefficient or (more rarely)
incorrect code.  In some special cases, writing in assembler can
bring benefits of a factor of 2 or even 4 over compiler-generated
code, usually when SIMD is involved.

Assembler is a bit like Latin:  For most people, there is no need
to speak or write, but one should be able to read it.

[toc] | [prev] | [next] | [standalone]

#3409

On Wednesday, March 22, 2023 at 12:06:05 PM UTC-7, Thomas Koenig wrote:
> gah4 <ga...@u.washington.edu> schrieb:

(snip)

> > Compilers today don't write out the generated code in the same way,

> Quite the opposite.

> The standard on UNIXy systems is to write out assemblly language to
> a file, which is then further processed with the actual assembler.

Yes, not the same way.

Well, to be sure that this is about compilers, my favorite complaint
is the lost art of small memory compilers.  That is, ones that can
run in kilobytes instead of megabytes.

In any case, the OS/360 compilers don't write out assembly code
that an assembler would recognize.  It is meant for people.

Some write it out in two columns to save paper.
Labels don't have to agree with what assemblers recognize.
They don't have to be in the same order that they would be for
an assembler, though OS/360 object programs don't have to be
in order, either.

Having not thought about this for a while, I believe they put
in some comments that help human readers, though likely not
what an assembly programmer would say.

Unixy systems might put in some comments, but mostly don't.

[toc] | [prev] | [next] | [standalone]

#3410

gah4 <gah4@u.washington.edu> schrieb:

[...]

> Well, to be sure that this is about compilers, my favorite complaint
> is the lost art of small memory compilers.  That is, ones that can
> run in kilobytes instead of megabytes.

On the Internet, there is a project for almost everything - in this
case Tiny C, which still seems to be under active development.  Or
at least there are sill commits at https://repo.or.cz/w/tinycc.git .

However, there is a reason why compilers got so big - there is
always a balance to be struck between comilation speed, compiler
size and optimization.

An extreme example: According to "Abstracting Away the Machine", the
very first FORTRAN compiler was so slow that the size of programs
it could compile was limited by the MTBF of the IBM 704 of around
eight hours.

The balance has shifted over time, because of increasing computing
power and available memory that can be applied to compilation,
and because relatively more people use programs than use compilers
than ever before.  So, in today's environment, there is little
incentive for writing small compilers.

Also, languages have become bigger, more expressive, more powerful,
more bloated (take your pick), which also increases the size
of compilers.

[toc] | [prev] | [next] | [standalone]

#3414

On Friday, March 24, 2023 at 7:10:00 AM UTC-7, Thomas Koenig wrote:

(snip about the lost art of small memory compilers.)

> On the Internet, there is a project for almost everything - in this
> case Tiny C, which still seems to be under active development. Or
> at least there are sill commits at https://repo.or.cz/w/tinycc.git .

> However, there is a reason why compilers got so big - there is
> always a balance to be struck between comilation speed, compiler
> size and optimization.

When I was writing the above, I was looking at the Program Logic
Manual for the OS/360 Fortran G compiler.
(G means it is supposed to run in 128K.)

Fortran G was not written by IBM, but contracted out.  And is not
(mostly) in assembler, but in something called POP.  That is, it
is interpreted by the POP interpreter, with POPcode written using
assembler macros.   Doing that, for one, allows reusing the code
for other machines, though you still need to rewrite the code
generator.  But also, at least likely, it decreases the size of
the compiler.  POP instructions are optimized for things that
compiler need to do.

I also had the source to that so many years ago, but not the
manual describing it.

> An extreme example: According to "Abstracting Away the Machine", the
> very first FORTRAN compiler was so slow that the size of programs
> it could compile was limited by the MTBF of the IBM 704 of around
> eight hours.

I remember stories about how well its optimizer worked, when
it was believed that they had to compete in code speed with
experienced assembly programmers.  I don't remember anything
about how fast it was.


> The balance has shifted over time, because of increasing computing
> power and available memory that can be applied to compilation,
> and because relatively more people use programs than use compilers
> than ever before. So, in today's environment, there is little
> incentive for writing small compilers.

I first thought about this, when reading about the Hercules project
of an IBM S/370 emulator, and couldn't run gcc in 16MB.
(Well, subtract some for the OS, but it still wouldn't fit.)

> Also, languages have become bigger, more expressive, more powerful,
> more bloated (take your pick), which also increases the size
> of compilers.

OK, the IBM PL/I (F) compiler, for what many consider a bloated
language, is designed to run (maybe not well) in 64K.
At the end of every compilation it tells how much memory was
used, how much available, and how much to keep the symbol table
in memory.

[toc] | [prev] | [next] | [standalone]

#3415 — Re: ancient PL/I, was fledgling assembler programmer

 > OK, the IBM PL/I (F) compiler, for what many consider a bloated
 > language, is designed to run (maybe not well) in 64K.
 > At the end of every compilation it tells how much memory was
 > used, how much available, and how much to keep the symbol table
 > in memory.

It's... 30-some passes, iirc?

De
[Well, phases or overlays but yes, IBM was really good at slicing compilers
into pieces they could overlay. -John]

[toc] | [prev] | [next] | [standalone]

#3416 — Re: ancient PL/I, was fledgling assembler programmer

On Friday, March 24, 2023 at 9:13:05 PM UTC-7, Dennis Boone wrote:

(after I wrote)
> > OK, the IBM PL/I (F) compiler, for what many consider a bloated
> > language, is designed to run (maybe not well) in 64K.
> > At the end of every compilation it tells how much memory was
> > used, how much available, and how much to keep the symbol table
> > in memory.

> It's... 30-some passes, iirc?

> [Well, phases or overlays but yes, IBM was really good at slicing compilers
> into pieces they could overlay. -John]

It is what IBM calls, I believe, dynamic overlay.  Each module specifically
requests others to be loaded into memory.  If there is enough memory,
they can stay, otherwise they are removed.

And there are a few disk files to be used, when it is actually
a separate pass.  The only one I actually know, is if the preprocessor
is used, it writes a disk file with the preprocessor output.

And as noted, if it is really short on memory, the symbol table
goes out to disk.

Fortran H, on the other hand, uses the overlay system generated
by the linkage editor.  When running on virtual storage system, it is
usual to run the compiler through the linkage editor to remove
the overlay structure.  (One of the few linkers that knows how
to read its own output.)  Normally it is about 300K, without
overlay closer to 450K.
[Never heard of dynamic overlays on S/360. -John]

[toc] | [prev] | [next] | [standalone]

#3417 — Re: ancient PL/I, was fledgling assembler programmer

On Saturday, March 25, 2023 at 12:09:30 AM UTC-7, gah4 wrote:

(snip)

> It is what IBM calls, I believe, dynamic overlay. Each module specifically
> requests others to be loaded into memory. If there is enough memory,
> they can stay, otherwise they are removed.

Traditional overlays are generated by the linkage editor, and have
static offsets determined at link time.

PL/I (F) uses OS/360 LINK, LOAD, and DELETE macros to dynamically
load and unload modules.  The addresses are not static. IBM says:

    "The compiler consists of a number of phases
     under the supervision of compiler control
     routines. The compiler communicates with
     the control program of the operating
     system, for input/output and other
     services, through the control routines."

All described in:

http://bitsavers.trailing-edge.com/pdf/ibm/360/pli/GY28-6800-5_PL1_F_Program_Logic_Manual_197112.pdf

They do seem to be called phases, but there are both physical and
logical phases, where physical phases are what are more commonly
called phases. There are way more than 100 modules, but I stopped
counting.

(snip)
> [Never heard of dynamic overlays on S/360. -John]

It seems not to actually have a name.

[toc] | [prev] | [next] | [standalone]

#3419

On 3/24/23 10:17 PM, gah4 wrote:

> Fortran G was not written by IBM, but contracted out.  And is not
> (mostly) in assembler, but in something called POP.  That is, it
> is interpreted by the POP interpreter, with POPcode written using
> assembler macros.   Doing that, for one, allows reusing the code
> for other machines, though you still need to rewrite the code
> generator.  But also, at least likely, it decreases the size of
> the compiler.  POP instructions are optimized for things that
> compiler need to do.

After a look at "open software" I was astonished by the number of
languages and steps involved in writing portable C code. Also updates of
popular programs (Firefox...) are delayed by months on some platforms,
IMO due to missing manpower on the target systems for checks and the
adaptation of "configure". Now I understand why many people prefer
interpreted languages (Java, JavaScript, Python, .NET...) for a
simplification of their software products and spreading.

What's the actual ranking of programming languages? A JetBrains study
does not list any compiled language in their first 7 ranks in 2022. C++
follows on rank 8.

What does that trend mean to a compiler group? Interpreted languages
still need a front-end (parser) and back-end (interpreter), but don't
these tasks differ between languages compiled to hardware or interpretation?

DoDi

[toc] | [prev] | [next] | [standalone]

#3424

On Sat, 25 Mar 2023 13:07:57 +0100, Hans-Peter Diettrich
<DrDiettrich1@netscape.net> wrote:

>After a look at "open software" I was astonished by the number of
>languages and steps involved in writing portable C code. Also updates of
>popular programs (Firefox...) are delayed by months on some platforms,
>IMO due to missing manpower on the target systems for checks and the
>adaptation of "configure". Now I understand why many people prefer
>interpreted languages (Java, JavaScript, Python, .NET...) for a
>simplification of their software products and spreading.


Actually Python is the /only/ one of those that normally is
interpreted.  And the interpreter is so slow the language would be
unusable were it not for the fact that all of its standard library
functions and most of its useful extensions are written in C.

In practice Java and Javascript almost always are JIT compiled to
native code rather than interpreted.  There also exist offline (AOT)
compilers for both.

Many JIT runtimes do let you choose to have programs interpreted
rather than compiled, but running interpreted reduces performance so
much that it is rarely done unless memory is very tight.


.NET is not a language itself but rather a runtime system like the
Jave Platform.  .NET consists of a virtual machine: the Common
Language Runtime (CLR); and a set of standard libraries.  Similarly
the Java Platform consists of a virtual machine: the Java Virtual
Machine (JVM); and a set of standard libraries.  Compilers target
these runtime systems.

The .NET CLR does not include an interpreter ... I'm not aware that
there even is one for .NET.  There is an offline (AOT) compiler that
can be used instead of the JIT.



>What's the actual ranking of programming languages? A JetBrains study
>does not list any compiled language in their first 7 ranks in 2022. C++
>follows on rank 8.
>
>What does that trend mean to a compiler group? Interpreted languages
>still need a front-end (parser) and back-end (interpreter), but don't
>these tasks differ between languages compiled to hardware or interpretation?

The trend is toward "managed" environments which offer niceties like
GC, objects with automagic serialized access, etc., all to help
protect average programmers from themselves ... err, um, from being
unable to produce working software.


>DoDi
George

[toc] | [prev] | [next] | [standalone]

#3431 — Portable Software (was: fledgling assembler programmer)

On 3/26/23 1:54 AM, George Neuner wrote:
> On Sat, 25 Mar 2023 13:07:57 +0100, Hans-Peter Diettrich
> <DrDiettrich1@netscape.net> wrote:
>
>> After a look at "open software" I was astonished by the number of
>> languages and steps involved in writing portable C code. Also updates of
>> popular programs (Firefox...) are delayed by months on some platforms,
>> IMO due to missing manpower on the target systems for checks and the
>> adaptation of "configure". Now I understand why many people prefer
>> interpreted languages (Java, JavaScript, Python, .NET...) for a
>> simplification of their software products and spreading.
>
> Actually Python is the /only/ one of those that normally is
> interpreted.  And the interpreter is so slow the language would be
> unusable were it not for the fact that all of its standard library
> functions and most of its useful extensions are written in C.

My impression of "interpretation" was aimed at the back-end, where
tokenized (virtual machine...) code has to be brought to a physical
machine, with a specific firmware (OS). Then the real back-end has to
reside on the target machine and OS, fully detached from the preceding
compiler stages.

Then, from the compiler writer viewpoint, it's not sufficient to define
a new language and a compiler for it, instead it must placed on top of
some popular "firmware" like Java VM, CLR or C/C++ standard libraries,
or else a dedicated back-end and libraries have to be implemented on
each supported platform.

My impression was that the FSF favors C and ./configure for "portable"
code. That's why I understand that any other way is easier for the
implementation of really portable software, that deserves no extra
tweaks for each supported target platform, for every single program. Can
somebody shed some light on the current practice of writing portable
C/C++ software, or any other compiled language, that (hopefully) does
not require additional human work before or after compilation for a
specific target platform?

DoDi

[toc] | [prev] | [next] | [standalone]

#3433 — Re: Portable Software (was: fledgling assembler programmer)

In article <23-03-029@comp.compilers>,
Hans-Peter Diettrich  <DrDiettrich1@netscape.net> wrote:
>My impression was that the FSF favors C and ./configure for "portable"
>code.

Like many things, this is the result of evolution. Autoconf is well
over 20 years old, and when it was created the ISO C and POSIX standards
had not yet spread throughout the Unix/Windows/macOS world. It and the
rest of the autotools solved a real problem.

Today, the C and C++ worlds are easier to program in, but it's still
not perfect and I don't think I'd want to do without the autotools.
Particularly for the less POSIX-y systems, like MinGW and OpenVMS.

>Can somebody shed some light on the current practice of writing portable
>C/C++ software, or any other compiled language, that (hopefully) does
>not require additional human work before or after compilation for a
>specific target platform?

Well, take a look at Go.  The trend there (as in the Python, Java and
C# worlds) is to significantly beef up the standard libraries. Go
has regular expressions, networking, file system, process and all kinds
of other stuff in its libraries, all things that regular old C and C++ code
often has to (or had to) hand-roll.  That makes it a lot easier for
someone to just write the code to get their job done, as well as
providing for uniformity across both operating systems and applications
written in Go.

Go goes one step further, even. Following the Plan 9 example, the
golang.org Go compilers are also cross compilers. I can build a Linux
x86_64 executable on my macOS system just by setting some environment
variables when running 'go build'. Really nice.

The "go" tool itself also takes over a lot of the manual labor, such
as downloading libraries from the internet, managing build dependencies
(no need for "make") and much more.  I suspect that that is also a
trend.

Does that answer your question?

Arnold

[toc] | [prev] | [next] | [standalone]

#3438 — Re: configuguration tools, Portable Software (was: fledgling assembler programmer)

On 2023-03-28, Aharon Robbins <arnold@freefriends.org> wrote:
>
> In article <23-03-029@comp.compilers>,
> Hans-Peter Diettrich  <DrDiettrich1@netscape.net> wrote:
>>My impression was that the FSF favors C and ./configure for "portable"
>>code.
>
> Like many things, this is the result of evolution. Autoconf is well
> over 20 years old, and when it was created the ISO C and POSIX standards
> had not yet spread throughout the Unix/Windows/macOS world. It and the
> rest of the autotools solved a real problem.
>
> Today, the C and C++ worlds are easier to program in, but it's still
> not perfect and I don't think I'd want to do without the autotools.
> Particularly for the less POSIX-y systems, like MinGW and OpenVMS.

Counterpoint: Autotools are a real detriment to GNU project programs.

When a release is cut of a typical GNU program, special steps
are execute to prepare a tarball which has a compiled configure
script.

You cannot just do a "git clone" of a GNU program, and then run
./configure and build. You must run some "make boostrap" nonsense, and
that requires you to have various Autotools installed, and in specific
versions!

In the past what I have done to build a GNU program from version
control, as a quick and dirty shortcut, done was to find the tarball
which is the closest match to the baseline that I'm trying to build
(e.g. of GNU Make or GNU Awk or whatever). Unpack the tarball over the
repository and run ./configure.  Then "git reset --hard" the changes and
rebuild.

Most Autotools programs will not cleanly cross-compile. Autotools is tha
main reason why distro build systems use QEMU to create a virtual target
environment with native tools and libraries, and then build the
"cross-compiled" program as if it were native.

Among the problems are in Autoconf itself. If it knows the program
is being cross-compiled, any test which requires a test program to be
compiled and executed is disabled. Since the output of that configure
test is needed, bad defaults are substituted.
For instance, about a decade and a half ago I helped a company
replace Windriver cruft with an in-house distribution. Windriver's
cross-compiled Bash didn't have job control! Ctrl-Z, fg, bg stuff no
workie.  The reason was that it was just cross-compiled straight, on an
x86 build box.  It couldn't run the test to detect job control support,
and so it defaulted it off, even though the target machine had
"gnu-linux" in its string. In the in-house distro, my build steps for
bash exported numerous ac_cv_... internal variables to override the bad
defaults.

My TXR language project has a hand-written, not generated, ./configure
script. What you get in a txr-285.tar.gz tarball is exactly what you
get if you do a "git clone" and "git checkout txr-285", modulo
the presence of a .git directory and differing timestamps.

You just ./configure and make.

I have a "./configure --maintainer" mode which will require flex and bison
instead of using the shipped parser stuff, and that's about it.
You don't have to use that to do development work.

There is no incomprehensible nonsense in the build system at all.

None of my configure-time tests require the execution of a program;
For some situations, I have developed clever tricks to avoid it. For
instance, if you want to know the size of a data type:. Here
is a fragment:

printf "Checking what C integer type can hold a pointer ... "

if [ -z "$intptr" ] ; then
  cat > conftest.c <<!
#include <stddef.h>
#include <limits.h>
#include "config.h"

#define D(N, Z) ((N) ? (N) + '0' : Z)
#define UD(S) D((S) / 10, ' ')
#define LD(S) D((S) % 10, '0')
#define DEC(S) { UD(S), LD(S) }

struct sizes {
  char h_BYTE[32], s_BYTE[2];
#if HAVE_SUPERLONG_T
  char h_SUPERLONG[32], s_SUPERLONG[2];
#endif
#if HAVE_LONGLONG_T
  char h_LONGLONG[32], s_LONGLONG[2];
#endif
  char h_PTR[32], s_PTR[2];
  char h_LONG[32], s_LONG[2];
  char h_INT[32], s_INT[2];
  char h_SHORT[32], s_SHORT[2];
  char h_WCHAR[32], s_WCHAR[2];
  char nl[2];
} foo = {
  "\nSIZEOF_BYTE=", DEC(CHAR_BIT),
#if HAVE_SUPERLONG_T
  "\nSIZEOF_SUPERLONG_T=", DEC(sizeof (superlong_t)),
#endif
#if HAVE_LONGLONG_T
  "\nSIZEOF_LONGLONG_T=", DEC(sizeof (longlong_t)),
#endif
  "\nSIZEOF_PTR=", DEC(sizeof (char *)),
  "\nSIZEOF_LONG=", DEC(sizeof (long)),
  "\nSIZEOF_INT=", DEC(sizeof (int)),
  "\nSIZEOF_SHORT=", DEC(sizeof (short)),
  "\nSIZEOF_WCHAR_T=", DEC(sizeof (wchar_t)),
  "\n"
};
!

In this generated program the sizes are encoded as two-digit decimal
strings, at compile time. So the compiled object file will contain
something like "SIZEOF_PTR= 8" surrounded by newlines. The configure
script can look for these strings and get the values out:

  if ! conftest_o ; then   # conftest_o is a function to build the .o
    printf "failed\n\n"

    printf "Errors from compilation: \n\n"
    cat conftest.err
    exit 1
  fi

The script gets the SIZEOF lines out and evals them as shell
assignments. That's why we avoided SIZEOF_PTR=08; that would become
octal in the shell:

  eval $(tr '\0' ' ' < conftest.o  | grep SIZEOF | sed -e 's/  *//')

It also massages these SIZEOFs into header file material:

  tr '\0' ' ' < conftest.o  | grep SIZEOF | sed -e 's/=  */ /' -e 's/^/#define /' >> config.h

  if [ $SIZEOF_PTR -eq 0 -o $SIZEOF_BYTE -eq 0 ] ; then
    printf "failed\n"
    exit 1
  fi

Here is how it then looks like in config.h:

  #define SIZEOF_BYTE 8
  #define SIZEOF_LONGLONG_T 8
  #define SIZEOF_PTR 4
  #define SIZEOF_LONG 4
  #define SIZEOF_INT 4
  #define SIZEOF_SHORT 2
  #define SIZEOF_WCHAR_T 4

There is a minor cross-compiling complication in txr in that you need
txr to compile the standard library. So you must build a native txr
first and then specify TXR=/path/to/native/txr to use that one for
building the standard lib.  Downstream distro people have figured this
out on their own.


--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

[toc] | [prev] | [next] | [standalone]

#3443 — Re: configuguration tools, Portable Software (was: fledgling assembler programmer)

In article <23-03-037@comp.compilers>,
Kaz Kylheku  <864-117-4973@kylheku.com> wrote:
>On 2023-03-28, Aharon Robbins <arnold@freefriends.org> wrote:
>> Today, the C and C++ worlds are easier to program in, but it's still
>> not perfect and I don't think I'd want to do without the autotools.
>> Particularly for the less POSIX-y systems, like MinGW and OpenVMS.
>
>Counterpoint: Autotools are a real detriment to GNU project programs.
>
>When a release is cut of a typical GNU program, special steps
>are execute to prepare a tarball which has a compiled configure
>script.
>
>You cannot just do a "git clone" of a GNU program, and then run
>./configure and build. You must run some "make boostrap" nonsense, and
>that requires you to have various Autotools installed, and in specific
>versions!

This is not inherent in the autotools; it's laziness on the part of the
maintainers. For exactly this reason gawk has a very simple bootstrap.sh
program that simply does a touch on various files so that configure will
run without wanting to run the autotools.

>Most Autotools programs will not cleanly cross-compile. Autotools is the
>main reason why distro build systems use QEMU to create a virtual target
>environment with native tools and libraries, and then build the
>"cross-compiled" program as if it were native.

QEMU wasn't around when the Autotools were first designed and
implemented. Most end users don't need to cross compile either, and it
is for them that I (and other GNU maintainers, I suppose) build my
configure scripts.

Yes, the world is different today than when the autotools were
designed. No, the autotools are not perfect. I don't know of a better
alternative though. And don't tell me CMake. CMake is an abomination,
interweaving configuration with building instead of cleanly separating
the jobs. Not to mention its stupid caching which keeps you from
running a simple "make" after you've changed a single file.

>My TXR language project has a hand-written, not generated, ./configure
>script. What you get in a txr-285.tar.gz tarball is exactly what you
>get if you do a "git clone" and "git checkout txr-285", modulo
>the presence of a .git directory and differing timestamps.
>
>You just ./configure and make.

And for gawk it's ./bootstrap.sh && ./configure && make
where bootstrap.sh only takes a few seconds.

>None of my configure-time tests require the execution of a program;
>For some situations, I have developed clever tricks to avoid it.

And why should you, or anyone, be forced to develop such clever tricks?

All of this simply justifies more the approach taken by newer languages,
which is to move all the hard crap into the libraries.  The language
developers do all the hard work, instead of the application developers
having to do it.  This is great for people who want to just get their
job done, which includes me most of the time.  However, and this is a
different discussion, it does lead to a generation of programmers who
have *no clue* as to how to do the hard stuff should they ever need to.

My opinion, of course.

Arnold
--
Aharon (Arnold) Robbins 		arnold AT skeeve DOT com

[toc] | [prev] | [next] | [standalone]

#3446 — Re: configuguration tools, Portable Software (was: fledgling assembler programmer)

Kaz Kylheku <864-117-4973@kylheku.com> writes:
>When a release is cut of a typical GNU program, special steps
>are execute to prepare a tarball which has a compiled configure
>script.
>
>You cannot just do a "git clone" of a GNU program, and then run
>./configure and build. You must run some "make boostrap" nonsense, and
>that requires you to have various Autotools installed, and in specific
>versions!

And the problem is?

The git repo contains only the source code, useful for developers.
The developers have stuff installed that someone who just wants to
install the program does not necessarily want to install.  E.g., in
the case of Gforth, you need an older Gforth to build the kernel
images that contain Forth code compiled to an intermediate
representation.  Therefore the tarballs contain a number of generated
(or, as you say, "compiled") files, e.g., the configure script, the
kernel images in case of Gforth, or the C files generated by Bison in
case of some other compilers.

If you go for the "git clone" route rather than building from the
tarball, you don't get these amenities, but have to install all the
tools that the developers use, and have to perform an additional step
(usually ./autogen.sh) to produce the configure file.  "make
bootstrap" is unlikely to work, because at that stage you don't have a
Makefile.

I remember "make bootstrap" from gcc, where IIRC it compiles gcc first
(stage1) with the pre-installed C compiler, then (stage2) with the
result of stage1, and finally (stage3) again with the result of
stage2; if there is a difference between stage2 and stage3, something
is amiss.

Anyway, tl;dr: If you just want to do "./configure; make", use the
tarball.

>Most Autotools programs will not cleanly cross-compile. Autotools is tha
>main reason why distro build systems use QEMU to create a virtual target
>environment with native tools and libraries, and then build the
>"cross-compiled" program as if it were native.

Clever!  Let the machine do the work, rather than having to do manual
work for each package.

>For instance, about a decade and a half ago I helped a company
>replace Windriver cruft with an in-house distribution. Windriver's
>cross-compiled Bash didn't have job control! Ctrl-Z, fg, bg stuff no
>workie.  The reason was that it was just cross-compiled straight, on an
>x86 build box.  It couldn't run the test to detect job control support,
>and so it defaulted it off, even though the target machine had
>"gnu-linux" in its string. In the in-house distro, my build steps for
>bash exported numerous ac_cv_... internal variables to override the bad
>defaults.

That's the way to do it.

Your idea seems to be that, when the value is not supplied, instead of
a safe default (typically resulting in not using a feature), one
should base the values on the configuration name of the system.  I
think the main problem with that is that for those systems most in
need of cross-compiling the authors of the tests don't know good
values for the configuration variables; for linux-gnu systems I
usually configure and compile on the system.

>For some situations, I have developed clever tricks to avoid it. For
>instance, if you want to know the size of a data type:. Here
>is a fragment:

Great!  Now we need someone who has enough time to replace the
AC_CHECK_SIZEOF autoconf macro with your technique, and a significant
part of the configuration variables that have to be supplied manually
when cross-configuring Gforth become fully automatic.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

[toc] | [prev] | [next] | [standalone]

#3442 — Re: Portable Software

On 3/28/23 4:42 PM, Aharon Robbins wrote:
> In article <23-03-029@comp.compilers>,
> Hans-Peter Diettrich  <DrDiettrich1@netscape.net> wrote:
>> My impression was that the FSF favors C and ./configure for "portable"
>> code.
>
> Like many things, this is the result of evolution. Autoconf is well
> over 20 years old, and when it was created the ISO C and POSIX standards
> had not yet spread throughout the Unix/Windows/macOS world. It and the
> rest of the autotools solved a real problem.

About 20 years ago I could not build any open source program on Windows.
Messages like "Compiler can not build executables" popped up when using
MinGW or Cygwin. I ended up in ./configure in a Linux VM and fixing the
resulting compiler errors manually on Windows. Without that trick I had
no chance to load the "portable" source code into any development
environment for inspection in readable (compilable) form. Often I had
the impression that the author wanted the program not for use on Windows
machines. Kind of "source open for specific OS only" :-(

DoDi

[toc] | [prev] | [next] | [standalone]

#3447 — Re: Portable Software

Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>Often I had
>the impression that the author wanted the program not for use on Windows
>machines. Kind of "source open for specific OS only" :-(

Whatever we want, it's also a question of what the OS vendor wants.

For a Unix, there were a few hoops we had to jump through to make
Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
workaround for that; HP/UX's make dealt with files with the same mtime
differently from other makes, so we put in a workaround for that.
Windows, even with Cygwin, puts up many more hoops to jump through;
Bernd Paysan actually jumped through them for Gforth, but a Windows
build is still quite a bit of work, so he does that only occasionally.

It's no surprise to me that other developers don't jump through these
hoops; maybe if someone payed them for it, but why should they do it
on their own time?

As a recent example of another OS, Apple has intentionally reduced the
functionality of mmap() on MacOS on Apple silicon compared to MacOS on
Intel.  As a result, the development version of Gforth does not work
on MacOS on Apple Silicon (it works fine on Linux on Apple Silicon).
I spent a day last summer on the MacOS laptop of a friend (an
extremely unpleasant experience) trying to find the problem and fix
it, and I found the problem, but time ran out before I had a working
fix (it did not help that I had to spend a lot of time on working
around things that I missed in MacOS).  Since then this problem has
not reached the top of my ToDo list; and when it does, I will go for
the minimal fix, with the result that Gforth on MacOS will run without
dynamic native-code generation, i.e., slower than on Linux.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

[toc] | [prev] | [next] | [standalone]

#3448 — Re: Portable Software

On 4/2/23 12:04 PM, Anton Ertl wrote:

> For a Unix, there were a few hoops we had to jump through to make
> Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
> workaround for that; HP/UX's make dealt with files with the same mtime
> differently from other makes, so we put in a workaround for that.
> Windows, even with Cygwin, puts up many more hoops to jump through;
> Bernd Paysan actually jumped through them for Gforth, but a Windows
> build is still quite a bit of work, so he does that only occasionally.

Too bad that not all existing OS are POSIX compatible? ;-)

So my impression still is: have a language (plus library) and an
interpreter (VM, browser, compiler...) on each target system. Then
adaptations to a target system have to be made only once, for each
target, not for every single program.

Even for programs with extreme speed requirements the development can be
done from the general implementation, for tests etc., and a version
tweaked for a very specific target system, instead of the single target
version in the first place and problematic ports to many other platforms.

Of course it's up to the software developer or principal to order or
build a software for a (more or less) specific target system only, or a
primarily unbound software.

(G)FORTH IMO is a special case because it's (also) a development system.
Building (bootstrapping) a new FORTH system written in FORTH is quite
complicated, in contrast to languages with stand alone tools like
compiler, linker etc. Some newer (umbilical?) FORTH versions also
compile to native code.

DoDi

[toc] | [prev] | [next] | [standalone]

#3449 — Re: Portable Software

Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>On 4/2/23 12:04 PM, Anton Ertl wrote:
>
>> For a Unix, there were a few hoops we had to jump through to make
>> Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
>> workaround for that; HP/UX's make dealt with files with the same mtime
>> differently from other makes, so we put in a workaround for that.
>> Windows, even with Cygwin, puts up many more hoops to jump through;
>> Bernd Paysan actually jumped through them for Gforth, but a Windows
>> build is still quite a bit of work, so he does that only occasionally.
>
>Too bad that not all existing OS are POSIX compatible? ;-)

Like many standards, POSIX is a subset of the functionality that
programs use.  Windows NT used to have a POSIX subsystem in order to
make WNT comply with FIPS 151-2 needed to make WNT eligible for
certain USA government purchases.  From what I read, it was useful for
that, but not much else.

>So my impression still is: have a language (plus library) and an
>interpreter (VM, browser, compiler...) on each target system. Then
>adaptations to a target system have to be made only once, for each
>target, not for every single program.

You mean: Write your program in Java, Python, Gforth, or the like?
Sure, they deal with compatibility problems for you, but you may want
to do things (or have performance) that they do not offer, or only
offer through a C interface (and in the latter case you run into the
C-level compatibility again).

>Even for programs with extreme speed requirements the development can be
>done from the general implementation, for tests etc., and a version
>tweaked for a very specific target system, instead of the single target
>version in the first place and problematic ports to many other platforms.

Well, if you go that route, the result can easily be that your program
does not run on Windows.  Especially for GNU programs: The primary
goal is that they run on GNU.  Any effort spent on a Windows port is
extra effort that not everybody has time for.

>(G)FORTH IMO is a special case because it's (also) a development system.
>Building (bootstrapping) a new FORTH system written in FORTH is quite
>complicated, in contrast to languages with stand alone tools like
>compiler, linker etc.

Not really.  Most self-respecting languages have their compiler(s)
implemented in the language itself, resulting in having to bootstrap.
AFAIK the problem Gforth has with Windows is not the bootstrapping;
packaging and installation are different than for Unix.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.compilers

csiph-web