Groups | Search | Server Info | Login | Register

Groups > comp.arch.embedded > #32419

Re: Improving build system

Show all headers | View raw

On 13/05/2025 17:57, pozz wrote:
> As some of you remember, some weeks ago we had a discussion on the build 
> system of an embedded project.  I declared I usually use the graphical 
> IDE released by silicon manufacturer (in my case, Atmel Studio and 
> MCUXpresso IDE) and some of you suggested to improve this build system 
> using a command line tool, such as the universal make.  Better if used 
> in Linux or Linux-based system, such as msys or WSL.
> 
> I also declared I had many issues fine tuning a cross-platform Makefile 
> that works in Windows and Linux-like shells at the same time and some of 
> you suggested to use only WSL or msys, not Windows CMD shell.
> 
> Recently I found some time to try again and I wrote a decent Makefile as 
> a starting point.  Now the command make, launched in a msys/mingw32 
> shell, is able to build my project, at least a specific build 
> configuration, what I name "simulator".
> 
> My projects usually have multiple build configurations. A few for 
> different models of the same device, such as LITE, NORMAL and FULL.
> Moreover, I have at least two different targets: embedded and simulator. 
> The embedded target is the normal product, usually running on a Cortex-M 
> or AVR8 MCU. The simulator target runs directly on Windows. I use it 
> very often, because I found it's much faster and simpler to build native 
> binaries and debug such processes. Of course, building a simulator needs 
> a different compilers, such as msys2/mingw32 or WSL/gcc.
> I also have a DEBUG build configuration (target=embedded) useful for 
> some debugging directly on the target (no watchdog, uart logging enabled 
> and so on).
> 
> So I could have 7 different build configurations: LITE|NORMAL|FULL for 
> EMBEDDED|SIMULATOR plus DEBUG.
> 
> I think it isn't difficult to change my Makefile to process commands of 
> type:
> 
>     make CONFIG=LITE TARGET=embedded
>     make CONFIG=FULL TARGET=simulator
>     make CONFIG=DEBUG
> 
> There are many compiler options that are common to all builds (-Wall, 
> -std=c99 and so on).  Some options are target specific (for example 
> -DQUARTZ_FREQ_MHZ=16 -Isrc/ports/avr8 for embedded or 
> -Isrc/ports/mingw32 for simulator).
> 
> I could generate the correct options by using ifeq() in Makefile.
> 
> How to choose the correct toolchain? Embedded target needs arm-gcc 
> toolchain, for example in
> 
>    C:\Program Files 
> (x86)\Atmel\Studio\7.0\toolchain\arm\arm-gnu-toolchain\bin
> 
> while simulator targets needs simply gcc.
> 
> How do you choose toolchain in Makefile?  I think one trick is using the 
> prefix.  Usually arm-gcc is arm-none-eabi-gcc.exe, with "arm-none-eabi-" 
> prefix.  Is there other approach?
> 
> I don't know if I could install arm-gcc in msys2 (I'm quite sure I can 
> install it in WSL), but for legacy projects I need to use the Atmel 
> Studio toolchain.  How to call Atmel Studio arm toolchain installed in
> 
>    C:\Program Files 
> (x86)\Atmel\Studio\7.0\toolchain\arm\arm-gnu-toolchain\bin
> 
> from msys shell?  Should I change the PATH and use arm-none-eabi- prefix?
> 

You are asking a lot of questions here.  They are good questions, but it 
would be a very long post if I tried to answer them all fully.  So 
instead, I will try to give a few hints and suggestions that you can 
take further.  I'll put numbers on them in case you want to reference 
them in replies.

1.

Windows path naming is insane.  Fortunately, you can almost always 
override it.  Whenever you install any serious program in Windows, 
especially if you ever want to refer to it from the command line, batch 
files, makefiles, etc., avoid names with spaces or "awkward" characters. 
  I recommend making top-level directories like "progs" or "compilers" 
and putting the tools in there as appropriate.  This also makes it 
vastly easier to copy tools to other machines.  And since you should 
never upgrade your toolchains - merely add new versions to your 
collection, in separate directories - it is easier if they are better 
organised.


2.

You don't need to use bash or other *nix shells for makefile or other 
tools if you don't want to.  When I do builds on Windows, I run "make" 
from a normal command line (or from an editor / IDE).  It is helpful to 
have msys2's usr/bin on your path so that make can use *nix command-line 
utilities like cp, mv, sed, etc.  But if you want to make a minimal 
build system, you don't need a full msys2 installation - you only need 
the utilities you want to use, and they can be copied directly (unlike 
with Cygwin or WSL).

Of course you /can/ use fuller shells if you want.  But don't make your 
makefiles depend on that, as it will be harder to use them from IDEs, 
editors, or any other automation.

And of course you will want an msys2/mingw64 (/not/ old mingw32) for 
native gcc compilation.  Don't bother with WSL unless you actually need 
a fuller Linux system - and if you /do/ need that, dump the wasteful 
crap that is Windows and use Linux for your development.  Build speeds 
will double on the same hardware.  (In my testing, done a good while 
back, I did some comparisons of a larger build on different setups on 
the same PC, using native Windows build as the baseline.  Native Linux 
builds were twice the speed.  Running VirtualBox on Windows host, with a 
Linux virtual machine, or running VirtualBox on Linux with a Windows 
virtual machine, both beat native Windows solidly.)


3.

Makefiles can be split up.  Use "include" - and remember that you can do 
so using macros.  In my makefile setups, I have a file "host.mk" that is 
used to identify the build host, then pull in a file that is specific to 
the host:

# This is is for identifying host computer to get the paths right

ifeq ($(OS),Windows_NT)
   # We are on a Windows machine
   host_os := windows
   host := $(COMPUTERNAME)
else
   # Linux machine
   host_os := linux
   host := $(shell hostname)
endif

ifeq "$(call file-exists,makes/host_$(host).mk)" "1"
   include makes/host_$(host).mk
else
   $(error No host makefile host_$(host).mk found)
endif

Then I have files like "host_xxx.mk" for a computer named "xxx", 
containing things like :

toolchain_path := /opt/gcc-arm-none-eabi-10-2020-q4-major/bin/

or

toolchain_path := c:/micros/gcc-arm-none-eabi-10_2020-q4-major/bin/


All paths to compilers and other build-related programs are specified in 
these files.  The only things that are taken from the OS path are 
standard and common programs that do not affect the resulting binary files.


Then I have a "commands.mk" file with things like :

ATDEP := @

toolchain_prefix := arm-none-eabi-

CCDEP := $(ATDEP)$(toolchain_path)$(toolchain_prefix)gcc
CC := $(AT)$(CCACHE) $(toolchain_path)$(toolchain_prefix)gcc
LD := $(AT)$(toolchain_path)$(toolchain_prefix)gcc
OBJCOPY := $(AT)$(CCACHE) $(toolchain_path)$(toolchain_prefix)objcopy
OBJDUMP := $(AT)$(CCACHE) $(toolchain_path)$(toolchain_prefix)dump
SIZE := $(AT)$(CCACHE) $(toolchain_path)$(toolchain_prefix)size


Put CONFIG dependent stuff in "config_full.mk" and similar files.  Put 
TARGET specific stuff in "target_simulator.mk".  And so on.  It makes it 
much easier to keep track of things, and you only need a few high-level 
"ifeq".


Keep your various makefiles in a separate directory.  Your project 
makefile is then clear and simple - much of it will be comments about 
usage (parameters like CONFIG).


4.

Generate dependency files, using the same compiler and the same include 
flags and -D flags as you have for the normal compilation, but with 
flags like -MM -MP -MT and -MF to make .d dependency files.  Include 
them all in the makefile, using "-include" so that your makefile does 
not stop before they are generated.


5.

Keep your build directories neat, separate from all source directories, 
and mirroring the tree structure of the source files.  So if you have a 
file "src/gui/main_window.c", and you are building with CONFIG=FULL 
TARGET=embedded, the object file generated should go in something akin 
to "builds/FULL/embedded/obj/src/gui/main_window.o".  I like to have 
separate parts for obj (.o files), dep (.d files), and bin (linked 
binaries, map files, etc.).  You could also mix .d and .o files in the 
same directory if you prefer.

This means you can happily do incremental builds for all your 
configurations and targets, and don't risk mixing object files from 
different setups.


6.

Learn to use submakes.  When you use plain "make" (or, more 
realistically, "make -j") to build multiple configurations, have each 
configuration spawned off in a separate submake.  Then you don't need to 
track multiple copies of your "TARGET" macro in the same build - each 
submake has just one target, and one config.

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Improving build system pozz <pozzugno@gmail.com> - 2025-05-13 17:57 +0200
  Re: Improving build system Nicolas Paul Colin de Glocester <Spamassassin@irrt.De> - 2025-05-13 22:48 +0200
  Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-14 11:03 +0200
    Re: Improving build system George Neuner <gneuner2@comcast.net> - 2025-05-14 15:21 -0400
      Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-15 09:48 +0200
      Re: Improving build system Nioclás Pól Caileán de Ghloucester <Spamassassin@irrt.De> - 2025-07-04 18:38 +0200
    Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-14 23:51 +0200
      Re: Improving build system Nicolas Paul Colin de Glocester <Spamassassin@irrt.De> - 2025-05-15 01:00 +0200
        Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-15 11:17 +0200
          Re: Improving build system Nicolas Paul Colin de Glocester <Spamassassin@irrt.De> - 2025-05-16 12:21 +0200
            Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-16 14:42 +0200
      Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-15 11:03 +0200
        Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-15 23:25 +0200
          Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-16 11:12 +0200
            Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-16 12:46 +0200
              Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-16 15:30 +0200
                Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-16 16:17 +0200
    Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-16 15:45 +0200
      Re: Improving build system David Brown <david.brown@hesbynett.no> - 2025-05-16 17:20 +0200
        Re: Improving build system pozz <pozzugno@gmail.com> - 2025-05-17 10:56 +0200
  Re: Improving build system Stefan Reuther <stefan.news@arcor.de> - 2025-05-14 18:06 +0200

csiph-web