Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #62724 > unrolled thread

why is the build so complex ?

Back to article view | Back to linux.debian.kernel

Contents

  why is the build so complex ? "Enrico Weigelt, metux IT consult" <lkml@metux.net> - 2018-12-14 14:20 +0100
    Re: why is the build so complex ? Bjørn Mork <bjorn@mork.no> - 2018-12-14 15:10 +0100
    Re: why is the build so complex ? Yves-Alexis Perez <corsac@debian.org> - 2018-12-14 16:00 +0100
    Re: [devuan-dev] why is the build so complex ? Nico Schottelius <nico.schottelius@ungleich.ch> - 2018-12-14 22:20 +0100
    Re: why is the build so complex ? Ben Hutchings <ben@decadent.org.uk> - 2018-12-15 16:40 +0100
      Re: why is the build so complex ? "Enrico Weigelt, metux IT consult" <lkml@metux.net> - 2018-12-17 13:20 +0100
        Re: why is the build so complex ? Ben Hutchings <ben@decadent.org.uk> - 2018-12-17 16:30 +0100

#62724 — why is the build so complex ?

Hi folks,


while trying to build a debian kernel w/ some minor config changes
(actually, just need the gpio keyboard modules), I've stumpled across
several strange problems and finally found out that the patches
haven't been applied on the main tree (in contrast to rt tree) .. this
ate up a lot of time, as the build takes so long (unfortunately, I don't
have the luxory of possessing a 120 core machine ;-)).

When I look at the build rules, I really wonder why it's all so complex.

Some things that IMHO contribute to that complexity:

* several source trees - rt vs none:
  --> can't we do this with one tree ?
  --> if necessary, fixup the rt patches, so they don't do anything in
      non-rt build (eg. proper #ifdef's) ?
* arch specific patches - same story like rt
* auxillary tools
  --> can't we build them completely separately ?
      (IMHO, shouldn't have an hard dependency on currently running
      kernel)

If it's just a matter of resources, I'd like to join in and do the job.


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287

[toc] | [next] | [standalone]

#62725

"Enrico Weigelt, metux IT consult" <lkml@metux.net> writes:

> while trying to build a debian kernel w/ some minor config changes
> (actually, just need the gpio keyboard modules),

Can't help with answers to your questions, but i believe I can help with
this task.  It's as simple as

 $ git clone https://github.com/notro/fbtft_tools.git
 Cloning into 'fbtft_tools'...
 remote: Enumerating objects: 95, done.
 remote: Total 95 (delta 0), reused 0 (delta 0), pack-reused 95
 Unpacking objects: 100% (95/95), done.
 $ cd fbtft_tools/gpio_keys_device/
 $ make
 make -C /lib/modules/4.18.0-3-amd64/build M=/usr/local/src/git/fbtft_tools/gpio_keys_device modules
 make[1]: Entering directory '/usr/src/linux-headers-4.18.0-3-amd64'
   CC [M]  /usr/local/src/git/fbtft_tools/gpio_keys_device/gpio_keys_device.o
   Building modules, stage 2.
   MODPOST 1 modules
   CC      /usr/local/src/git/fbtft_tools/gpio_keys_device/gpio_keys_device.mod.o
   LD [M]  /usr/local/src/git/fbtft_tools/gpio_keys_device/gpio_keys_device.ko
 make[1]: Leaving directory '/usr/src/linux-headers-4.18.0-3-amd64'


provided you have installed the linux-headers package matching your
kernel.  No need to care about the Debian kernel patches and build
infrastructure at all.

Or did you mean something else?


Bjørn

[toc] | [prev] | [next] | [standalone]

#62726

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, 2018-12-14 at 14:13 +0100, Enrico Weigelt, metux IT consult wrote:
> When I look at the build rules, I really wonder why it's all so complex.

Hi,

there is no simple answer to that. /Maybe/ the build system could be
simplified a bit, but in the end the src:linux needs to build a lot of complex
stuff, needs to be generic/modular etc.

If you really want a simple way to rebuild one kernel, maybe you can just look
at make bindeb-pkg target of the upstream kernel Makefile?

I don't really think it makes sense for an end-user to maintain a local fork
of src:linux.

Regards,
- -- 
Yves-Alexis
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAlwTxTYACgkQ3rYcyPpX
RFs0AggAmltKGCjTf6mFwRHxCpF1pqnMhmWzMkKUMUIL5MBGar3OtJECT2tPtJ3X
u0HkSygxUS8t0V5X5Atz3B1SLLeGe//OlOFKfCn4rdFaF7rPC+nFaZI4X0dg/kja
qmo13wdmeNtlFu92YGr5DEgdaHTGSL3nsuG+PNTmYp6yR/geC+T1sHwyE1gHuumC
t5PJKtOiwFhHhVv4m9bU8q4jbkELjfrt4X07v5jemh+JxMuH4JNgTu/CS03kWzXL
780kVLd4+KcdqbE/yopYYIWMti8Pk1D1LIVlRae9PzxYcXIxTCmmIzJu5ulAlkQb
qjlzupUytAPxC4zQ+DMj0kyu/8j2ww==
=0tR4
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#62728 — Re: [devuan-dev] why is the build so complex ?

Hey mtx,

would ungleich providing a beefy(*) VM for kernel builds make your life
easier or more productive for Devuan?

If so, we might be very open for sponsoring.

Best,

Nico

(*) Not sure if this is still politically correct, otherwise read it as
"full of tasty vegetables", please.


Enrico Weigelt, metux IT consult <lkml@metux.net> writes:

> Hi folks,
>
>
> while trying to build a debian kernel w/ some minor config changes
> (actually, just need the gpio keyboard modules), I've stumpled across
> several strange problems and finally found out that the patches
> haven't been applied on the main tree (in contrast to rt tree) .. this
> ate up a lot of time, as the build takes so long (unfortunately, I don't
> have the luxory of possessing a 120 core machine ;-)).
>
> When I look at the build rules, I really wonder why it's all so complex.
>
> Some things that IMHO contribute to that complexity:
>
> * several source trees - rt vs none:
>   --> can't we do this with one tree ?
>   --> if necessary, fixup the rt patches, so they don't do anything in
>       non-rt build (eg. proper #ifdef's) ?
> * arch specific patches - same story like rt
> * auxillary tools
>   --> can't we build them completely separately ?
>       (IMHO, shouldn't have an hard dependency on currently running
>       kernel)
>
> If it's just a matter of resources, I'd like to join in and do the job.
>
>
> --mtx


--
Your Swiss, Open Source and IPv6 Virtual Machine. Now on www.datacenterlight.ch.

[toc] | [prev] | [next] | [standalone]

#62741

[Multipart message — attachments visible in raw view] — view raw

On Fri, 2018-12-14 at 14:13 +0100, Enrico Weigelt, metux IT consult wrote:
> Hi folks,
> 
> 
> while trying to build a debian kernel w/ some minor config changes
> (actually, just need the gpio keyboard modules), I've stumpled across
> several strange problems and finally found out that the patches
> haven't been applied on the main tree (in contrast to rt tree) .. this
> ate up a lot of time, as the build takes so long (unfortunately, I don't
> have the luxory of possessing a 120 core machine ;-)).

There is a script to do what you want, debian/bin/test-patches.

> When I look at the build rules, I really wonder why it's all so complex.
> 
> Some things that IMHO contribute to that complexity:
> 
> * several source trees - rt vs none:
>   --> can't we do this with one tree ?
>   --> if necessary, fixup the rt patches, so they don't do anything in
>       non-rt build (eg. proper #ifdef's) ?

That would be a huge amount of work to maintain.

> * arch specific patches - same story like rt

There aren't arch-specific patches - that is, they aren't applied only
when building a particular arch.

> * auxillary tools
>   --> can't we build them completely separately ?
>       (IMHO, shouldn't have an hard dependency on currently running
>       kernel)

They can be disabled with the "pkg.linux.notools" profile.

> If it's just a matter of resources, I'd like to join in and do the job.

If you want to help simplify - and I'm sure some simplification is
possible - you have to start by understanding why the complexity is
there in the first place.

Ben.

-- 
Ben Hutchings
If you seem to know what you are doing, you'll be given more to do.

[toc] | [prev] | [next] | [standalone]

#62751

On 15.12.18 16:37, Ben Hutchings wrote:

Hi,

>> while trying to build a debian kernel w/ some minor config changes>> (actually, just need the gpio keyboard modules), I've stumpled
across>> several strange problems and finally found out that the
patches>> haven't been applied on the main tree (in contrast to rt tree)
.. this>> ate up a lot of time, as the build takes so long
(unfortunately, I don't>> have the luxory of possessing a 120 core
machine ;-)).> > There is a script to do what you want,
debian/bin/test-patches.
Shouldn't the patches be automatically applied when calling the 'binary'
rule (./debian/rules) ? If not, this doesn't seem to make much sense :o

I'm still in the middle of hacking, what I did so far is changing the
build tree for 'none' to a separate directory, just like for 'rt', using
the same rules. (essentially dropping special rules for
$(STAMPS_DIR)/source_none, etc) - now the patches are applied.

The only downside of this change I see so far is that I get yet another
temporary copy of the full source tree. OTOH, I'm thinking about
dropping the initial unpack (in the root dir) and change these rules
to directly unpack from tarball (or optionally pull a git repo).

>> When I look at the build rules, I really wonder why it's all so complex.>>>> Some things that IMHO contribute to that complexity:>>>> * several
source trees - rt vs none:>>   --> can't we do this with one tree ?>>
--> if necessary, fixup the rt patches, so they don't do anything in>>
     non-rt build (eg. proper #ifdef's) ?> > That would be a huge amount
of work to maintain.
Initial work or permanent maintenance ?
I agree that this takes some initial effort, and I'd like to volounteer
for that.

BTW: On a quick look, I've seen that patches from
'debian/patches/series' are not included in in series-rt. Are they
supposed to be
already applied, before the rt tree copy is made ?

>> * auxillary tools>>   --> can't we build them completely separately ?>>       (IMHO,
shouldn't have an hard dependency on currently running>>       kernel)>
> They can be disabled with the "pkg.linux.notools" profile.
Yes, but then they aren't built at all. Fine for my current local case,
but the distro still needs them. So, my proposal is making them
completely own source packages - then we wouldn't need play around w/
profiles anymore and therefore reduce complexity.

>> If it's just a matter of resources, I'd like to join in and do the job.> > If you want to help simplify - and I'm sure some simplification is>
possible - you have to start by understanding why the complexity is>
there in the first place.
Yes, that's why I've started this thread: to ask you guys what that's
really all about :)



--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287

[toc] | [prev] | [next] | [standalone]

#62753

[Multipart message — attachments visible in raw view] — view raw

On Mon, 2018-12-17 at 13:16 +0100, Enrico Weigelt, metux IT consult wrote:
> On 15.12.18 16:37, Ben Hutchings wrote:
> 
> Hi,
> 
> > > while trying to build a debian kernel w/ some minor config changes>> (actually, just need the gpio keyboard modules), I've stumpled
> across>> several strange problems and finally found out that the
> patches>> haven't been applied on the main tree (in contrast to rt tree)
> .. this>> ate up a lot of time, as the build takes so long
> (unfortunately, I don't>> have the luxory of possessing a 120 core
> machine ;-)).> > There is a script to do what you want,
> debian/bin/test-patches.
> Shouldn't the patches be automatically applied when calling the 'binary'
> rule (./debian/rules) ? If not, this doesn't seem to make much sense :o

If you build from git, the patches are applied by "debian/rules orig",
or if you build from a source package they are applied by
"dpkg-source -x" (by default).

debian/bin/test-patches is a simplified way to add more patches and
build a single kernel flavour, for testing purposes.

> I'm still in the middle of hacking, what I did so far is changing the
> build tree for 'none' to a separate directory, just like for 'rt', using
> the same rules. (essentially dropping special rules for
> $(STAMPS_DIR)/source_none, etc) - now the patches are applied.
> 
> The only downside of this change I see so far is that I get yet another
> temporary copy of the full source tree. OTOH, I'm thinking about
> dropping the initial unpack (in the root dir) and change these rules
> to directly unpack from tarball (or optionally pull a git repo).

What would be the benefit of this?

> > > When I look at the build rules, I really wonder why it's all so complex.>>>> Some things that IMHO contribute to that complexity:>>>> * several
> source trees - rt vs none:>>   --> can't we do this with one tree ?>>
> --> if necessary, fixup the rt patches, so they don't do anything in>>
>      non-rt build (eg. proper #ifdef's) ?> > That would be a huge amount
> of work to maintain.
> Initial work or permanent maintenance ?
> I agree that this takes some initial effort, and I'd like to volounteer
> for that.
> 
> BTW: On a quick look, I've seen that patches from
> 'debian/patches/series' are not included in in series-rt. Are they
> supposed to be
> already applied, before the rt tree copy is made ?

Correct.  The "$(STAMPS_DIR)/source" target in debian/rules.real has a
check for this.

> > > * auxillary tools>>   --> can't we build them completely separately ?>>       (IMHO,
> shouldn't have an hard dependency on currently running>>       kernel)>
> > They can be disabled with the "pkg.linux.notools" profile.
> Yes, but then they aren't built at all. Fine for my current local case,
> but the distro still needs them. So, my proposal is making them
> completely own source packages - then we wouldn't need play around w/
> profiles anymore and therefore reduce complexity.

We used to do this (linux-tools source package).  It required more work
to maintain.

> > > If it's just a matter of resources, I'd like to join in and do the job.> > If you want to help simplify - and I'm sure some simplification is>
> possible - you have to start by understanding why the complexity is>
> there in the first place.
> Yes, that's why I've started this thread: to ask you guys what that's
> really all about :)

OK.  Many of your questions are likely to be answered in
debian/README.source or in the debian-kernel-handbook, so you should at
least browse through those.

Ben.

-- 
Ben Hutchings
Anthony's Law of Force: Don't force it, get a larger hammer.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web