Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #81323 > unrolled thread

Immediate fallouts from the big linux changes, and actions

Back to article view | Back to linux.debian.kernel

Contents

  Immediate fallouts from the big linux changes, and actions Cyril Brulebois <kibi@debian.org> - 2023-12-24 08:40 +0100
    Re: Immediate fallouts from the big linux changes, and actions Cyril Brulebois <kibi@debian.org> - 2023-12-24 09:20 +0100
      Re: Immediate fallouts from the big linux changes, and actions Michael Tokarev <mjt@tls.msk.ru> - 2023-12-24 09:40 +0100
        Re: Re: Immediate fallouts from the big linux changes, and actions Luca Boccassi <bluca@debian.org> - 2023-12-24 11:40 +0100
    Re: Immediate fallouts from the big linux changes, and actions Bastian Blank <waldi@debian.org> - 2023-12-24 13:10 +0100
      Re: Immediate fallouts from the big linux changes, and actions Cyril Brulebois <kibi@debian.org> - 2023-12-24 17:40 +0100
    Re: Immediate fallouts from the big linux changes, and actions Holger Levsen <holger@layer-acht.org> - 2023-12-25 12:50 +0100
      Re: Immediate fallouts from the big linux changes, and actions Holger Levsen <holger@layer-acht.org> - 2023-12-29 12:20 +0100

#81323 — Immediate fallouts from the big linux changes, and actions

[Multipart message — attachments visible in raw view] — view raw

Hi,

This is mostly for information: linux went through a lot of big changes,
initially staged in experimental, and uploaded to unstable as of linux
6.6.8-1. These impacted the installer in different ways. This is a
summary of the obvious impacts, there might be some others down the line
(just by glancing at hw-detect's massaging of module list by stripping
the .ko suffix).

- kernel-image-* packages are now shipping /boot/vmlinuz-* (or
  /boot/vmlinux-* depending on the arch), instead of just /boot/vmlinuz
  (respectively /boot/vmlinux). I didn't dive into all the changes but
  I'm not sure whether that was on purpose or a side effect of the big
  changes. In any case, on the short term, since the build system was
  relying on a single file there, that should be moved out of the way,
  I decided to unversion those files to restore the old names. If kernel
  maintainers decide to go back to unversioned files, two reverts should
  be enough:
    https://salsa.debian.org/installer-team/debian-installer/-/commit/c17360e632b6f133d4b43a47461944b6203ed730
    https://salsa.debian.org/installer-team/debian-installer/-/commit/865c66e497d8e809872bad04aa2269d2a9f889c8

- Modules are compressed now, so the drm workaround needed an updated to
  cope with the extra .xz suffix:
    https://salsa.debian.org/installer-team/debian-installer/-/commit/bd0f1106f90756e6f4514108492d71e1f2e695ea

- I'm skipping two commits regarding the new way the ABI is written
  down, that's in line with what was announced initially.

- YIL: If you don't see anything move in the graphical installer (no
  keyboard, no mouse), if your video is garbled, and if you can't load a
  NIC module in the text installer, you might have a totally broken
  support for kernel modules in your installer. This required adjusting
  src:xz-utils to get a liblzma udeb and src:kmod to enable xz support
  in its udebs; “thankfully” the former is NMU-maintained, plus the ftp
  team speed-reviewed it from NEW; thankfully Marco ACK'd my NMU-ing
  kmod so that we unbreak things ASAP. Everything should be sorted out
  in unstable. Plus an extra commit to be extra careful between the
  kmod upload and the cronned daily builds (which were going to start
  before the 01:52Z dinstall):
    https://tracker.debian.org/news/1489255/accepted-xz-utils-545-02-source-amd64-all-into-unstable/
    https://tracker.debian.org/news/1489268/accepted-xz-utils-545-03-source-into-unstable/
    https://salsa.debian.org/md/kmod/-/merge_requests/13
    https://tracker.debian.org/news/1489283/accepted-kmod-3020230601-21-source-into-unstable/
    https://salsa.debian.org/installer-team/debian-installer/-/commit/c65bdb8128bc68cb566b302feb46267e5c177eb4

- Finally, the armel build fails because it can't find its kernel. The
  marvell flavour seems to have been dropped entirely (at least that's
  how I read the linux changelog for 6.6.3-1~exp1:
    https://tracker.debian.org/news/1482751/accepted-linux-663-1exp1-source-into-experimental/

I didn't investigate the last point and I'd appreciate it if porters
could double check that we really want to drop marvell at this point,
and maybe submit (or commit) patches to drop it from the installer:
  https://salsa.debian.org/installer-team/debian-installer

Various netboot and netboot-gtk checks on amd64 seem very positive
after those changes, but I haven't built a netinst image and checked
on actual hardware with the combination of Secure Boot + firmware
packages + reloading of kernel modules. The latter should primarily work
by using module *names* and firmware *paths* as seen in kernel logs, so
having different module *paths* shouldn't make a difference, but that's
just my gut feeling based on my recollection (so nobody should trust
me on this), and seeing how \.ko$ stripping appears in some other files,
I cannot be certain until it's been actually tested.

Feel free to reply to this thread if you spot other fallouts. Of course,
as usual, the best way to report installer bugs is to file them against
the relevant component, X-Debbugs-Cc-ing this list; but leaving an
explicit reference in this thread wouldn't hurt.


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [next] | [standalone]

#81324

[Multipart message — attachments visible in raw view] — view raw

Cyril Brulebois <kibi@debian.org> (2023-12-24):
> Feel free to reply to this thread if you spot other fallouts.

Another FYI: one thing that I spotted early on (before realizing module
support was entirely broken), when I first diffed the list of files
available in the initramfs, building against testing's or unstable's
linux udebs: a bunch of virtio modules disappeared.

Comparing 6.5.0-5-amd64 and 6.6.8-amd64:

     usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_balloon.ko
     usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_dma_buf.ko
     usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_input.ko
    -usr/lib/modules/$ABI/kernel/drivers/virtio/virtio.ko
     usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_mmio.ko
    -usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_pci.ko
    -usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_pci_legacy_dev.ko
    -usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_pci_modern_dev.ko
    -usr/lib/modules/$ABI/kernel/drivers/virtio/virtio_ring.ko

A cursory look in linux.git didn't yield an obvious answer:

    git diff debian/6.5.13-1..debian/6.6.8-1 -- debian/installer/

Indeed, that's mostly about kernel-versions's getting dropped, fuse
going away, some Depends field being updated, and some i2c-hid-* now
getting excluded in one place, and added conditionally in another one…

Searching for information about fuse and virtio, I finally noticed this
entry, which probably explains both fuse's “going away” and ditto for
some (but not all) virtio modules:

    * Set CONFIG_VIRTIO_FS and its dependencies to builtin, to allow building
      images that boot directly to rootfs (skipping the initrd)

as it changes:

    -CONFIG_VIRTIO_PCI=m
    +CONFIG_VIRTIO_PCI=y
    -CONFIG_FUSE_FS=m
    +CONFIG_FUSE_FS=y
    -CONFIG_VIRTIO_FS=m
    +CONFIG_VIRTIO_FS=y

I thought I'd mention the answer by mail since I brought up the topic
earlier (even if only on IRC).


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [prev] | [next] | [standalone]

#81325

24.12.2023 11:16, Cyril Brulebois :
...
> Searching for information about fuse and virtio, I finally noticed this
> entry, which probably explains both fuse's “going away” and ditto for
> some (but not all) virtio modules:
> 
>      * Set CONFIG_VIRTIO_FS and its dependencies to builtin, to allow building
>        images that boot directly to rootfs (skipping the initrd)
> 
> as it changes:
> 
>      -CONFIG_VIRTIO_PCI=m
>      +CONFIG_VIRTIO_PCI=y
>      -CONFIG_FUSE_FS=m
>      +CONFIG_FUSE_FS=y
>      -CONFIG_VIRTIO_FS=m
>      +CONFIG_VIRTIO_FS=y

Hm. This same argument can be used to include every storage- and filesystem-related
module into the kernel.  Why don't we have ahci and sd_mod built-in?  This does
look quite a bit strange to me to include this stuff..

(This commit is not about big linux changes but about small debian changes ;)

/mjt

[toc] | [prev] | [next] | [standalone]

#81326

[Multipart message — attachments visible in raw view] — view raw

> Hm. This same argument can be used to include every storage- and
> filesystem-related
> module into the kernel.  Why don't we have ahci and sd_mod built-in?
> This does
> look quite a bit strange to me to include this stuff..

If you have a block device then there are very little advantages in
terms of workflows, as it's still going through a lot of slow paths.

This on the other hand enables the virtiofsd + qemu direct boot fast
path, which is increasingly popular for fast development cycles, and
changing those kconfigs brings us in line with other distros.

-- 
Kind regards,
Luca Boccassi

[toc] | [prev] | [next] | [standalone]

#81327

On Sun, Dec 24, 2023 at 08:38:57AM +0100, Cyril Brulebois wrote:
> - kernel-image-* packages are now shipping /boot/vmlinuz-* (or
>   /boot/vmlinux-* depending on the arch), instead of just /boot/vmlinuz
>   (respectively /boot/vmlinux).

This was even dependent on architecture.  A lot of secondary
architectures already had various suffixes.

> - Modules are compressed now, so the drm workaround needed an updated to
>   cope with the extra .xz suffix:
>     https://salsa.debian.org/installer-team/debian-installer/-/commit/bd0f1106f90756e6f4514108492d71e1f2e695ea

This now hardcodes .xz.  And I'm not really sure what this does and why
this can't be fixed in the kernel packages.

> - Finally, the armel build fails because it can't find its kernel. The
>   marvell flavour seems to have been dropped entirely (at least that's
>   how I read the linux changelog for 6.6.3-1~exp1:
>     https://tracker.debian.org/news/1482751/accepted-linux-663-1exp1-source-into-experimental/

Yes, the kermel was broken and the checks for this disabled since quite
some time.

Bastian

-- 
Extreme feminine beauty is always disturbing.
		-- Spock, "The Cloud Minders", stardate 5818.4

[toc] | [prev] | [next] | [standalone]

#81330

[Multipart message — attachments visible in raw view] — view raw

Bastian Blank <waldi@debian.org> (2023-12-24):
> On Sun, Dec 24, 2023 at 08:38:57AM +0100, Cyril Brulebois wrote:
> > - kernel-image-* packages are now shipping /boot/vmlinuz-* (or
> >   /boot/vmlinux-* depending on the arch), instead of just /boot/vmlinuz
> >   (respectively /boot/vmlinux).
> 
> This was even dependent on architecture.  A lot of secondary
> architectures already had various suffixes.

Looking at kernel-image-* 6.5.13-1 for release archs, I'm only seeing
mips64el (-6.5.0-5-5kc-malta, -6.5.0-5-loongson-3, -6.5.0-5-mips64r2el
suffixes), and its build targets didn't hit the codepath I had to patch
for the general case.

Anyway, since you're not reacting with “oops, we didn't mean to”, I'm
going to count this as a deliberate change.

> > - Modules are compressed now, so the drm workaround needed an updated to
> >   cope with the extra .xz suffix:
> >     https://salsa.debian.org/installer-team/debian-installer/-/commit/bd0f1106f90756e6f4514108492d71e1f2e695ea
> 
> This now hardcodes .xz.  And I'm not really sure what this does and why
> this can't be fixed in the kernel packages.

Because that's a workaround for a problem that's not in the kernel and
that remains to be fully investigated.


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [prev] | [next] | [standalone]

#81332

[Multipart message — attachments visible in raw view] — view raw

Hi Roland,

FYI, you might want to read the full thread! ;)

On Sun, Dec 24, 2023 at 08:38:57AM +0100, Cyril Brulebois wrote:
> This is mostly for information: linux went through a lot of big changes,
> initially staged in experimental, and uploaded to unstable as of linux
> 6.6.8-1. These impacted the installer in different ways. This is a
> summary of the obvious impacts, there might be some others down the line
> (just by glancing at hw-detect's massaging of module list by stripping
> the .ko suffix).
> 
> - kernel-image-* packages are now shipping /boot/vmlinuz-* (or
>   /boot/vmlinux-* depending on the arch), instead of just /boot/vmlinuz
>   (respectively /boot/vmlinux).
[...]

this change also lead to failing live-builds, for trixie (the unstable live-builds
are of course also affected but fail earlier due to #1058994), this can be
seen in 
https://jenkins.debian.net/job/reproducible_debian_live_build_xfce_trixie/35/console
which failed with

cp: cannot stat 'chroot/debian-installer/build/dest/cdrom/vmlinuz': No such file or directory


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Just 100 companies are responsible for 71% of global emissions.
https://www.theguardian.com/sustainable-business/2017/jul/10/100-fossil-fuel-companies-investors-responsible-71-global-emissions-cdp-study-climate-change

[toc] | [prev] | [next] | [standalone]

#81346

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 29, 2023 at 08:04:19AM +0100, Roland Clobus wrote:
> Yesterday I published 3 fixes which got merged really quickly.
[...]
> Now Jenkins is running fine for the live sid images again :-)
> In the next step, I'll check the trixie live builds.

awesome, thank you!


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

20230709: Today was the warmest day on earth in 125,000 years. Today was also
the day with the most planes in the air at one time ever in history. By the time
you read this both of these records have probably been broken.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web