Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #86382 > unrolled thread

Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs

Back to article view | Back to linux.debian.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Marc Haber <mh+debian-bugs@zugschlus.de> - 2025-03-07 20:50 +0100
    Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Chris Hofstaedtler <zeha@debian.org> - 2025-03-08 13:00 +0100
    Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Ben Hutchings <ben@decadent.org.uk> - 2025-03-16 20:20 +0100
      Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Ben Hutchings <ben@decadent.org.uk> - 2025-03-16 21:10 +0100
        Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Ben Hutchings <ben@decadent.org.uk> - 2025-03-25 01:20 +0100
          Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Ben Hutchings <ben@decadent.org.uk> - 2025-03-28 01:40 +0100
            Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs Ben Hutchings <ben@decadent.org.uk> - 2025-04-30 03:30 +0200

#86382 — Bug#1099655: initramfs-tools 146 generates incorrect initramfs : does not boot, does not find root fs

On Fri, Mar 07, 2025 at 05:44:00PM +0100, Eric Valette wrote:
>See additionnal info. What type of compression do you have for initrd 
>and modules?

2 [2/5411]mh@swivel:~ $ grep CONFIG_RD /boot/config-6.12.17-amd64 
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_RDMA_RXE=m
# CONFIG_RDMA_SIW is not set
[3/5412]mh@swivel:~ $ grep CONFIG_MODULE /boot/config-6.12.17-amd64 
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
# CONFIG_MODULE_SIG_SHA1 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
# CONFIG_MODULE_SIG_SHA3_256 is not set
# CONFIG_MODULE_SIG_SHA3_384 is not set
# CONFIG_MODULE_SIG_SHA3_512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
CONFIG_MODULE_COMPRESS=y
# CONFIG_MODULE_COMPRESS_GZIP is not set
CONFIG_MODULE_COMPRESS_XZ=y
# CONFIG_MODULE_COMPRESS_ZSTD is not set
CONFIG_MODULE_COMPRESS_ALL=y
CONFIG_MODULE_DECOMPRESS=y
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_MODULE_SIG_KEY_TYPE_RSA=y
CONFIG_MODULE_ALLOW_BTF_MISMATCH=y
[4/5413]mh@swivel:~ $ 

I am using the Debian kernel on unstable.

Greetings
Marc
-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

[toc] | [next] | [standalone]

#86385

On Sat, Mar 08, 2025 at 12:29:06PM +0100, Eric Valette wrote:
> On Fri, 7 Mar 2025 20:43:08 +0100 Marc Haber <mh+debian-bugs@zugschlus.de>
> wrote:
> > On Fri, Mar 07, 2025 at 05:44:00PM +0100, Eric Valette wrote:
> 
> > CONFIG_MODULE_DECOMPRESS=y
> 
> So the culprit is there. It is not set in my own kernel build (never been so
> far although config has been created via the official linux-source-6.6) and
> thus I cannot load compressed modules without user space helpers that
> probably not exist in initrd busybox version. Of course I can load
> compressed modules once the pivot_root/switch_root is done.
> 
> So it remains a bug because the script store compressed modules in initrd
> without checking this compilation flag and the kernel does not boot. If was
> not doing this in 145. This is the reason for the bug I have.
> 
> A second bug is that the initrd does not seems to be compressed while config
> says it should be with zstd (and the CONFIG_RD_* is ok). I can change to xz.
> 
> A third one is that compressing compressed file is not efficient in term of
> size. There has been bug opened exactly on this theme on other
> distributions.

AFAICT this is an intentional change:
https://salsa.debian.org/kernel-team/initramfs-tools/-/merge_requests/128

C.

[toc] | [prev] | [next] | [standalone]

#86464

[Multipart message — attachments visible in raw view] — view raw

Control: tag -1 moreinfo unreproducible

On Sat, 2025-03-08 at 12:29 +0100, Eric Valette wrote:
> On Fri, 7 Mar 2025 20:43:08 +0100 Marc Haber 
> <mh+debian-bugs@zugschlus.de> wrote:
> > On Fri, Mar 07, 2025 at 05:44:00PM +0100, Eric Valette wrote:
> 
> > CONFIG_MODULE_DECOMPRESS=y
> 
> So the culprit is there. It is not set in my own kernel build (never 
> been so far although config has been created via the official 
> linux-source-6.6) and thus I cannot load compressed modules without user 
> space helpers that probably not exist in initrd busybox version. Of 
> course I can load compressed modules once the pivot_root/switch_root is 
> done.

The initramfs is supposed to use modprobe etc. from kmod, not from
busybox.  So there should be no difference in capabilities from the main
Debian system.

Does your initramfs image include the right version of modprobe?
Boot with "break=top" and run "modprobe --version" to check this.


[...]
> A second bug is that the initrd does not seems to be compressed while 
> config says it should be with zstd (and the CONFIG_RD_* is ok). I can 
> change to xz.

As explained, it is intentional that the modules are not recompressed,
while other files are.  We made a trade-off between size and speed (of
mkinitramfs), and I don't intend to revisit that.

[...]
> Additional note : enabling this compilation flags, cause the signature
> of external modules to be incorrect as XZ with this flag *must* be used
> with CRC32 checksum and not the xz default CRC64. I had to add
> --check=crc32 to my signature scripts for external modules.
[...]

When does this signature script run?

I built a kernel from Linux 6.6.80 using the cloud-amd64 configuration
from linux-config-6.6, and an initramfs using initramfs-tools 0.146. 
Module loading worked OK, so I don't believe this is a general problem.

Ben.

-- 
Ben Hutchings
Lowery's Law:
        If it jams, force it. If it breaks, it needed replacing anyway.

[toc] | [prev] | [next] | [standalone]

#86465

[Multipart message — attachments visible in raw view] — view raw

On Sun, 2025-03-16 at 20:21 +0100, Eric Valette wrote:
> On 16/03/2025 20:13, Ben Hutchings wrote:
[...]

> > Does your initramfs image include the right version of modprobe?
> > Boot with "break=top" and run "modprobe --version" to check this.
> 
> It use wahever is put in. I did not make any chnage.

And yet module loading is going wrong in a way that can't be reproduced
elsewhere.  So please check.

[...]

> > > Additional note : enabling this compilation flags, cause the signature
> > > of external modules to be incorrect as XZ with this flag *must* be used
> > > with CRC32 checksum and not the xz default CRC64. I had to add
> > > --check=crc32 to my signature scripts for external modules.
> > [...]
> > 
> > When does this signature script run?
> 
> After I do a dkms. I do not want to put the signature key available 
> directly in the fs.

Yes, that's a sensible policy.

I was wondering whether the modules installed in the main system are
properly signed but due to some mis-ordering the modules copied into the
initramfs are not.

Can you unpack the initramfs (with unmkinitramfs) and check that the
modules have the same contents as those installed in the main system?
(With the current version of unmkinitramfs, they will be unpacked under
a cpio<n> subdirectory.)

> > 
> > I built a kernel from Linux 6.6.80 using the cloud-amd64 configuration
> > from linux-config-6.6, and an initramfs using initramfs-tools 0.146.
> > Module loading worked OK, so I don't believe this is a general problem.
> 
> I do not use cloud config. Did you check if CONFIG_MODULE_DECOMPRESS=y 
> is set in this config? If yes then the bug goes untoticed as the kernel 
> itself decompress the modules not the module loader.

It is not set.

Ben.

-- 
Ben Hutchings
Lowery's Law:
        If it jams, force it. If it breaks, it needed replacing anyway.

[toc] | [prev] | [next] | [standalone]

#86600

[Multipart message — attachments visible in raw view] — view raw

Hi Eric,

You wrote:
[...]
> kmod binaries are identical
[...]
> more generally, the modules that exist in the CPIO and the one in the
> rootfs are identical
[...]
> here is my complete module config if it helps (note that I keep the
> DECOMPRESS so far).
> 
> grep CONFIG_MODULE /boot/config-6.6.83
> CONFIG_MODULES_USE_ELF_RELA=y
> CONFIG_MODULE_SIG_FORMAT=y
> CONFIG_MODULES=y
> # CONFIG_MODULE_DEBUG is not set
> CONFIG_MODULE_FORCE_LOAD=y
> CONFIG_MODULE_UNLOAD=y
> CONFIG_MODULE_FORCE_UNLOAD=y
> # CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set
> # CONFIG_MODULE_SRCVERSION_ALL is not set
> CONFIG_MODULE_SIG=y
> # CONFIG_MODULE_SIG_FORCE is not set
> CONFIG_MODULE_SIG_ALL=y
> # CONFIG_MODULE_SIG_SHA1 is not set
> # CONFIG_MODULE_SIG_SHA224 is not set
> CONFIG_MODULE_SIG_SHA256=y
> # CONFIG_MODULE_SIG_SHA384 is not set
> # CONFIG_MODULE_SIG_SHA512 is not set
> CONFIG_MODULE_SIG_HASH="sha256"
> # CONFIG_MODULE_COMPRESS_NONE is not set
> # CONFIG_MODULE_COMPRESS_GZIP is not set
> CONFIG_MODULE_COMPRESS_XZ=y
> # CONFIG_MODULE_COMPRESS_ZSTD is not set
> CONFIG_MODULE_DECOMPRESS=y
> # CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
> CONFIG_MODULES_TREE_LOOKUP=y
> CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"
> # CONFIG_MODULE_SIG_KEY_TYPE_RSA is not set
> CONFIG_MODULE_SIG_KEY_TYPE_ECDSA=y

This all matches what I used in my local test (except for the
CONFIG_MODULE_DECOMPRESS which was disabled).  So I still don't
understand what's special about your configuration that results in the
failure.

Could you please send:

- The *complete* kernel config that you were using (before enabling
CONFIG_MODULE_DECOMPRESS).

- All the error messages from the initramfs when using this kernel and
initramfs-tools 0.146.  (A photograph of the screen is fine, if it's
readable.)

Ben.

-- 
Ben Hutchings
Man invented language to satisfy his deep need to complain.
                                                          - Lily Tomlin

[toc] | [prev] | [next] | [standalone]

#86647

[Multipart message — attachments visible in raw view] — view raw

On Thu, 2025-03-27 at 16:46 +0100, Eric Valette wrote:
> On 25/03/2025 01:12, Ben Hutchings wrote:
> > 
> > Hi Eric,
> 
> > - The *complete* kernel config that you were using (before enabling
> > CONFIG_MODULE_DECOMPRESS).
> 
> I switched tp 6.12.20 but same problem. Config attached.
> > 
> > - All the error messages from the initramfs when using this kernel and
> > initramfs-tools 0.146.  (A photograph of the screen is fine, if it's
> > readable.)
> 
> Photo attached
> 
> 
> No module loaded at all.

Indeed.  But the root device is there anyway, presumably because the
nvme driver is built-in.  So I think the "No such device" error is
actually because the *filesystem* module fails to load.

So I have more questions:

- Your kernel config has ext4 enabled as a module.  Is that the root
  filesystem type?
- What does the command "/usr/lib/klibc/bin/fstype /dev/nvme0n1p5"
  print?
- Using the bad initramfs, in the rescue shell:
  - What does "echo $ROOTFSTYPE" print?
  - What does "cat /proc/filesystems" print?
  - What does "modprobe fs-ext4" print, if anything?
  - If you then exit the rescue shell, does the boot process succeed?

Ben.

-- 
Ben Hutchings
If at first you don't succeed, you're doing about average.

[toc] | [prev] | [next] | [standalone]

#87140

[Multipart message — attachments visible in raw view] — view raw

On Fri, 2025-04-25 at 19:38 +0200, Eric Valette wrote:
[...]
> I think the bug #1099801 Marco d'Itri <md@Linux.IT>  closed is a 
> dupplicate from mine that must also be closed I guess.

Do you mean that the new version of kmod fixes this for you?

Ben.

-- 
Ben Hutchings
If more than one person is responsible for a bug, no one is at fault.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web