Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #268615 > unrolled thread

Could Gnome's "install pending software updates" cause installation scripts to misbehave?

Back to article view | Back to linux.debian.user

Contents

  Could Gnome's "install pending software updates" cause installation  scripts to misbehave? "Lucas B. Cohen" <lbc@members.fsf.org> - 2024-03-29 12:20 +0100
    Re: Could Gnome's "install pending software updates" cause  installation scripts to misbehave? Henning Follmann <hfollmann@itcfollmann.com> - 2024-03-29 16:10 +0100
      Re: Could Gnome's "install pending software updates" cause  installation scripts to misbehave? David Wright <deblis@lionunicorn.co.uk> - 2024-03-29 16:50 +0100
    Re: Could Gnome's "install pending software updates" cause  installation scripts to misbehave? "Lucas B. Cohen" <lbc@members.fsf.org> - 2024-03-29 21:00 +0100

#268615 — Could Gnome's "install pending software updates" cause installation scripts to misbehave?

Hi,

I've had a bit of a headache understanding why my Debian bookworm system 
suddenly panicked at boot with an 'unable to mount root fs' error. Turns 
out the first of my two menuentries in grub.cfg were no longer 
specifying the linux root by its device UUID (as I was expecting it to 
do, by honoring GRUB_DISABLE_LINUX_UUID != true) ; instead these 
menuentries were using the device node/file (/dev/md0 in this case, 
hence the kernel panic).

I've poured through the grub scripts a bit but they're quite complex. 
I've noticed that :

- uninstalling the second of two kernels caused the remaining one to 
correctly use the device UUID in grub.cfg ;

- reinstalling that second kernel caused grub.cfg to use UUIDs in all 
menuentries, as expected.

(Kernel were the two most recent stable ones: 6.1.0-17 and -18.)

This leads me to suspect that my grub.cfg might have been damaged in the 
way described above because update-grub might have been called in some 
unusual, limited execution environment. I'd very recently powered off my 
system and let the default "install pending software updates" option 
checked by accident, which caused every updated package from the 12.5 
release mark to be pulled. I'm guessing that linux-image-6.1.0-18 was 
part of it.

Has anyone witnessed something similar? Would anyone here care to check 
this somehow? Or should I open a bug against gnome-desktop without waiting?

Thank you for any insight.

Apologies for possible e-mail client misconfiguration.

Regards,

--
Lucas

[toc] | [next] | [standalone]

#268616 — Re: Could Gnome's "install pending software updates" cause installation scripts to misbehave?

On Fri, Mar 29, 2024 at 12:01:27PM +0100, Lucas B. Cohen wrote:
> Hi,
> 
> I've had a bit of a headache understanding why my Debian bookworm system
> suddenly panicked at boot with an 'unable to mount root fs' error. Turns out
> the first of my two menuentries in grub.cfg were no longer specifying the
> linux root by its device UUID (as I was expecting it to do, by honoring
> GRUB_DISABLE_LINUX_UUID != true) ; instead these menuentries were using the
> device node/file (/dev/md0 in this case, hence the kernel panic).
> 

Was there any error message during the update?
I think what might have gone wrong, that you ran out of space on /boot.


> I've poured through the grub scripts a bit but they're quite complex. I've
> noticed that :

Yeah, don't do that. These files are all automatically managed.
All changes should be done in /etc/default/grub or in the config files in
/etc/default/grub.d
Then the grub config files are created by running
update-grub


> 
> - uninstalling the second of two kernels caused the remaining one to
> correctly use the device UUID in grub.cfg ;

and that might have freed enough space on /boot.
So now everything works again :)

> 
> - reinstalling that second kernel caused grub.cfg to use UUIDs in all
> menuentries, as expected.
> 
> (Kernel were the two most recent stable ones: 6.1.0-17 and -18.)
> 
> This leads me to suspect that my grub.cfg might have been damaged in the way
> described above because update-grub might have been called in some unusual,
> limited execution environment. I'd very recently powered off my system and
> let the default "install pending software updates" option checked by
> accident, which caused every updated package from the 12.5 release mark to
> be pulled. I'm guessing that linux-image-6.1.0-18 was part of it.
> 
> Has anyone witnessed something similar? Would anyone here care to check this
> somehow? Or should I open a bug against gnome-desktop without waiting?
>

Usually it requires some trickery to install a new kernel on machines which
might not have enough remaining space on the boot partition.

For simple housekeeping it often is sufficient to run 
apt autoremove
after recent updates (after you confirmed that the newly installed kernel
boots fine).
That usually frees enough space for a possible new update. 


-H

-- 
Henning Follmann           | hfollmann@itcfollmann.com

[toc] | [prev] | [next] | [standalone]

#268618 — Re: Could Gnome's "install pending software updates" cause installation scripts to misbehave?

On Fri 29 Mar 2024 at 11:06:45 (-0400), Henning Follmann wrote:
> On Fri, Mar 29, 2024 at 12:01:27PM +0100, Lucas B. Cohen wrote:
> > 
> > I've had a bit of a headache understanding why my Debian bookworm system
> > suddenly panicked at boot with an 'unable to mount root fs' error. Turns out
> > the first of my two menuentries in grub.cfg were no longer specifying the
> > linux root by its device UUID (as I was expecting it to do, by honoring
> > GRUB_DISABLE_LINUX_UUID != true) ; instead these menuentries were using the
> > device node/file (/dev/md0 in this case, hence the kernel panic).
> 
> Was there any error message during the update?
> I think what might have gone wrong, that you ran out of space on /boot.
> 
> > I've poured through the grub scripts a bit but they're quite complex. I've
> > noticed that :
> 
> Yeah, don't do that. These files are all automatically managed.
> All changes should be done in /etc/default/grub or in the config files in
> /etc/default/grub.d
> Then the grub config files are created by running
> update-grub
> 
> > - uninstalling the second of two kernels caused the remaining one to
> > correctly use the device UUID in grub.cfg ;
> 
> and that might have freed enough space on /boot.
> So now everything works again :)
> 
> > - reinstalling that second kernel caused grub.cfg to use UUIDs in all
> > menuentries, as expected.
> > 
> > (Kernel were the two most recent stable ones: 6.1.0-17 and -18.)
> > 
> > This leads me to suspect that my grub.cfg might have been damaged in the way
> > described above because update-grub might have been called in some unusual,
> > limited execution environment. I'd very recently powered off my system and
> > let the default "install pending software updates" option checked by
> > accident, which caused every updated package from the 12.5 release mark to
> > be pulled. I'm guessing that linux-image-6.1.0-18 was part of it.

I'd write "upgraded" rather than "pulled", if that's what you meant.

> > Has anyone witnessed something similar? Would anyone here care to check this
> > somehow? Or should I open a bug against gnome-desktop without waiting?
> >
> Usually it requires some trickery to install a new kernel on machines which
> might not have enough remaining space on the boot partition.
> 
> For simple housekeeping it often is sufficient to run 
> apt autoremove
> after recent updates (after you confirmed that the newly installed kernel
> boots fine).
> That usually frees enough space for a possible new update. 

You can also reduce the space taken up by initrd files, which are
getting rather large nowadays if they are built with MODULES=most
rather than MODULES=dep.

When you have at least two working kernels, remove any unnecessary
backups, copy the older kernel's initrd somewhere else, then rebuild
it with MODULES=dep. If that kernel still boots ok, then you probably
have a lot more room available now for the next kernel upgrade.
Finally, reboot the newer kernel.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#268630 — Re: Could Gnome's "install pending software updates" cause installation scripts to misbehave?

On Fri 29 Mar 2024 at 11:06:45 (-0400), Henning Follmann wrote:
> On Fri, Mar 29, 2024 at 12:01:27PM +0100, Lucas B. Cohen wrote:
>> Hi,
>> 
>> I've had a bit of a headache understanding why my Debian bookworm system
>> suddenly panicked at boot with an 'unable to mount root fs' error. Turns out
>> the first of my two menuentries in grub.cfg were no longer specifying the
>> linux root by its device UUID (as I was expecting it to do, by honoring
>> GRUB_DISABLE_LINUX_UUID != true) ; instead these menuentries were using the
>> device node/file (/dev/md0 in this case, hence the kernel panic).
>> 
> 
> Was there any error message during the update?
> I think what might have gone wrong, that you ran out of space on /boot.

Space on /boot couldn't have been the issue, I have 1GB allocated to 
that partition, and those 2 kernels only take up about a third of that 
space.

The was no visible error message at the time, as it's all hidden from 
the user's view by Gnome, right before power off. However I'm checking 
my /var/log/apt/term.log where it was handily stored, and here's what 
I'm seeing:

- seems that grub-mkconfig (the grub script called by Debian's 
update-grub wrapper) was in fact never called during that update 
sequence! (Therefore Gnome's handling of updates is off the hook.) 
Perhaps it was because of some bad dkms and linux-headers interaction. 
Some module failed to build, which cascaded into leaving the kernel and 
headers packages into the 'unconfigured' state:

Building module:
Cleaning build area...
env NV_VERBOSE=1 make -j12 modules 
KERNEL_UNAME=6.1.0-18-amd64.......................(bad exit status: 2)
Error! Bad return status for module build on kernel: 6.1.0-18-amd64 (x86_64)
Consult /var/lib/dkms/nvidia-current/525.147.05/build/make.log for more 
information.
Error! One or more modules failed to install during autoinstall.
Refer to previous errors for more information.
dkms: autoinstall for kernel: 6.1.0-18-amd64 failed!
run-parts: /etc/kernel/postinst.d/dkms exited with return code 11
dpkg: error processing package linux-image-6.1.0-18-amd64 (--configure):
  installed linux-image-6.1.0-18-amd64 package post-installation script 
subprocess returned error exit status 1

dpkg: dependency problems prevent configuration of linux-headers-amd64:
  linux-headers-amd64 depends on linux-headers-6.1.0-18-amd64 (= 
6.1.76-1); however:
   Package linux-headers-6.1.0-18-amd64 is not configured yet.

- Consequence: my grub.cfg was only regenerated two days later, 
incidentally , during a scheduled unattended-upgrades run. Where

Log started: 2024-03-28  09:56:03
[...]
Removing linux-image-6.1.0-15-amd64 (6.1.66-1) ...
/etc/kernel/prerm.d/dkms:
[...]
depmod...
/etc/kernel/postrm.d/initramfs-tools:
[...]
/etc/kernel/postrm.d/zz-update-grub:
Generating grub configuration file ...
Found background image: /usr/share/images/desktop-base/desktop-grub.png
Found linux image: /boot/vmlinuz-6.1.0-18-amd64
Found linux image: /boot/vmlinuz-6.1.0-17-amd64
Found initrd image: /boot/initrd.img-6.1.0-17-amd64
Warning: Not executing os-prober.
done
[...]
Error! One or more modules failed to install during autoinstall.
Refer to previous errors for more information.
dkms: autoinstall for kernel: 6.1.0-18-amd64 failed!
run-parts: /etc/kernel/header_postinst.d/dkms exited with return code 11
Failed to process /etc/kernel/header_postinst.d at 
/var/lib/dpkg/info/linux-headers-6.1.0-18-amd64.postinst line 11.
dpkg: error processing package linux-headers-6.1.0-18-amd64 (--configure):
  installed linux-headers-6.1.0-18-amd64 package post-installation 
script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of linux-image-amd64:
  linux-image-amd64 depends on linux-image-6.1.0-18-amd64 (= 6.1.76-1); 
however:
   Package linux-image-6.1.0-18-amd64 is not configured yet.

dpkg: error processing package linux-image-amd64 (--configure):
  dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-headers-amd64:
  linux-headers-amd64 depends on linux-headers-6.1.0-18-amd64 (= 
6.1.76-1); however:
   Package linux-headers-6.1.0-18-amd64 is not configured yet.

dpkg: error processing package linux-headers-amd64 (--configure):
  dependency problems - leaving unconfigured
Errors were encountered while processing:
  linux-image-6.1.0-18-amd64
  linux-headers-6.1.0-18-amd64
  linux-image-amd64
  linux-headers-amd64
Log ended: 2024-03-28  09:58:24

Something's now apparent: the initrd hadn't been created for this new 
-18 kernel until after grub-mkconfig's execution. My backed up erroneous 
grub.cfg confirms this. Maybe grub-mkconfig doesn't allow the use of 
UUID= absent an initrd? That would be enough to explain everything.

Anyway, this is not an easy thing to reproduce. I guess it just calls 
attention to the danger of unattended/automatic upgrades in odd cases 
like these.

Thanks Henning, and thank you David for your help. (Apologies for not 
replying to your messages; I'd forgotten to subscribe to the ML.)


> -H
> 
> -- 
> Henning Follmann           | hfollmann@itcfollmann.com

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web