Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #263793 > unrolled thread

Boot fails to load network or USB, piix4_smbus - SMBus Host Controller, after update to dbus (1.14.10-3) unstable

Back to article view | Back to linux.debian.user

Contents

  Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable Andy Dorman <adorman@ironicdesign.com> - 2023-11-23 14:30 +0100
    Re: Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable Andy Dorman <adorman@ironicdesign.com> - 2023-11-23 22:10 +0100
      Re: Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable Jeffrey Walton <noloader@gmail.com> - 2023-11-23 22:40 +0100
      Re: Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable Tim Woodall <debianuser@woodall.me.uk> - 2023-11-24 05:20 +0100
        Re: Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable Andy Dorman <adorman@ironicdesign.com> - 2023-11-24 15:00 +0100
        Re: Boot fails to load network or USB, piix4_smbus - SMBus Host  Controller, after update to dbus (1.14.10-3) unstable SOLVED Andy Dorman <adorman@ironicdesign.com> - 2023-11-25 17:20 +0100

#263793 — Boot fails to load network or USB, piix4_smbus - SMBus Host Controller, after update to dbus (1.14.10-3) unstable

This may actually be a usrmerge problem, both usrmerge and usr-is-merged 
(>= 38~) are installed). But I do not know enough about how firmware 
controllers are loaded during boot to determine which package, dbus or 
usrmerge, I should submit a bug report to.

We have multiple old (14+ years) Tyan S3950 mobo servers running debian 
Unstable with various recent kernels, the latest being 6.1.0-3-amd64.

Despite being on the bleeding edge with the unstable distro, this has 
been a very reliable setup over the years. I love Debian.  However, 
after a recent update, we had to reboot a couple of servers and both 
booted with NO network or usb console keyboard. ie, completely unusable.

I have been able to mount the failed server HDs on another server and 
compared dmesg logs from the failed and last successful boots. The 
failed boots were not detecting the floppy (I told you they were old) 
and not loading the kernel piix4_smbus driver. These are the key lines 
present in the earlier good boots and missing from the failed boots:

...
Floppy drive(s): fd0 is 1.44M
piix4_smbus 0000:00:02.0: SMBus Host Controller at 0x580, revision 0
...

A failed boot did not have the above lines and subsequently failed to 
detect and enable the network interfaces, the console USB keyboard or 
any other usb devices.

FWIW, the 3ware RAID drivers are still loading fine in the bad boots. So 
some drivers are still loading.

I think this problem stems from the ongoing Debian move to a merged-/usr 
system. I have fully and successfully (I thought) installed both the 
usrmerge and usr-is-merged (>= 38~). The only hint I have is the last 
dbus package update contained a warning about assuming that a system is 
fully merged-/usr system "In the case of dbus, the symptom when this 
assumption is broken is particularly bad (various key system services 
will not start"...which sums up my problem quite nicely.

I will be happy to submit a formal bug report to the appropriate package 
using reportbug as soon as someone with way more knowledge than I tells 
me which package would be more appropriate to receive it. The last thing 
I want to do is create unnecessary work for anyone in the Debian community.

Thank you in advance for any help in fixing this problem.

-- 
Andy Dorman
Ironic Design, Inc.
AnteSpam.com

[toc] | [next] | [standalone]

#263802

I have continued to research this and I think I found the problem.

I also think the dbus update timing mentioned in the subject is entirely 
coincidental. I hope I haven't caused any unnecessary excitement or work 
for anyone in the dbus package team. My apologies if I did.

A few months back we ran into a package conflict with the 
initramfs-tools that we have used since the beginning. I believe the 
conflict was with systemd, but I could be mis-rememberiung that.  We 
were able to switch to tiny-initramfs which did the same job (we 
thought) and didn't have a conflict.

Since that switch and before our recent problems, we rebooted one server 
which just happened to be different from all our others in that it hever 
had the initramfs package conflict and was still running the original 
initramfs package.

On a hunch (since I really have no clue about how debian boot actually 
works) I compared initram image file sizes and used lsinitramfs to list 
the content of the initram images built with tiny-initramfs and the 
standard initramsfs. The results:

- the tiny-initramfs image file size was under 1MB and lsinitramfs 
listed a very few modules to be loaded and specifcally NOT the 
piix4_smbus driver

- the standard intramfs image file was ~ 34MB and included the missing
> usr/lib/modules/6.5.0-4-amd64/kernel/drivers/i2c/busses/i2c-piix4.ko

So based on my very sketchy knowledge of Debeian boot and what I am 
seeing on our working servers, I think the problem is with 
tiny-initramfs not including the needed kernel driver modules in the 
image file OR our not understanding how to correctly use tiny-initramfs.

The fix as far as we are concerned with our servers that are running has 
been to switch back to the standard initramfs package (the previous 
conflict is gone now thank goodness) and rebuild the initram images, 
which we have done.

I have not yet figured out how to fix our two broken servers since we 
can't boot them to update them.  Since we have several identical running 
servers and can mount and manipulate the file system of the dead 
servers, is it possible to just copy a good initrd.img-6.5.0-4-amd64 to 
the dead server /boot directory and boot to that image? That sounds way 
too simple to me.

-- 
Andy Dorman
Ironic Design, Inc.
AnteSpam.com

[toc] | [prev] | [next] | [standalone]

#263803

On Thu, Nov 23, 2023 at 4:09 PM Andy Dorman <adorman@ironicdesign.com> wrote:
>
> I have continued to research this and I think I found the problem.
>
> I also think the dbus update timing mentioned in the subject is entirely
> coincidental. I hope I haven't caused any unnecessary excitement or work
> for anyone in the dbus package team. My apologies if I did.
>
> A few months back we ran into a package conflict with the
> initramfs-tools that we have used since the beginning. I believe the
> conflict was with systemd, but I could be mis-rememberiung that.  We
> were able to switch to tiny-initramfs which did the same job (we
> thought) and didn't have a conflict.
>
> Since that switch and before our recent problems, we rebooted one server
> which just happened to be different from all our others in that it hever
> had the initramfs package conflict and was still running the original
> initramfs package.
>
> On a hunch (since I really have no clue about how debian boot actually
> works) I compared initram image file sizes and used lsinitramfs to list
> the content of the initram images built with tiny-initramfs and the
> standard initramsfs. The results:
>
> - the tiny-initramfs image file size was under 1MB and lsinitramfs
> listed a very few modules to be loaded and specifcally NOT the
> piix4_smbus driver
>
> - the standard intramfs image file was ~ 34MB and included the missing
> > usr/lib/modules/6.5.0-4-amd64/kernel/drivers/i2c/busses/i2c-piix4.ko
>
> So based on my very sketchy knowledge of Debeian boot and what I am
> seeing on our working servers, I think the problem is with
> tiny-initramfs not including the needed kernel driver modules in the
> image file OR our not understanding how to correctly use tiny-initramfs.
>
> The fix as far as we are concerned with our servers that are running has
> been to switch back to the standard initramfs package (the previous
> conflict is gone now thank goodness) and rebuild the initram images,
> which we have done.
>
> I have not yet figured out how to fix our two broken servers since we
> can't boot them to update them.  Since we have several identical running
> servers and can mount and manipulate the file system of the dead
> servers, is it possible to just copy a good initrd.img-6.5.0-4-amd64 to
> the dead server /boot directory and boot to that image? That sounds way
> too simple to me.

According to <https://www.linuxquestions.org/questions/linux-server-73/copying-initrd-img-file-to-different-server-4175414604/>,
you cannot copy initrd.img from one machine to another.

Maybe you can boot to a live CD, and then regenerate initrd.img for the machine?

Jeff

[toc] | [prev] | [next] | [standalone]

#263805

On Thu, 23 Nov 2023, Andy Dorman wrote:

> I have not yet figured out how to fix our two broken servers since we can't 
> boot them to update them.  Since we have several identical running servers 
> and can mount and manipulate the file system of the dead servers, is it 
> possible to just copy a good initrd.img-6.5.0-4-amd64 to the dead server 
> /boot directory and boot to that image? That sounds way too simple to me.
>

I'm pretty sure you can bind mount /proc, /sys, /dev, /run, chroot and
then update-initramfs to regen.

[toc] | [prev] | [next] | [standalone]

#263810

> I'm pretty sure you can bind mount /proc, /sys, /dev, /run, chroot and
> then update-initramfs to regen.

Thanks Tim. You make it sound so simple.

I searched for "chroot to mounted disk to update initramfs" and found 
several detailed descriptions of the process.

https://forums.debian.net/viewtopic.php?t=143279

Thank you so much.  I will give it a shot today and report back.

Cheers.

-- 
Andy Dorman

[toc] | [prev] | [next] | [standalone]

#263831 — Re: Boot fails to load network or USB, piix4_smbus - SMBus Host Controller, after update to dbus (1.14.10-3) unstable SOLVED

> I'm pretty sure you can bind mount /proc, /sys, /dev, /run, chroot and
> then update-initramfs to regen.
Thanks Tim. You make it sound so simple.

I searched for "chroot to mounted disk to update initramfs" and found 
several detailed descriptions of the process.

https://forums.debian.net/viewtopic.php?t=143279

Thank you so much.  I will give it a shot today and report back.

Thank you again Tim. It worked.

I found an excellent overall guide in the debian wiki that could be 
applied to fix any number of issues with a dead system:

https://wiki.debian.org/RescueLive

Here are the exact steps I took, just in case it can help anyone else 
with problems similar to ours.

> 1. Download, make, then boot into live CD or USB flash & drop into shell. >
> 2. Get root access with live CD just so I don't have to preface every
> command below with sudo.
> 
> sudo su -
> 
> 3. Use fdisk to examine HDs and determine main fs partition(s) on the
> RAID to be repaired, then mount it/them...Replace "sdax" below with
> the main (and boot if separate) RAID partitions. In my case we did not
> have a separate boot partition, so I only had to mount one device.
> 
> /sbin/fdisk -l
> mount /dev/sdax /mnt
> 
> 4. Confirm we see all the appropriate directories
> 
> ls /mnt/
> 
> 5. Bind mounts of key file system dirs
> 
> for name in proc sys dev ; do mount --bind /$name /mnt/$name; done
> 
> 6. mount /etc/resolve.conf and /dev/pts
> 
> mount --bind /dev/pts /mnt/dev/pts
> mount --bind /etc/resolv.conf /mnt/etc/resolv.conf
> 
> 7. chroot into mounted system
> 
> chroot /mnt/ /bin/bash
> 
> 8. I had to temporarily disable our apt proxy since the server can't
> see the proxy from our office where we are doing this repair.
> 
> mv /etc/apt/apt.conf.d/99proxy /etc/apt/apt.conf.d/99proxy.bak
> 
> 9. Install initramfs-tools and initramfs-tools-core which automatically
> removed tiny-initramfs-tools which caused all this in the first place.
> 
> apt install initramfs-tools initramfs-tools-core
> 
> 10. Rgen initramfs and update grub
> 
> update-initramfs -uv; update-grub
> 
> 11. Put the apt proxy back
> 
> mv /etc/apt/apt.conf.d/99proxy.bak /etc/apt/apt.conf.d/99proxy
> 
> 10. exit out of chroot, unmount everything, do a clean shutdown
> 
> exit
> 
> umount /mnt/dev/pts
> umount /mnt/dev
> umount /mnt/etc/resolv.conf
> umount /mnt/proc
> umount /mnt/sys
> umount /mnt/boot ???
> umount /mnt
> 
> shutdown -r now
> 
> 11. remove the LIVE flash then power up to confirm a good boot
> with the repaired RAID

Andy

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web