Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #72474 > unrolled thread

Bug#989863: debian-installer: Firmware problems in bullseye

Back to article view | Back to linux.debian.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#989863: debian-installer: Firmware problems in bullseye Cyril Brulebois <kibi@debian.org> - 2021-07-26 06:20 +0200
    Bug#989863: debian-installer: Firmware problems in bullseye Salvatore Bonaccorso <carnil@debian.org> - 2021-07-26 21:30 +0200
      Bug#969264: Bug#989863: debian-installer: Firmware problems in bullseye Cyril Brulebois <kibi@debian.org> - 2021-07-27 00:40 +0200
    Bug#989863: debian-installer: Firmware problems in bullseye Cyril Brulebois <kibi@debian.org> - 2021-07-28 10:00 +0200

#72474 — Bug#989863: debian-installer: Firmware problems in bullseye

[Multipart message — attachments visible in raw view] — view raw

TL;DR: No progress on the social side (doc) yet, but lots of progress on
       the technical side (making firmware-enabled images work better)!

Kernel team, feel free to skip to @KT for a little question.

I'd like to thank Ben again for the modalias thing, this made it very
easy to deploy firmware packages!

Cyril Brulebois <kibi@debian.org> (2021-07-24):
> # main-only images
> 
> ## isenkram-based proof-of-concept
[…]
>  - The issues I fixed in isenkram-cli were around some fallback code
>    path, and the AppStream maintainer suggested there might have been
>    some issues in AppStream causing the primary code path to fail:
>      https://bugs.debian.org/989884#24
>    The relevant update was accepted into testing since then, but I
>    haven't had a chance to check that hypothesis. It would be great to
>    know whether we actually have two working lookup implementation and
>    not just the fallback one. (This could mean more reliability during
>    a release cycle, in case one of them breaks — again.)

I've tried downgrading isenkram-cli to the version before my fixes
regarding the fallback code path, and it didn't find anything, so either
the primary code path or appstream is still broken. I've made a note to
possibly investigate this further, but this definitely doesn't block the
release.

> ## proposed solution
> 
> If others agree, I suggest we add the following somewhere in the
> installation guide (I don't think we should have a specific warning in
> the installer; or more specifically it would be way too late to have it
> be shown localized at this stage; more on that in the second part of my
> mail). Heavily summarized (to give readers of this bug report an idea,
> not for direct user consumption!!!): […]

Given Paul's ACK, I'll turn this into a bug report against the
installation guide, with a slightly better draft.

> ----------------------------------------------------------------------
> 
> # firmware-enabled images
> 
> ## easy, but ugly fix

Simon's solution makes the fallback plan even more straightforward. I
don't think we'll need it though. Read on!

> ## more targeted approach: modalias information
> 
> Ben suggested augmenting AppStream metadata (see DEP-11) of
> firmware-nonfree binaries with modalias information, so that it would
> be feasible to establish some hardware → firmware mapping. It's not been
> included in the archive yet, see MR#19 for src:firmware-nonfree:
>   https://salsa.debian.org/kernel-team/firmware-nonfree/-/merge_requests/19

This was uploaded to unstable, I'm currently waiting for the updated
metadata to show up in the archive.

Meanwhile, I've written a new tool for debian-cd, which Steve has merged
and refined further to match other choices made in that area (something
like: due to size constraints, we don't ship all firmware packages), so
I'm confident either of us should be able to refine this further once we
actually have metadata in place. That happened in debian-cd master, and
was uploaded as debian-cd 3.1.35:
  https://salsa.debian.org/images-team/debian-cd/-/blob/master/tools/generate_firmware_patterns

Basically, we extract the relevant bits from the DEP-11 YAML file, and
generate patterns files that we ship on the firmware-enabled images:
 - /firmware has firmware-*.deb
 - /firmware/dep-11 has firmare-*.patterns

Meanwhile, I've hacked d-i to include those files, built manually while
the rest was being worked on, so that I could test the runtime on my
test laptops.

> This doesn't help us (yet) regarding Intel SOF, since that package is
> (ultimately) built from src:firmware-sof, which doesn't have this
> logic yet. But I don't think we should delay the release for that (it
> could be listed in errata, and/or fixed in 11.1, whenever the package
> is ready), and if the machine at least boots fine, including the
> graphics stack, the isenkram trick above should let users install the
> remaining firmware packages on their own.

The isenkram-based fallback (once we have the actual doc) is still a
valid way to ensure this package can get installed from within the
installed system, but I've also included the relevant IDs as a special
case on the debian-cd side, so that we don't have to worry about that
one (it seems to show up quite often on laptops, so…):
  https://salsa.debian.org/images-team/debian-cd/-/blob/master/tools/generate_firmware_patterns#L135-160

> There might be other “widely spread” firmware that might benefit from
> the same logic, but I won't be spending too much time thinking about
> this right now; such patches can come later, and those updated package
> firmwares should be pickled up automatically afterwards.
> 
> Regarding the implementation, my current approach would be:
>  - have firmware-nonfree's MR merged and uploaded;

DONE.

>  - make sure the relevant information bubbles up to the dep11/Components
>    files (YAML format, even if the excerpts above were XML);

TODO.

>  - have debian-cd extract the relevant firmware package←→modalias
>    mapping from YAML into one or N flat files to be consumed by d-i, for
>    firmware-enabled images primarily (or exclusively);

DONE, to be checked once the previous item is complete, and some fixups
might be required until a full image build behaves like my heavily
hacked netboot-gtk images (meaning purely from debian-installer.git, no
debian-cd involved).

>  - patch d-i to consume that or those files to queue up as many firmware
>    packages as required.

DONE.

I've patched hw-detect to use that information in addition to the usual
grepping of dmesg to find missing firmware, and reused the logic that
was there already to enable non-free and contrib (in that order, which I
didn't think it was reasonable to fix at this stage of the freeze[1]),
and install firmware packages inside /target (the installed system, as
opposed to the installer's context).

 1. https://salsa.debian.org/installer-team/apt-setup/-/commit/ba70d27d150c5fd869653082ee9180410346436b

This seems to work great on both test laptops so far, and after
installation, isenkram-autoinstall-firmware basically does nothing,
since d-i took care of approximately everything.

I have still a slight discrepancy for some BT firmware files that aren't
seen in dmesg but that are apparently documented in modinfo, which
isenkram spots, leading to an extra firmware-iwlwifi on one of my test
laptops. I haven't tested full desktop installations, so I'm not sure
whether that's actually needed, but that seems pretty minor at this
point. If users are actually wondering about that, they can still follow
the isenkram-based fallback (once we have the actual doc). Therefore,
I'm considering this very minor, keeping a note about it, but I won't
look into it before 11.0.

I've also fixed and tested #973733 since one of those laptops was
playing tricks regarding actual module names, making the reloading of
modules after firmware installation… insufficient. Plus fixed a few
other things so that logs are somewhat cleaner:
  https://salsa.debian.org/installer-team/hw-detect/-/commits/master

@KT:

I haven't uploaded it since I'm seeing iwlwifi requesting what appears
to be a debug-only firmware (iwl-debug-yoyo.bin), that's even requested
via an aptly-named firmware_request_nowarn() function in linux.git; I'd
like to check with the kernel maintainers whether it's expected for this
message to show up in dmesg still (given the name and comments in
linux.git, I'm not sure it should). In any cases, it's slightly sad to
pause and prompt users from firmware files we don't ship anywhere
(AFAICT), that they don't actually need.

Therefore, I'm tempted to blacklist this firmware file in hw-detect
(i.e. pretend it was never requested).

> Note: it's likely that at least some firmware packages are going to be
> installed rather early in the installation process; network drivers
> (wired and/or wireless ones) are likely going to be probed early, and
> have generated the relevant messages in dmesg.

That's exactly what happens, we have:
 - check-missing-firmware: dealing with hot-deploying firmware files and
   reloading modules.
 - install-firmware: dealing with the installation of firmware packages
   into /target (carrying over those seen by check-missing-firmware,
   plus the new bunch thanks to modalias information).

> I have still to familiarize myself with the whole (existing)
> “automatic installation of firmware packages when running on a
> firmware-enabled image” logic though, to see where the patches would
> be best suited.

All done.

As a bonus, I've (re)discovered that netcfg doesn't propagate WPA
settings to the installed system when NM isn't installed there.
Considering this not-a-new-problem©®™, and that shouldn't affect end
users too much given they're likely going to install a desktop
environment (others might know about wpa_supplicant); I'd like to
investigate this at some later point, and hopefully get a fix merged
into 11.1, and 10.n if I'm successful.

> ----------------------------------------------------------------------
> 
> # final thought
> 
> Circling back to the main-only images, having that hardware→firmware
> lookup would let us have a chance to mention that “this and that and
> that firmware package might be needed on your system”, adding a pointer
> to the “isenkram procedure” of the installation guide. If that'd be
> desired, the relevant mapping file would need to end up not only in
> firmware-enabled images, and we would need new strings, new
> translations, and that's almost certainly only achievable during the
> next release cycle.
> 
> In any cases, I'm happy to start with something specific to the
> firmware-enabled images anyway, so that the possible fallouts are
> contained within a rather known and well-defined area. We should have
> enough time to implement that for 11.0, but we don't have *plenty*.

We went exactly that route with Steve: changing things under /firmware
only, so that there can't (or really shouldn't) be any fallouts on
main-only images. The rest can wait until bookworm (if we even want to
implement such a thing).


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [next] | [standalone]

#72482

HI Cyril,

On Mon, Jul 26, 2021 at 06:14:09AM +0200, Cyril Brulebois wrote:
> @KT:
> 
> I haven't uploaded it since I'm seeing iwlwifi requesting what appears
> to be a debug-only firmware (iwl-debug-yoyo.bin), that's even requested
> via an aptly-named firmware_request_nowarn() function in linux.git; I'd
> like to check with the kernel maintainers whether it's expected for this
> message to show up in dmesg still (given the name and comments in
> linux.git, I'm not sure it should). In any cases, it's slightly sad to
> pause and prompt users from firmware files we don't ship anywhere
> (AFAICT), that they don't actually need.
> 
> Therefore, I'm tempted to blacklist this firmware file in hw-detect
> (i.e. pretend it was never requested).

This reminds me of the long(ish) standing bug report #969264 and
#966218.

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]

#72483 — Bug#969264: Bug#989863: debian-installer: Firmware problems in bullseye

[Multipart message — attachments visible in raw view] — view raw

Hi Salvatore,

Salvatore Bonaccorso <carnil@debian.org> (2021-07-26):
> On Mon, Jul 26, 2021 at 06:14:09AM +0200, Cyril Brulebois wrote:
> > @KT:
> > 
> > I haven't uploaded it since I'm seeing iwlwifi requesting what appears
> > to be a debug-only firmware (iwl-debug-yoyo.bin), that's even requested
> > via an aptly-named firmware_request_nowarn() function in linux.git; I'd
> > like to check with the kernel maintainers whether it's expected for this
> > message to show up in dmesg still (given the name and comments in
> > linux.git, I'm not sure it should). In any cases, it's slightly sad to
> > pause and prompt users from firmware files we don't ship anywhere
> > (AFAICT), that they don't actually need.
> > 
> > Therefore, I'm tempted to blacklist this firmware file in hw-detect
> > (i.e. pretend it was never requested).
> 
> This reminds me of the long(ish) standing bug report #969264 and
> #966218.

I must confess I didn't spend time checking the BTS, only the source
code, woops. Thanks for the pointer, I think I'll consider this a “known
issue” (for it to be requested while it's neither needed nor to be found
anywhere), and blacklist that file in hw-detect.

For the avoidance of doubt, I don't recall any major problems with my
wireless connection, at least during all installation tests: one full
GNOME installation at the beginning, and many no-desktop installation
tests after that. I haven't used the installed systems much though,
since I'm working on the installer side. :)

Adding #969264 in cc to chime in: that's still current as of
linux-image-5.10.0-8-amd64 (version 5.10.46-2), both in the Debian
Installer context, and within the installed system.

I'm seeing this in a current Dell G3 15 machine (DELL G3 15-3500 –
3500-1294 if my notes are correct).


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [prev] | [next] | [standalone]

#72502

[Multipart message — attachments visible in raw view] — view raw

Cyril Brulebois <kibi@debian.org> (2021-07-26):
> > ## proposed solution
> > 
> > If others agree, I suggest we add the following somewhere in the
> > installation guide (I don't think we should have a specific warning
> > in the installer; or more specifically it would be way too late to
> > have it be shown localized at this stage; more on that in the second
> > part of my mail). Heavily summarized (to give readers of this bug
> > report an idea, not for direct user consumption!!!): […]
> 
> Given Paul's ACK, I'll turn this into a bug report against the
> installation guide, with a slightly better draft.

This is my next action.

> This was uploaded to unstable, I'm currently waiting for the updated
> metadata to show up in the archive.

That propagated to unstable, resulted in successful testing with a
hacked debian-cd setup (pulling metadata from unstable instead of
testing), so I augmented Paul's unblock hint for firmware-nonfree with a
urgent, and we tested a non-hacked debian-cd setup as well.

> Basically, we extract the relevant bits from the DEP-11 YAML file, and
> generate patterns files that we ship on the firmware-enabled images:
>  - /firmware has firmware-*.deb
>  - /firmware/dep-11 has firmare-*.patterns
> 
> Meanwhile, I've hacked d-i to include those files, built manually
> while the rest was being worked on, so that I could test the runtime
> on my test laptops.

A difference between both (hacked d-i vs. proper debian-cd build) is
where the firmware directory appears, and that was addressed in this
upload (pushed to testing already):
  https://salsa.debian.org/installer-team/hw-detect/-/commit/bc214296310c86f2144d50953b39d2f7628be136?w=1
  https://tracker.debian.org/news/1245070/accepted-hw-detect-1146-source-into-unstable/

With that fix in place, I discovered another annoyance (this wouldn't
have been a blocker in my book, but that could surprise people):
  https://bugs.debian.org/991587 [cdrom-detect]
  https://bugs.debian.org/991590 [iso-scan, cloned]

which I've just fixed (even if that was only manually tested) in
hw-detect + cdrom-detect; the iso-scan bug report can stay around, I
don't think we need to rush acting on it: the cdrom-detect codepath is
much more likely to be hit as far as I understand it (that's what
happens with ISO images like netinst at least…).

I'll retest “for real” once packages are in the archive, once I've
triggered a new d-i daily build, and once debian-cd has spinned another
firmware-enabled netinst daily build, but besides a possible typo here
or there, I think I'm out of obvious issues that need fixing!

(At least on those two machines I've been re-installing over and over.)

> >  - make sure the relevant information bubbles up to the dep11/Components
> >    files (YAML format, even if the excerpts above were XML);
> 
> TODO.

Done, with unstable, then with testing; everything looks good.

> >  - have debian-cd extract the relevant firmware package←→modalias
> >    mapping from YAML into one or N flat files to be consumed by d-i, for
> >    firmware-enabled images primarily (or exclusively);
> 
> DONE, to be checked once the previous item is complete, and some
> fixups might be required until a full image build behaves like my
> heavily hacked netboot-gtk images (meaning purely from
> debian-installer.git, no debian-cd involved).

Done checking… and fixing (see details above).

> Therefore, I'm tempted to blacklist this firmware file in hw-detect
> (i.e. pretend it was never requested).

Given Salvatore's mention of #969264 & #966218 (thanks!), I've added
this commit:
  https://salsa.debian.org/installer-team/hw-detect/-/commit/82519629d9db3ccbcf7cc45ce87c0a7238008efd

Besides some builds (hw-detect, cdrom-detect, linux) and some final
tests (then migration to testing), I think we have all the pieces for
D-I Bullseye RC 3.

Whether we need a D-I Bullseye RC 4 (or 5…) for the final 11.0 builds
will depend on bug reports we might received on RC 3 is out, and our
ability (and confidence) with respect to triaging/fixing issues in a
timely manner. If RC 3 doesn't end in seasonal fireworks, I'd be happy
to defer any more code changes to 11.1… Time (and tests) will tell!


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web