Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #66464 > unrolled thread

virtio_mmio.device parameter unknown as kernel config option is disabled

Back to article view | Back to linux.debian.kernel

Contents

  virtio_mmio.device parameter unknown as kernel config option is disabled João Mikos <j.eduardo@gmail.com> - 2020-03-05 22:40 +0100
    Re: virtio_mmio.device parameter unknown as kernel config option is  disabled Noah Meyerhans <noahm@debian.org> - 2020-03-06 00:50 +0100
      Re: virtio_mmio.device parameter unknown as kernel config option is disabled João Mikos <j.eduardo@gmail.com> - 2020-03-06 09:50 +0100
        Re: virtio_mmio.device parameter unknown as kernel config option is  disabled Noah Meyerhans <noahm@debian.org> - 2020-03-07 20:40 +0100

#66464 — virtio_mmio.device parameter unknown as kernel config option is disabled

Dear Debian Kernel Team,

I am experimenting with running a Debian Buster microVM under the
Firecracker hypervisor, using a stock Debian kernel (converted to
vmlinux format) and an initrd.img file. When booting the microVM with
this setup, I receive the following error message inside the microVM,
which halts booting:

[ 2.136564] virtio_mmio: unknown parameter 'device' ignored

This is the command line passed by Firecracker to the kernel:

[ 0.000000] Command line: console=ttyS0 reboot=k panic=1 pci=off
root=/dev/vda rw virtio_mmio.device=4K@0xd0000000:5

I could trace this to the flag CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES
being disabled in the default Debian kernel config.

I have reconfigured the Debian Buster kernel with the flag
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y, enabling the ".device" parameter
to the virtio_mmio module. I have then installed the newly compiled
kernel in another host and regenerated an initrd.img image with the
new virtio_mmio module.

When I try to boot the microVM again with the recompiled module
everything works as expected.

So my questions would be:

1) Is there any specific reason for this flag to be disabled in the
Debian kernel?
2) If the flag is disabled for no specific reason, is there any chance
for this flag to be enabled in future Debian versions, so the
virtio_mmio module recognizes the .device parameter by default?

Best regards,
João

[toc] | [next] | [standalone]

#66465 — Re: virtio_mmio.device parameter unknown as kernel config option is disabled

On Thu, Mar 05, 2020 at 10:31:37PM +0100, João Mikos wrote:
> I am experimenting with running a Debian Buster microVM under the
> Firecracker hypervisor, using a stock Debian kernel (converted to
> vmlinux format) and an initrd.img file. When booting the microVM with
> this setup, I receive the following error message inside the microVM,
> which halts booting:

Hi João.  I have extensive experience with Firecracker, with both Debian
and other distros as the guest OS.  In general, the expectation with
Firecracker is that it will not work with distro kernels out-of-the-box,
but that it requires a custom kernel specifically configured for MicroVM
deployments.  Keep in mind that Firecracker originally didn't even
provide initramfs support, meaning that all device and filesystem
drivers needed to be statically linked into the kernel.  Although
Firecracker has since added initramfs support, its goal of minimal
feature sets and extremely fast boot times still encourages you to use a
customized kernel.

> I could trace this to the flag CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES
> being disabled in the default Debian kernel config.
...
> 1) Is there any specific reason for this flag to be disabled in the
> Debian kernel?

It's not generally useful, so it hasn't been deemed appropriate for
inclusion in the standard kernel builds.  It'll be up to the kernel team
to decide if supporting Firecracker with the default kernel is a
sufficiently interesting use-case to warrant changing it.  IMO, it is
not.

> 2) If the flag is disabled for no specific reason, is there any chance
> for this flag to be enabled in future Debian versions, so the
> virtio_mmio module recognizes the .device parameter by default?

You should open a wishlist severity bug against the src:linux package to
officially request this change.

noah

[toc] | [prev] | [next] | [standalone]

#66466

On Fri, Mar 6, 2020 at 12:23 AM Noah Meyerhans <noahm@debian.org> wrote:

> Hi João.  I have extensive experience with Firecracker, with both Debian
> and other distros as the guest OS.  In general, the expectation with
> Firecracker is that it will not work with distro kernels out-of-the-box,
> but that it requires a custom kernel specifically configured for MicroVM
> deployments.  Keep in mind that Firecracker originally didn't even
> provide initramfs support, meaning that all device and filesystem
> drivers needed to be statically linked into the kernel.  Although
> Firecracker has since added initramfs support, its goal of minimal
> feature sets and extremely fast boot times still encourages you to use a
> customized kernel.

Hey Noah, thanks for the quick response and for the insights on
Firecracker. Indeed, the main reason why I resumed these experiments
were because of the recent addition of initrd support.

>
> > I could trace this to the flag CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES
> > being disabled in the default Debian kernel config.
> ...
> > 1) Is there any specific reason for this flag to be disabled in the
> > Debian kernel?
>
> It's not generally useful, so it hasn't been deemed appropriate for
> inclusion in the standard kernel builds.  It'll be up to the kernel team
> to decide if supporting Firecracker with the default kernel is a
> sufficiently interesting use-case to warrant changing it.  IMO, it is
> not.

Reflecting about what you said the default kernel might indeed not be
the best place to enable this, but maybe the cloud kernel could be a
good candidate instead. Official support for Firecracker might be a
stretch, especially due to the goals you've mentioned, but I am seeing
projects (e.g. Ignite) using Firecracker as a basis. Any such derived
project would be benefited by this change. But indeed, the kernel team
has the best view on whether they want to support (with all that
entails) any new flag or feature.

>
> > 2) If the flag is disabled for no specific reason, is there any chance
> > for this flag to be enabled in future Debian versions, so the
> > virtio_mmio module recognizes the .device parameter by default?
>
> You should open a wishlist severity bug against the src:linux package to
> officially request this change.

Thanks for the tip! I'll follow this direction and see what the team
thinks about it. Worst case scenario if anyone even stumbles upon a
similar issue the documentation of what to do is now here for
posterity.

Best regards,
João

On Fri, Mar 6, 2020 at 12:23 AM Noah Meyerhans <noahm@debian.org> wrote:
>
> On Thu, Mar 05, 2020 at 10:31:37PM +0100, João Mikos wrote:
> > I am experimenting with running a Debian Buster microVM under the
> > Firecracker hypervisor, using a stock Debian kernel (converted to
> > vmlinux format) and an initrd.img file. When booting the microVM with
> > this setup, I receive the following error message inside the microVM,
> > which halts booting:
>
> Hi João.  I have extensive experience with Firecracker, with both Debian
> and other distros as the guest OS.  In general, the expectation with
> Firecracker is that it will not work with distro kernels out-of-the-box,
> but that it requires a custom kernel specifically configured for MicroVM
> deployments.  Keep in mind that Firecracker originally didn't even
> provide initramfs support, meaning that all device and filesystem
> drivers needed to be statically linked into the kernel.  Although
> Firecracker has since added initramfs support, its goal of minimal
> feature sets and extremely fast boot times still encourages you to use a
> customized kernel.
>
> > I could trace this to the flag CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES
> > being disabled in the default Debian kernel config.
> ...
> > 1) Is there any specific reason for this flag to be disabled in the
> > Debian kernel?
>
> It's not generally useful, so it hasn't been deemed appropriate for
> inclusion in the standard kernel builds.  It'll be up to the kernel team
> to decide if supporting Firecracker with the default kernel is a
> sufficiently interesting use-case to warrant changing it.  IMO, it is
> not.
>
> > 2) If the flag is disabled for no specific reason, is there any chance
> > for this flag to be enabled in future Debian versions, so the
> > virtio_mmio module recognizes the .device parameter by default?
>
> You should open a wishlist severity bug against the src:linux package to
> officially request this change.
>
> noah
>


-- 
j.

[toc] | [prev] | [next] | [standalone]

#66471 — Re: virtio_mmio.device parameter unknown as kernel config option is disabled

On Fri, Mar 06, 2020 at 09:44:26AM +0100, João Mikos wrote:
> Reflecting about what you said the default kernel might indeed not be
> the best place to enable this, but maybe the cloud kernel could be a
> good candidate instead. Official support for Firecracker might be a
> stretch, especially due to the goals you've mentioned, but I am seeing
> projects (e.g. Ignite) using Firecracker as a basis. Any such derived
> project would be benefited by this change. But indeed, the kernel team
> has the best view on whether they want to support (with all that
> entails) any new flag or feature.

On the contrary, tools that integrate Firecracker directly, like Ignite,
Kata Containers, and firecracker-containerd, are the ones that will
benefit the most from using a custom kernel.  Indeed, all of those
projects provide their own kernel images for use with Firecracker.
These kernel images are configured to omit drivers for devices that are
not present in Firecracker's device model, some of of which lead to long
delays in the boot process while they probe for nonexistent devices.
Support for Firecracker in the default Debian kernel could be
interesting for experimentation, but I think you'd want to switch to a
customized kernel as soon as possible before running in production.

Given that the kernel in a Firecracker MicroVM is loaded by the host,
rather than being embedded in the guest VM image, I think a kernel
supporting Firecracker might be better handled in a manner similar to
the user-mode-linux kernel package.
https://packages.debian.org/buster/user-mode-linux  That would be a
package that'd be installed on the system hosting MicroVMs, where it
could provide an uncompressed kernel image for use by Firecracker.

noah

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web