Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1417497 > unrolled thread

Using DT overlays for adding virtual hardware

Started byJan Kiszka <jan.kiszka@siemens.com>
First post2016-06-08 16:20 +0200
Last post2016-06-08 18:40 +0200
Articles 10 — 4 participants

Back to article view | Back to linux.kernel


Contents

  Using DT overlays for adding virtual hardware Jan Kiszka <jan.kiszka@siemens.com> - 2016-06-08 16:20 +0200
    Re: Using DT overlays for adding virtual hardware Mark Rutland <mark.rutland@arm.com> - 2016-06-08 17:20 +0200
      Re: Using DT overlays for adding virtual hardware Jan Kiszka <jan.kiszka@siemens.com> - 2016-06-08 17:30 +0200
      Re: Using DT overlays for adding virtual hardware Pantelis Antoniou <pantelis.antoniou@konsulko.com> - 2016-06-08 18:00 +0200
        Re: Using DT overlays for adding virtual hardware Mark Rutland <mark.rutland@arm.com> - 2016-06-08 18:30 +0200
          Re: Using DT overlays for adding virtual hardware Jan Kiszka <jan.kiszka@siemens.com> - 2016-06-08 18:40 +0200
            Re: Using DT overlays for adding virtual hardware Jan Kiszka <jan.kiszka@siemens.com> - 2016-06-09 08:10 +0200
            Re: Using DT overlays for adding virtual hardware Arnd Bergmann <arnd@arndb.de> - 2016-06-09 09:30 +0200
              Re: Using DT overlays for adding virtual hardware Jan Kiszka <jan.kiszka@siemens.com> - 2016-06-10 17:00 +0200
          Re: Using DT overlays for adding virtual hardware Pantelis Antoniou <pantelis.antoniou@konsulko.com> - 2016-06-08 18:40 +0200

#1417497 — Using DT overlays for adding virtual hardware

FromJan Kiszka <jan.kiszka@siemens.com>
Date2016-06-08 16:20 +0200
SubjectUsing DT overlays for adding virtual hardware
Message-ID<rHMuT-1eQ-43@gated-at.bofh.it>
Hi all,

already started the discussion off-list with Pantelis, but it's better
done in public:

I'm currently exploring ways to make Linux recognize dynamically added
virtual hardware when running under the Jailhouse hypervisor [1]. We
need to load drivers for inter-partition communication devices that only
appear after Jailhouse started (which is done from within Linux, i.e.
long after boot) or when a partition was added later on. Probably, we
will simply add a virtual PCI host bridge on systems without physical
PCI and let the IPC device be explored that way (already works on x86).
Still, that leaves us with hotplug and unplug on hypervisor activation
and deactivation.

Inspired by how FPGA bitstream loaders and the caps/hats for BeagleBones
and RasPis work, I thought of using device tree overlays for this.
However, it seems there are still some gaps in upstream Linux /wrt
overlays. So I would like to find out what already works and what may
need further patches.

The plan (open for suggestions) is as follows:

- have an overlay template for a virtual platform device
- compile it into the Jailhouse loader kernel module
- adjust key parameters (e.g. register base addresses) after
  instantiating a device (of_update_property?)
- push modified dt into of_overlay_create

To my understanding, the last step will trigger driver probing which
will cause the device being activated (provided the driver is available,
of course).

I played with the fragment from
Documentation/devicetree/overlay-notes.txt, but already that failed
because the in-kernel dtc does not support /plugin/ yet, right? (Why
not, btw?) What else do I need to patch/update to make the above work?
Or are the simpler ways to achieve what we need?

Thanks,
Jan

[1] https://github.com/siemens/jailhouse

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[toc] | [next] | [standalone]


#1417574

FromMark Rutland <mark.rutland@arm.com>
Date2016-06-08 17:20 +0200
Message-ID<rHNqW-1PK-17@gated-at.bofh.it>
In reply to#1417497
On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
> Hi all,
> 
> already started the discussion off-list with Pantelis, but it's better
> done in public:
> 
> I'm currently exploring ways to make Linux recognize dynamically added
> virtual hardware when running under the Jailhouse hypervisor [1]. We
> need to load drivers for inter-partition communication devices that only
> appear after Jailhouse started (which is done from within Linux, i.e.
> long after boot) or when a partition was added later on. Probably, we
> will simply add a virtual PCI host bridge on systems without physical
> PCI and let the IPC device be explored that way (already works on x86).
> Still, that leaves us with hotplug and unplug on hypervisor activation
> and deactivation.

If I've understood correctly you want to use overlays to inject the
virtual PCI host bridge?

Given that you know precisely what you want to inject, I'm not sure I
see the value of using an overlay. 

Is there some reason you can't just create a device without having to go
via an intermediate step? As I understand it, Xen does that for (some)
virtual devices provided to Dom0 and DomU.

Thanks,
Mark.

[toc] | [prev] | [next] | [standalone]


#1417575

FromJan Kiszka <jan.kiszka@siemens.com>
Date2016-06-08 17:30 +0200
Message-ID<rHNAB-1Ti-7@gated-at.bofh.it>
In reply to#1417574
On 2016-06-08 17:17, Mark Rutland wrote:
> On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
>> Hi all,
>>
>> already started the discussion off-list with Pantelis, but it's better
>> done in public:
>>
>> I'm currently exploring ways to make Linux recognize dynamically added
>> virtual hardware when running under the Jailhouse hypervisor [1]. We
>> need to load drivers for inter-partition communication devices that only
>> appear after Jailhouse started (which is done from within Linux, i.e.
>> long after boot) or when a partition was added later on. Probably, we
>> will simply add a virtual PCI host bridge on systems without physical
>> PCI and let the IPC device be explored that way (already works on x86).
>> Still, that leaves us with hotplug and unplug on hypervisor activation
>> and deactivation.
> 
> If I've understood correctly you want to use overlays to inject the
> virtual PCI host bridge?
> 
> Given that you know precisely what you want to inject, I'm not sure I
> see the value of using an overlay. 
> 
> Is there some reason you can't just create a device without having to go
> via an intermediate step? As I understand it, Xen does that for (some)
> virtual devices provided to Dom0 and DomU.

I don't want to write yet another virtual driver but ideally reuse
pci-host-generic. Possibly a dumb question, but bow can I make Linux
instantiate a device node compatible with that driver and with the
desired resources? That's what made me think of overlays.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[toc] | [prev] | [next] | [standalone]


#1417618

FromPantelis Antoniou <pantelis.antoniou@konsulko.com>
Date2016-06-08 18:00 +0200
Message-ID<rHO3J-24Z-35@gated-at.bofh.it>
In reply to#1417574
Hi Mark,

> On Jun 8, 2016, at 18:17 , Mark Rutland <mark.rutland@arm.com> wrote:
> 
> On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
>> Hi all,
>> 
>> already started the discussion off-list with Pantelis, but it's better
>> done in public:
>> 
>> I'm currently exploring ways to make Linux recognize dynamically added
>> virtual hardware when running under the Jailhouse hypervisor [1]. We
>> need to load drivers for inter-partition communication devices that only
>> appear after Jailhouse started (which is done from within Linux, i.e.
>> long after boot) or when a partition was added later on. Probably, we
>> will simply add a virtual PCI host bridge on systems without physical
>> PCI and let the IPC device be explored that way (already works on x86).
>> Still, that leaves us with hotplug and unplug on hypervisor activation
>> and deactivation.
> 
> If I've understood correctly you want to use overlays to inject the
> virtual PCI host bridge?
> 
> Given that you know precisely what you want to inject, I'm not sure I
> see the value of using an overlay. 
> 
> Is there some reason you can't just create a device without having to go
> via an intermediate step? As I understand it, Xen does that for (some)
> virtual devices provided to Dom0 and DomU.
> 

As far as I understand it PCI is just one of the cases. You could conceivably
inject any kind of virtio device like serial/storage networking etc.

The question is since overlays exist and do work, why should he do anything else
besides using them? I doubt he’s interested in writing yet another driver for
something that already exists and works.

> Thanks,
> Mark.

Regards

— Pantelis

[toc] | [prev] | [next] | [standalone]


#1417651

FromMark Rutland <mark.rutland@arm.com>
Date2016-06-08 18:30 +0200
Message-ID<rHOwG-2ud-15@gated-at.bofh.it>
In reply to#1417618
On Wed, Jun 08, 2016 at 06:57:37PM +0300, Pantelis Antoniou wrote:
> Hi Mark,
> 
> > On Jun 8, 2016, at 18:17 , Mark Rutland <mark.rutland@arm.com> wrote:
> > 
> > On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
> >> Hi all,
> >> 
> >> already started the discussion off-list with Pantelis, but it's better
> >> done in public:
> >> 
> >> I'm currently exploring ways to make Linux recognize dynamically added
> >> virtual hardware when running under the Jailhouse hypervisor [1]. We
> >> need to load drivers for inter-partition communication devices that only
> >> appear after Jailhouse started (which is done from within Linux, i.e.
> >> long after boot) or when a partition was added later on. Probably, we
> >> will simply add a virtual PCI host bridge on systems without physical
> >> PCI and let the IPC device be explored that way (already works on x86).
> >> Still, that leaves us with hotplug and unplug on hypervisor activation
> >> and deactivation.
> > 
> > If I've understood correctly you want to use overlays to inject the
> > virtual PCI host bridge?
> > 
> > Given that you know precisely what you want to inject, I'm not sure I
> > see the value of using an overlay. 
> > 
> > Is there some reason you can't just create a device without having to go
> > via an intermediate step? As I understand it, Xen does that for (some)
> > virtual devices provided to Dom0 and DomU.
> 
> As far as I understand it PCI is just one of the cases. You could conceivably
> inject any kind of virtio device like serial/storage networking etc.

Sure, but we already have PCI transport for virtio devices, and per the
above PCI is the transport used on x86, so I assume that the devices we
really care about are going to be PCI anyhow.

> The question is since overlays exist and do work, why should he do anything else
> besides using them?

For one thing, they only work with DT, and there are ACPI ARM server
platforms out there, for which people may wish to use jailhouse. Tying
this to DT is not necessarily the best idea.

To be clear, I'm not arguing *against* overlays as such, just making
sure that we're not prematurely choosing a solution just becasue it's
the one we're aware of.

Thanks,
Mark.

[toc] | [prev] | [next] | [standalone]


#1417659

FromJan Kiszka <jan.kiszka@siemens.com>
Date2016-06-08 18:40 +0200
Message-ID<rHOGl-2xG-17@gated-at.bofh.it>
In reply to#1417651
On 2016-06-08 18:31, Pantelis Antoniou wrote:
> Hi Mark,
> 
>> On Jun 8, 2016, at 19:23 , Mark Rutland <mark.rutland@arm.com> wrote:
>>
>> On Wed, Jun 08, 2016 at 06:57:37PM +0300, Pantelis Antoniou wrote:
>>> Hi Mark,
>>>
>>>> On Jun 8, 2016, at 18:17 , Mark Rutland <mark.rutland@arm.com> wrote:
>>>>
>>>> On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
>>>>> Hi all,
>>>>>
>>>>> already started the discussion off-list with Pantelis, but it's better
>>>>> done in public:
>>>>>
>>>>> I'm currently exploring ways to make Linux recognize dynamically added
>>>>> virtual hardware when running under the Jailhouse hypervisor [1]. We
>>>>> need to load drivers for inter-partition communication devices that only
>>>>> appear after Jailhouse started (which is done from within Linux, i.e.
>>>>> long after boot) or when a partition was added later on. Probably, we
>>>>> will simply add a virtual PCI host bridge on systems without physical
>>>>> PCI and let the IPC device be explored that way (already works on x86).
>>>>> Still, that leaves us with hotplug and unplug on hypervisor activation
>>>>> and deactivation.
>>>>
>>>> If I've understood correctly you want to use overlays to inject the
>>>> virtual PCI host bridge?
>>>>
>>>> Given that you know precisely what you want to inject, I'm not sure I
>>>> see the value of using an overlay. 
>>>>
>>>> Is there some reason you can't just create a device without having to go
>>>> via an intermediate step? As I understand it, Xen does that for (some)
>>>> virtual devices provided to Dom0 and DomU.
>>>
>>> As far as I understand it PCI is just one of the cases. You could conceivably
>>> inject any kind of virtio device like serial/storage networking etc.
>>
>> Sure, but we already have PCI transport for virtio devices, and per the
>> above PCI is the transport used on x86, so I assume that the devices we
>> really care about are going to be PCI anyhow.
>>
> 
> PCI on VMs is a hack, it’s all emulated.
> 
> We’re using it as crutch because it’s ubiquitous and is capable
> of probing, but it comes with a considerable amount of baggage.
> Jailhouse is a particular kind of a hypervisor where it is intended for
> safety critical applications and designed to be certified as such.
> The less amount of code it contains the better, and much easier to certify. 

That's true, but we already have to live with PCI on x86, thus the code
is there, and it's more and more present on ARM[64] as well.

We are trying hard to make it simple, primarily for the hypervisor, and
if it is simpler to plug a trivial virtual PCI bridge than to emulate
individual platform IPC devices - also fine.

> 
>>> The question is since overlays exist and do work, why should he do anything else
>>> besides using them?
>>
>> For one thing, they only work with DT, and there are ACPI ARM server
>> platforms out there, for which people may wish to use jailhouse. Tying
>> this to DT is not necessarily the best idea.
>>
> 
> I just don’t see how an ACPI based hypervisor can ever be certified for
> safety critical applications. It might be possible but it should be
> an enormous undertaking; perhaps a subset without AML, but then again
> can you even boot an ACPI box without it?

ACPI is out of scope for us. We will probably continue to feed the
hypervisor with static platform information, generated in advance and
validated. Can be DT-based one day, but even that is more complex to
parse than our current structures.

But does ACPI usually mean that the kernel no longer has DT support and
would not be able to handle any overlay? That could be a killer.

> 
> DT is safer since it contains state only.
> 
>> To be clear, I'm not arguing *against* overlays as such, just making
>> sure that we're not prematurely choosing a solution just becasue it's
>> the one we're aware of.

I'm open for any suggestion that is simple. Maybe we can extend a
trivial existing pci host driver (like pci-host-generic) to work also
without DT overlays - also fine, at least from Jailhose POV. However,
any unneeded kernel patch is even better.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[toc] | [prev] | [next] | [standalone]


#1418098

FromJan Kiszka <jan.kiszka@siemens.com>
Date2016-06-09 08:10 +0200
Message-ID<rI1kd-2sF-13@gated-at.bofh.it>
In reply to#1417659
On 2016-06-08 18:39, Jan Kiszka wrote:
>>>> The question is since overlays exist and do work, why should he do anything else
>>>> besides using them?
>>>
>>> For one thing, they only work with DT, and there are ACPI ARM server
>>> platforms out there, for which people may wish to use jailhouse. Tying
>>> this to DT is not necessarily the best idea.
>>>
>>
>> I just don’t see how an ACPI based hypervisor can ever be certified for
>> safety critical applications. It might be possible but it should be
>> an enormous undertaking; perhaps a subset without AML, but then again
>> can you even boot an ACPI box without it?
> 
> ACPI is out of scope for us. We will probably continue to feed the
> hypervisor with static platform information, generated in advance and
> validated. Can be DT-based one day, but even that is more complex to
> parse than our current structures.
> 
> But does ACPI usually mean that the kernel no longer has DT support and
> would not be able to handle any overlay? That could be a killer.

However, I suspect that those machines with ACPI will also come with
PCI, in which case we do not need the virtual host bridge anyway.

> 
>>
>> DT is safer since it contains state only.
>>
>>> To be clear, I'm not arguing *against* overlays as such, just making
>>> sure that we're not prematurely choosing a solution just becasue it's
>>> the one we're aware of.
> 
> I'm open for any suggestion that is simple. Maybe we can extend a
> trivial existing pci host driver (like pci-host-generic) to work also
> without DT overlays - also fine, at least from Jailhose POV. However,
> any unneeded kernel patch is even better.

OK, trial and error, and some interesting insights: I've played with DT
fragments and the overlay configfs patch of Pantelis [1] to have a
convenient start. Interestingly, I wasn't able to load a fragment that
followed the format specification for overlays ("Failed to resolve
tree"). By chance, I got this one working:

/dts-v1/;
/ {
	fragment {
		target-path = "/soc@01c00000";
		__overlay__ {
			#address-cells = <2>;
			#size-cells = <2>;

			vpci@0x2000000 {
				compatible = "pci-host-cam-generic";
				device_type = "pci";
				#address-cells = <3>;
				#size-cells = <2>;
				reg = <0 0x2000000 0 0x1000000>;
				ranges =
					<0x02000000 0x00 0x10000000 0x00 0x10000000 0x00 0x30000000>;
			};
		};
	};
};

It successfully makes a BananaPi kernel add a pci host with the
specified config space and MMIO window.

[   81.619583] PCI host bridge /soc@01c00000/vpci@0x2000000 ranges:
[   81.619610]   No bus range found for /soc@01c00000/vpci@0x2000000, using [bus 00-ff]
[   81.619634]   MEM 0x10000000..0x3fffffff -> 0x10000000
[   81.620482] pci-host-generic 2000000.vpci: ECAM at [mem 0x02000000-0x02ffffff] for [bus 00-ff]
[   81.620779] pci-host-generic 2000000.vpci: PCI host bridge to bus 0000:00
[   81.620801] pci_bus 0000:00: root bus resource [bus 00-ff]
[   81.620814] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3fffffff]
[   81.620851] PCI: bus0: Fast back to back transfers enabled

So, no /plugin/ statement, no phandles resolution. This format even
builds with the in-kernel dtc. Any explanations? Does the code make
sense (at least it builds without warnings)?

Now I need to back this with some code in Jailhouse.

Jan

[1] https://github.com/pantoniou/linux-beagle-track-mainline/commit/160e68ec89eca33e8ed0abb13d52c07c54d7fc10

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[toc] | [prev] | [next] | [standalone]


#1418124

FromArnd Bergmann <arnd@arndb.de>
Date2016-06-09 09:30 +0200
Message-ID<rI2zE-3dZ-5@gated-at.bofh.it>
In reply to#1417659
On Wednesday, June 8, 2016 6:39:08 PM CEST Jan Kiszka wrote:
> >>
> > 
> > I just don’t see how an ACPI based hypervisor can ever be certified for
> > safety critical applications. It might be possible but it should be
> > an enormous undertaking; perhaps a subset without AML, but then again
> > can you even boot an ACPI box without it?
> 
> ACPI is out of scope for us. We will probably continue to feed the
> hypervisor with static platform information, generated in advance and
> validated. Can be DT-based one day, but even that is more complex to
> parse than our current structures.
> 
> But does ACPI usually mean that the kernel no longer has DT support and
> would not be able to handle any overlay? That could be a killer.

The kernel always has DT support built-in, but there may be some code
paths that do not look at DT properties when it was booted from ACPI.

In particular, communicating things like interrupt mappings may be
hard, as they are represented very differently on ACPI, so you no
longer have an 'interrupt-parent' node to point to from your overlay.

It's hard to say how things would work out when trying to load DT
overlays in this configuration. My guess is that it's actually
easier to do on x86 (which doesn't normally rely on ACPI for
describing the core system) than on arm64.

> > DT is safer since it contains state only.
> > 
> >> To be clear, I'm not arguing *against* overlays as such, just making
> >> sure that we're not prematurely choosing a solution just becasue it's
> >> the one we're aware of.
> 
> I'm open for any suggestion that is simple. Maybe we can extend a
> trivial existing pci host driver (like pci-host-generic) to work also
> without DT overlays - also fine, at least from Jailhose POV. However,
> any unneeded kernel patch is even better.

A few more observations:

- you can easily have an arbitrary number of PCI host bridges, so you
  can always add another PCI bridge just for the virtual devices even
  on systems that have access to physical PCI devices in passthrough.

- PCIe hotplugging seems well-defined enough to just make that work,
  without needing DT overlays.

- The really tricky question is what to do about passthrough of
  host devices that are not PCI. The current generation of server
  class arm64 machines tend to have a bunch of those, and the
  expectation seems to be that hardware passthrough is the only
  way to get decent I/O performance to make up for the relatively
  slow CPU cores. If you are only concerned about emulated devices,
  that won't be a problem though.

	Arnd

[toc] | [prev] | [next] | [standalone]


#1419503

FromJan Kiszka <jan.kiszka@siemens.com>
Date2016-06-10 17:00 +0200
Message-ID<rIw4G-5RL-17@gated-at.bofh.it>
In reply to#1418124
On 2016-06-09 09:22, Arnd Bergmann wrote:
> On Wednesday, June 8, 2016 6:39:08 PM CEST Jan Kiszka wrote:
>>>>
>>>
>>> I just don’t see how an ACPI based hypervisor can ever be certified for
>>> safety critical applications. It might be possible but it should be
>>> an enormous undertaking; perhaps a subset without AML, but then again
>>> can you even boot an ACPI box without it?
>>
>> ACPI is out of scope for us. We will probably continue to feed the
>> hypervisor with static platform information, generated in advance and
>> validated. Can be DT-based one day, but even that is more complex to
>> parse than our current structures.
>>
>> But does ACPI usually mean that the kernel no longer has DT support and
>> would not be able to handle any overlay? That could be a killer.
> 
> The kernel always has DT support built-in, but there may be some code
> paths that do not look at DT properties when it was booted from ACPI.
> 
> In particular, communicating things like interrupt mappings may be
> hard, as they are represented very differently on ACPI, so you no
> longer have an 'interrupt-parent' node to point to from your overlay.
> 
> It's hard to say how things would work out when trying to load DT
> overlays in this configuration. My guess is that it's actually
> easier to do on x86 (which doesn't normally rely on ACPI for
> describing the core system) than on arm64.

OK. But let's see if there are really systems with ACPI and without
pre-existing PCI. Currently, I would say the probability is low, because
ACPI means server, and servers love PCI...

> 
>>> DT is safer since it contains state only.
>>>
>>>> To be clear, I'm not arguing *against* overlays as such, just making
>>>> sure that we're not prematurely choosing a solution just becasue it's
>>>> the one we're aware of.
>>
>> I'm open for any suggestion that is simple. Maybe we can extend a
>> trivial existing pci host driver (like pci-host-generic) to work also
>> without DT overlays - also fine, at least from Jailhose POV. However,
>> any unneeded kernel patch is even better.
> 
> A few more observations:
> 
> - you can easily have an arbitrary number of PCI host bridges, so you
>   can always add another PCI bridge just for the virtual devices even
>   on systems that have access to physical PCI devices in passthrough.
> 
> - PCIe hotplugging seems well-defined enough to just make that work,
>   without needing DT overlays.

The point is about adding virtual devices when there is no physical PCI
- when there is, we can already sneak them in between physical ones.

Granted, when we run out of free slots, there is a need to do more,
either via virtual bridges (but hypervisor is the last place we'd like
to touch), by enforcing Linux to scan on slots outside of the physical
topology or by making it create bridge stubs for virtual devices that
are not assigned to a physical bus. But that's all PCI topics, not
directly related to the original point of adding the host bridge.

> 
> - The really tricky question is what to do about passthrough of
>   host devices that are not PCI. The current generation of server
>   class arm64 machines tend to have a bunch of those, and the
>   expectation seems to be that hardware passthrough is the only
>   way to get decent I/O performance to make up for the relatively
>   slow CPU cores. If you are only concerned about emulated devices,
>   that won't be a problem though.

Yes, that is tricky, but more from the analytical POV: which devices or
which parts of devices can we hand out to guests without jeopardizing
the system integrity? No generic answers here, for sure.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[toc] | [prev] | [next] | [standalone]


#1417663

FromPantelis Antoniou <pantelis.antoniou@konsulko.com>
Date2016-06-08 18:40 +0200
Message-ID<rHOGm-2xG-19@gated-at.bofh.it>
In reply to#1417651
Hi Mark,

> On Jun 8, 2016, at 19:23 , Mark Rutland <mark.rutland@arm.com> wrote:
> 
> On Wed, Jun 08, 2016 at 06:57:37PM +0300, Pantelis Antoniou wrote:
>> Hi Mark,
>> 
>>> On Jun 8, 2016, at 18:17 , Mark Rutland <mark.rutland@arm.com> wrote:
>>> 
>>> On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
>>>> Hi all,
>>>> 
>>>> already started the discussion off-list with Pantelis, but it's better
>>>> done in public:
>>>> 
>>>> I'm currently exploring ways to make Linux recognize dynamically added
>>>> virtual hardware when running under the Jailhouse hypervisor [1]. We
>>>> need to load drivers for inter-partition communication devices that only
>>>> appear after Jailhouse started (which is done from within Linux, i.e.
>>>> long after boot) or when a partition was added later on. Probably, we
>>>> will simply add a virtual PCI host bridge on systems without physical
>>>> PCI and let the IPC device be explored that way (already works on x86).
>>>> Still, that leaves us with hotplug and unplug on hypervisor activation
>>>> and deactivation.
>>> 
>>> If I've understood correctly you want to use overlays to inject the
>>> virtual PCI host bridge?
>>> 
>>> Given that you know precisely what you want to inject, I'm not sure I
>>> see the value of using an overlay. 
>>> 
>>> Is there some reason you can't just create a device without having to go
>>> via an intermediate step? As I understand it, Xen does that for (some)
>>> virtual devices provided to Dom0 and DomU.
>> 
>> As far as I understand it PCI is just one of the cases. You could conceivably
>> inject any kind of virtio device like serial/storage networking etc.
> 
> Sure, but we already have PCI transport for virtio devices, and per the
> above PCI is the transport used on x86, so I assume that the devices we
> really care about are going to be PCI anyhow.
> 

PCI on VMs is a hack, it’s all emulated.

We’re using it as crutch because it’s ubiquitous and is capable
of probing, but it comes with a considerable amount of baggage.
Jailhouse is a particular kind of a hypervisor where it is intended for
safety critical applications and designed to be certified as such.
The less amount of code it contains the better, and much easier to certify. 

>> The question is since overlays exist and do work, why should he do anything else
>> besides using them?
> 
> For one thing, they only work with DT, and there are ACPI ARM server
> platforms out there, for which people may wish to use jailhouse. Tying
> this to DT is not necessarily the best idea.
> 

I just don’t see how an ACPI based hypervisor can ever be certified for
safety critical applications. It might be possible but it should be
an enormous undertaking; perhaps a subset without AML, but then again
can you even boot an ACPI box without it?

DT is safer since it contains state only.

> To be clear, I'm not arguing *against* overlays as such, just making
> sure that we're not prematurely choosing a solution just becasue it's
> the one we're aware of.
> 
> Thanks,
> Mark.

Regards

— Pantelis

[toc] | [prev] | [standalone]


Back to top | Article view | linux.kernel


csiph-web