Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1315286 > unrolled thread

[PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-22 22:40 +0100
    Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-23 00:40 +0100
      Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Andrew Cooper <andrew.cooper3@citrix.com> - 2016-01-23 01:40 +0100
        Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-23 01:50 +0100
        Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-23 02:00 +0100
          Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Andrew Cooper <andrew.cooper3@citrix.com> - 2016-01-23 15:50 +0100
            Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "H. Peter Anvin" <hpa@zytor.com> - 2016-01-23 17:10 +0100
              Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-01-23 17:20 +0100
                Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "H. Peter Anvin" <hpa@zytor.com> - 2016-01-23 19:30 +0100
                Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Roger Pau Monné <royger@FreeBSD.org> - 2016-01-25 11:40 +0100
            Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-25 23:20 +0100
              Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-26 00:00 +0100
                Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-26 21:40 +0100
                  Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-26 23:00 +0100
                    Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-27 01:10 +0100
                      Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-27 03:20 +0100
                        Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest David Vrabel <david.vrabel@citrix.com> - 2016-01-27 16:00 +0100
                          Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-27 16:10 +0100
                            Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest David Vrabel <david.vrabel@citrix.com> - 2016-01-27 16:10 +0100
                              Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-27 16:20 +0100
                                Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-27 17:20 +0100
                                  Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-01-27 19:50 +0100
                                    Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-01-27 20:10 +0100
                                      Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-28 01:00 +0100
                          Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Borislav Petkov <bp@suse.de> - 2016-01-27 17:20 +0100
            Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-01-26 17:20 +0100
        Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-25 16:10 +0100
      Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-25 17:10 +0100
        Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-25 22:20 +0100
          Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "H. Peter Anvin" <hpa@zytor.com> - 2016-01-25 22:30 +0100
            Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-25 23:30 +0100
              Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-26 19:40 +0100
                Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Andy Lutomirski <luto@amacapital.net> - 2016-01-26 19:50 +0100
                  Re: [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2016-01-26 20:10 +0100
                    Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest "Luis R. Rodriguez" <mcgrof@suse.com> - 2016-01-26 20:20 +0100

Page 2 of 2 — ← Prev page 1 [2]

#1319125 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On 01/27/2016 10:29 AM, Konrad Rzeszutek Wilk wrote:
> On Wed, Jan 27, 2016 at 10:17:56AM -0500, Boris Ostrovsky wrote:
>> On 01/27/2016 10:09 AM, David Vrabel wrote:
>>> On 27/01/16 15:06, Boris Ostrovsky wrote:
>>>> On 01/27/2016 09:50 AM, David Vrabel wrote:
>>>>> On 27/01/16 14:42, Konrad Rzeszutek Wilk wrote:
>>>>>> On Tue, Jan 26, 2016 at 08:54:56PM -0800, Luis R. Rodriguez wrote:
>>>>>>> On Jan 26, 2016 6:16 PM, "Luis R. Rodriguez" <mcgrof@suse.com> wrote:
>>>>>>>> On Tue, Jan 26, 2016 at 4:04 PM, Luis R. Rodriguez <mcgrof@suse.com>
>>>>>>> wrote:
>>>>>>>>> You go:
>>>>>>>>>
>>>>>>>>> hvmlite_start_xen() -->
>>>>>>>>>           HVM stub
>>>>>>>>>           startup_64() | (startup_32()
>>>>>>>> Hrm, does HVMlite work well with load_ucode_bsp(), note the patches to
>>>>>>>> rebrand pv_enabled() to pv_legacy() or whatever, this PV type will not
>>>>>>>> be legacy or crap / old, so we'd need a way to catch it if we should
>>>>>>>> not use that code for this PV type. This begs the question, are you
>>>>>>>> also sure other callers in startup_32() or startup_64() might be OK as
>>>>>>>> well where previously guarded with pv_enabled() ?
>>>>>>> Actually this call can't be used, and if early code used it prior to
>>>>>>> setup_arch() it'd be a bug as its only properly set until later.
>>>>>>> Vetting
>>>>>>> for correctness of all code call is still required though and
>>>>>>> perhaps we do
>>>>>>> need something to catch now this PV type on early code such as this
>>>>>>> one if
>>>>>>> we don't want it. From what I've gathered before on other bsp ucode we
>>>>>>> don't want ucode loaded for PV guest types through these mechanisms.
>>>>>> It may help to not think of PVH/hvmlite as PV. It really is HVM with
>>>>>> a lot
>>>>>> of emulated devices removed.
>>>>>>
>>>>>> How does early microcode work on EFI? Does the EFI stub code have an
>>>>>> early
>>>>>> microcode loading code ?
>>>>> Surely the interesting comparison here is how is (early) microcode
>>>>> loading disabled in KVM guests?  We should use the same mechanism for
>>>             ^^^^^^^^
>>>>> HVMlite guests.
>>>> Why would we ever want to have a guest load microcode during boot? I can
>>>> see how a (privileged) guest may want to load microcode from a shell
>>>> (via microcode driver).
>>> I think you missed a word when you read my reply.
>> Yes, I missed it ;-)
>>
>> Why not continue relying on paravirt_enabled()? We are going to keep this in
>> some form for HVMlite.
> And this is where Luis comes in. He has posted an patchset which removes the
> paravirt_enabled with .. Here is the link https://lkml.org/lkml/2015/12/15/772

Yes, I saw that and this will be renamed as paravirt_legacy() (which I 
am not sure is really what it should be called.)

Another option is to have early microcode code query CPUID to see 
whether we are running on a hypervisor (this in fact is what we 
originally thought of doing before realizing that we have 
paravirt_enabled()).

But then how is HVMlite different from a regular HVM guest trying to 
load microcode?

(BTW, to answer David's question about what KVM is doing --- it is 
ignoring writes to microcode MSRs, see kvm_set_msr_common().)

-boris

[toc] | [prev] | [next] | [standalone]

#1319337 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

Bleh moving forward please use mcgrof@kernel.org, that will be sent to
my employer and my personal address. More below.

On Wed, Jan 27, 2016 at 8:15 AM, Boris Ostrovsky
<boris.ostrovsky@oracle.com> wrote:
> On 01/27/2016 10:29 AM, Konrad Rzeszutek Wilk wrote:
>> On Wed, Jan 27, 2016 at 10:17:56AM -0500, Boris Ostrovsky wrote:
>>> On 01/27/2016 10:09 AM, David Vrabel wrote:
>>>> On 27/01/16 15:06, Boris Ostrovsky wrote:
>>> Why not continue relying on paravirt_enabled()? We are going to keep this
>>> in
>>> some form for HVMlite.
>>
>> And this is where Luis comes in. He has posted an patchset which removes
>> the
>> paravirt_enabled with .. Here is the link
>> https://lkml.org/lkml/2015/12/15/772
>
>
> Yes, I saw that and this will be renamed as paravirt_legacy() (which I am
> not sure is really what it should be called.)

Given Konrad's pointers about some folks pushing for BIOS to have a
"legacy free option" where all legacy crap (PS/2, PnP, serial port,
parallel port) are all disabled [0], I'm inclined to respin the rename
patch to use x86_legacy_free(). This can later then be extended
provided such BIOS check becomes available.

[0] http://lkml.kernel.org/r/20160120193241.GA4769@char.us.oracle.com

But -- as I also pointed out to hpa recently, there is an issue with
using things like cpu_has_hypervisor() (or in this case
paravirt_enabled()) for early code, given that it relies on
init_hypervisor_platform() having been called first which is called on
setup_arch() [1]. So paravirt_enabled() really can only be used prior
to setup_arch() correctly (specifically after,
init_hypervisor_platform()), and anything else would be a bug. I
learned the hard way while doing some linker table work and trying to
find a good use / solution to the dead code concerns I've been aiming
to address due to Xen's separate entry point and the nature of pvops
[2] [3]. hpa's response to this issue was that we cannot and should
not abuse the boot_params hardware_subarch for this purpose (in this
case I was suggesting perhaps KVM could use it if it in the future
needed it for a kvm check earlier than setup_arch()), but he noted
that:

  "If you have a genuine need for a "hypervisor type" then that is a
separate thing and should be
   treated separately from subarch.  However, you need to consider
that some hypervisors can
  emulate other hypervisors and you may have more than one hypervisor
API available."

This goes along with my suggestion here earlier where I mentioned that
subarch is already used nicely to pivot off some specific subarch
entry after startup_32(), if we want to avoid yet-another-entry for
HVMlite (the PVH rebrand done cleanly) and still use startup_32() for
it we could perhaps follow similar strategy as with subarch but
instead add a hypervisor type and use that for a stub call the
hypervisor type if needed early on in startup_32(). This could perhaps
in turn also be used as a generic hypervisor type / and more robust /
easier / not-so-convoluted check. For instance of what I think is a
convoluted check refer to snd_intel8x0_inside_vm() in
sound/pci/intel8x0.c with its use of kvm_para_available(), #ifdefery
over boot_cpu_has(X86_FEATURE_HYPERVISOR) (which is just
cpu_has_hypervisor()). If we had a hypervisor type easily accessible
it could both be used by early init code and perhaps driver code to
replace these convoluted checks.

If this seems a bit sensible the next question to ask if -- how we'd
*set* the hypervisor type, do we extend the x86 boot protocol and
enable a hypervisor type on the zero page? That would clearly only
make sense if:

1) Its reasonable to x86 maintainers (perhaps pending this
discussion's outcome? If its preemptively known to not reasonable an
alternative suggestion would be appreciated)
2) Xen is willing to actually set this (and perhaps a new
hypervisor_type_data for the private data structure), but not much
more would be needed, and incorporate this as part of the DmLite
protocol, or whatever its called as an option for some OSes
3) The kernel could get access to this value from the zero page really
early on, immediately following startup_32()

Worth mentioning here also is hpa's clarification on when subarch type
PC (0) should be used: [it should be used if the hardware is]
"enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need
a special boot flow" -- does that fit HVMLite's description so far? If
so then The Xen subarch may need to be redefined as well to be clear
what it means. I don't think we need to be precise but at the very
least cover grounds to enable the definitions to meet its actual use
to not confuse users.

[1] http://lkml.kernel.org/r/CAB=NE6WoKsP8+KGnJEtigWYktCMjg6iherCOcq-jskxi4P2QqA@mail.gmail.com
[2] http://www.do-not-panic.com/2015/12/avoiding-dead-code-pvops-not-silver-bullet.html
[3] http://www.do-not-panic.com/2015/12/xen-and-x86-linux-zero-page.html
[4] http://lkml.kernel.org/r/56A17B01.9040708@zytor.com

> Another option is to have early microcode code query CPUID to see whether we
> are running on a hypervisor (this in fact is what we originally thought of
> doing before realizing that we have paravirt_enabled()).

Please keep in mind we want to consider use and setting of something
generic, not just a solution for Xen, and if we want to use this to
help avoid a new entry point then strive for something we actually
could get access to very early on the first x86 entry point, both for
32-bit and 64-bit. If its accessible for both 32-bit and 64-bit well
hell, we could unify not only the entry points for HVMLite thing, but
I'm pretty sure also unify the entry points for PV and x86 as well (if
we wanted that) !

  Luis

[toc] | [prev] | [next] | [standalone]

#1319413 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On Wed, Jan 27, 2016 at 10:48 AM, Luis R. Rodriguez <mcgrof@kernel.org> wrote:
>
> Worth mentioning here also is hpa's clarification on when subarch type
> PC (0) should be used: [it should be used if the hardware is]
> "enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need
> a special boot flow" -- does that fit HVMLite's description so far? If
> so then The Xen subarch may need to be redefined as well to be clear
> what it means. I don't think we need to be precise but at the very
> least cover grounds to enable the definitions to meet its actual use
> to not confuse users.

Another thing to consider for HVMlite is that if the 0 subarch (PC) is
used in light of my linker table work and x86's use of it with the
subarch and supported subarch bitmask, is that it would also mean
HVMLite would run all routines currently pegged as needing PC type
(the current KVM and bare metal path) and it would mean not running
anything only pegged with Xen subarch type (but note that today Xen
doesn't even set the subarch type). If there is nothing in common
between PV and HVMlite (no x86 init calls to share), and if HVMLite
*can* call *alllllllll* PC init calls, then by all means this is fine,
and if we just need to distinguish stuff between PC types that's fine,
it may still be possible to further extend hypervisor_type to the x86
init calls I'm adding as another supported_hyper_types to ensure even
though a subarch is being used, that we also check the supported
hypervisor type as well.

 Luis

[toc] | [prev] | [next] | [standalone]

#1320157 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On 01/27/2016 02:00 PM, Luis R. Rodriguez wrote:
> On Wed, Jan 27, 2016 at 10:48 AM, Luis R. Rodriguez <mcgrof@kernel.org> wrote:
>> Worth mentioning here also is hpa's clarification on when subarch type
>> PC (0) should be used: [it should be used if the hardware is]
>> "enumerable using standard PC mechanisms (PCI, ACPI) and doesn't need
>> a special boot flow" -- does that fit HVMLite's description so far? If
>> so then The Xen subarch may need to be redefined as well to be clear
>> what it means. I don't think we need to be precise but at the very
>> least cover grounds to enable the definitions to meet its actual use
>> to not confuse users.
> Another thing to consider for HVMlite is that if the 0 subarch (PC) is
> used in light of my linker table work and x86's use of it with the
> subarch and supported subarch bitmask, is that it would also mean
> HVMLite would run all routines currently pegged as needing PC type
> (the current KVM and bare metal path) and it would mean not running
> anything only pegged with Xen subarch type (but note that today Xen
> doesn't even set the subarch type). If there is nothing in common
> between PV and HVMlite (no x86 init calls to share), and if HVMLite
> *can* call *alllllllll* PC init calls, then by all means this is fine,

Yes, that's the idea. HVMlite jumps to startup_32|64 from the stub and 
runs from there with subarch 0.

-boris

> and if we just need to distinguish stuff between PC types that's fine,
> it may still be possible to further extend hypervisor_type to the x86
> init calls I'm adding as another supported_hyper_types to ensure even
> though a subarch is being used, that we also check the supported
> hypervisor type as well.
>
>   Luis

[toc] | [prev] | [next] | [standalone]

#1319122 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On Wed, Jan 27, 2016 at 02:50:36PM +0000, David Vrabel wrote:
> Surely the interesting comparison here is how is (early) microcode
> loading disabled in KVM guests?

It isn't - kvm simply ignores the write to the microcode application
MSRs MSR_AMD64_PATCH_LOADER and MSR_IA32_UCODE_REV, respectively.

> We should use the same mechanism for HVMlite guests.

Good idea.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[toc] | [prev] | [next] | [standalone]

#1318109 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

>However, this stub belongs in Linux, not in the Xen toolstack.  That
>way, when the Linux boot protocol is modified, both sides can be
>updated
>accordingly.

I would add that this idea is borrowed from the EFI stub code that Linux has which also constructs the boot parameter structure when invoked (either from firmware or from EFI shell).

[toc] | [prev] | [next] | [standalone]

#1316803 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On 01/22/2016 07:30 PM, Andrew Cooper wrote:
> On 22/01/2016 23:32, Luis R. Rodriguez wrote:
>> On Fri, Jan 22, 2016 at 04:35:50PM -0500, Boris Ostrovsky wrote:
>>> +	/*
>>> +	 * See Documentation/x86/boot.txt.
>>> +	 *
>>> +	 * Version 2.12 supports Xen entry point but we will use default x86/PC
>>> +	 * environment (i.e. hardware_subarch 0).
>>> +	 */
>>> +	xen_hvmlite_boot_params.hdr.version = 0x212;
>>> +	xen_hvmlite_boot_params.hdr.type_of_loader = 9; /* Xen loader */
>>> +}
>> I realize PV got away with setting up boot_params on C code but best
>> ask now that this new code is being introduced: why can't we just have
>> the Xen hypervisor fill this in? It'd save us all this code.
> I agree that this looks to be a mess.  Having said that, the DMLite boot
> protocol is OS agnostic, and will be staying that way.
>
> It happens to look suspiciously like multiboot; a flat 32bit protected
> mode entry (at a location chosen in an ELF note), with %ebx pointing to
> an in-ram structure containing things like a command line and module list.
>
> I would have though the correct way to do direct Linux support would be
> to have a very small init stub which constructs an appropriate zero
> page, and lets the native entry point get on with things.

Which is really what 
hvmlite_start_xen()->xen_prepare_hvmlite()->hvmlite_bootparams() is 
doing. Not much more than that (for 64-bit it also loads identity 
mapping because that's what startup_64 wants)

-boris

>
> This covers the usecase where people wish to boot a specific Linux
> kernel straight out of the dom0 filesystem.
>
> For the alternative usecase of general OS support, dom0 would boot
> something such as grub2 as the DMLite "kernel", at which point all
> stooging around in the guests filesystem is done from guest context,
> rather than control context (mitigating a substantial attack surface).
>
> ~Andrew

[toc] | [prev] | [next] | [standalone]

#1316931

On 01/22/2016 06:32 PM, Luis R. Rodriguez wrote:
> On Fri, Jan 22, 2016 at 04:35:50PM -0500, Boris Ostrovsky wrote:
>
>> +/*
>> + * This routine (and those that it might call) should not use
>> + * anything that lives in .bss since that segment will be cleared later
>> + */
>> +void __init xen_prepare_hvmlite(void)
>> +{
>> +	u32 eax, ecx, edx, msr;
>> +	u64 pfn;
>> +
>> +	cpuid(xen_cpuid_base() + 2, &eax, &msr, &ecx, &edx);
>> +	pfn = __pa(hypercall_page);
>> +	wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
>> +
>> +	pv_info.name = "Xen HVMlite";
>> +	xen_domain_type = XEN_HVM_DOMAIN;
>> +	xen_hvmlite = 1;
>> +
>> +	x86_init.oem.arch_setup = xen_init_kernel;
>> +	x86_init.oem.banner = xen_banner;
>> +
>> +	hvmlite_bootparams();
>> +}
>> +#endif
> If the boot_params.hdr.hardware_subarch_data pointed to a custom
> struct then the first C entry point for Xen could shuffle this and
> set this too, by still using less asm entry helpers. We'd still
> need this run but with the linker table I think we could have
> a stub small stub for hvm run, it would not be a call from asm.

Perhaps, but someone would still have to set hardware_subarch. And it's 
hvmlite_bootparams() that does it.

And that's not sufficient, I think. There are still some things that 
trampoline code sets up (e.g. page tables for 64-bit).

-boris

[toc] | [prev] | [next] | [standalone]

#1317306

On Mon, Jan 25, 2016 at 11:08:47AM -0500, Boris Ostrovsky wrote:
> On 01/22/2016 06:32 PM, Luis R. Rodriguez wrote:
> >On Fri, Jan 22, 2016 at 04:35:50PM -0500, Boris Ostrovsky wrote:
> >
> >>+/*
> >>+ * This routine (and those that it might call) should not use
> >>+ * anything that lives in .bss since that segment will be cleared later
> >>+ */
> >>+void __init xen_prepare_hvmlite(void)
> >>+{
> >>+	u32 eax, ecx, edx, msr;
> >>+	u64 pfn;
> >>+
> >>+	cpuid(xen_cpuid_base() + 2, &eax, &msr, &ecx, &edx);
> >>+	pfn = __pa(hypercall_page);
> >>+	wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32));
> >>+
> >>+	pv_info.name = "Xen HVMlite";
> >>+	xen_domain_type = XEN_HVM_DOMAIN;
> >>+	xen_hvmlite = 1;
> >>+
> >>+	x86_init.oem.arch_setup = xen_init_kernel;
> >>+	x86_init.oem.banner = xen_banner;
> >>+
> >>+	hvmlite_bootparams();
> >>+}
> >>+#endif
> >If the boot_params.hdr.hardware_subarch_data pointed to a custom
> >struct then the first C entry point for Xen could shuffle this and
> >set this too, by still using less asm entry helpers. We'd still
> >need this run but with the linker table I think we could have
> >a stub small stub for hvm run, it would not be a call from asm.
> 
> Perhaps, but someone would still have to set hardware_subarch. And
> it's hvmlite_bootparams() that does it.

No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
done in Xen.

> And that's not sufficient, I think. There are still some things that
> trampoline code sets up (e.g. page tables for 64-bit).

Sure, but only what is required, that should be rather smaller.

  Luis

[toc] | [prev] | [next] | [standalone]

#1317312

On 01/25/16 13:12, Luis R. Rodriguez wrote:
>>
>> Perhaps, but someone would still have to set hardware_subarch. And
>> it's hvmlite_bootparams() that does it.
> 
> No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
> done in Xen.
> 

Or a stub code.

	-hpa

[toc] | [prev] | [next] | [standalone]

#1317370

On 01/25/2016 04:21 PM, H. Peter Anvin wrote:
> On 01/25/16 13:12, Luis R. Rodriguez wrote:
>>> Perhaps, but someone would still have to set hardware_subarch. And
>>> it's hvmlite_bootparams() that does it.
>> No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
>> done in Xen.
>>
> Or a stub code.

This patch in fact is the stub for Xen HVMlite guests, after we are done 
with it we jump to bare-metal startup code (i.e startup_32|64)

-boris

[toc] | [prev] | [next] | [standalone]

#1318244

On Mon, Jan 25, 2016 at 05:28:08PM -0500, Boris Ostrovsky wrote:
> On 01/25/2016 04:21 PM, H. Peter Anvin wrote:
> >On 01/25/16 13:12, Luis R. Rodriguez wrote:
> >>>Perhaps, but someone would still have to set hardware_subarch. And
> >>>it's hvmlite_bootparams() that does it.
> >>No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
> >>done in Xen.
> >>
> >Or a stub code.
> 
> This patch in fact is the stub for Xen HVMlite guests, after we are
> done with it we jump to bare-metal startup code (i.e startup_32|64)

Right the point is the stub need not be in Linux, I'll explain in the other
thread where I provided more details on the different known approaches.

  Luis

[toc] | [prev] | [next] | [standalone]

#1318252

On Tue, Jan 26, 2016 at 10:34 AM, Luis R. Rodriguez <mcgrof@suse.com> wrote:
> On Mon, Jan 25, 2016 at 05:28:08PM -0500, Boris Ostrovsky wrote:
>> On 01/25/2016 04:21 PM, H. Peter Anvin wrote:
>> >On 01/25/16 13:12, Luis R. Rodriguez wrote:
>> >>>Perhaps, but someone would still have to set hardware_subarch. And
>> >>>it's hvmlite_bootparams() that does it.
>> >>No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
>> >>done in Xen.
>> >>
>> >Or a stub code.
>>
>> This patch in fact is the stub for Xen HVMlite guests, after we are
>> done with it we jump to bare-metal startup code (i.e startup_32|64)
>
> Right the point is the stub need not be in Linux, I'll explain in the other
> thread where I provided more details on the different known approaches.
>

ISTM if the Xen ABI-specified entry point has a different convention
than the Linux native entry, then the stub should live in Linux.  It
would be just a couple if lines of code, right?

The issue that caused headaches in the past isn't that there's code
that's executed only on native, it's that there are whole big
functions that are executed only on native for no good reason and that
aren't clearly marked.

If we had native_start_kernel and xen_start_kernel, and they both
called very quickly in to common_start_kernel, it would be very clear
what's going on.

--Andy

[toc] | [prev] | [next] | [standalone]

#1318266

On 01/26/2016 01:46 PM, Andy Lutomirski wrote:
> On Tue, Jan 26, 2016 at 10:34 AM, Luis R. Rodriguez <mcgrof@suse.com> wrote:
>> On Mon, Jan 25, 2016 at 05:28:08PM -0500, Boris Ostrovsky wrote:
>>> On 01/25/2016 04:21 PM, H. Peter Anvin wrote:
>>>> On 01/25/16 13:12, Luis R. Rodriguez wrote:
>>>>>> Perhaps, but someone would still have to set hardware_subarch. And
>>>>>> it's hvmlite_bootparams() that does it.
>>>>> No, Xen would do it as well, essentially all of hvmlite_bootparams() could be
>>>>> done in Xen.
>>>>>
>>>> Or a stub code.
>>> This patch in fact is the stub for Xen HVMlite guests, after we are
>>> done with it we jump to bare-metal startup code (i.e startup_32|64)
>> Right the point is the stub need not be in Linux, I'll explain in the other
>> thread where I provided more details on the different known approaches.
>>
> ISTM if the Xen ABI-specified entry point has a different convention
> than the Linux native entry, then the stub should live in Linux.  It
> would be just a couple if lines of code, right?

It's not exactly a couple of lines but it's not large neither. It mainly 
sets up boot_params (similar to what make_boot_params() does for EFI). 
Plus, for 64-bit, it loads identity page tables and switches to long 
mode. And then jumps to bare-meta startup code.


>
> The issue that caused headaches in the past isn't that there's code
> that's executed only on native, it's that there are whole big
> functions that are executed only on native for no good reason and that
> aren't clearly marked.

This won't happen with HVMlite.

>
> If we had native_start_kernel and xen_start_kernel, and they both
> called very quickly in to common_start_kernel, it would be very clear
> what's going on.

What is now xen_start_kernel() is no longer the entry point for HVMlite. 
It is called as x86_init.oem.arch_setup() (or I may even move it to 
x86_hyper_xen.init_platform() or something like that).

-boris

[toc] | [prev] | [next] | [standalone]

#1318269 — Re: [Xen-devel] [PATCH v1 04/12] xen/hvmlite: Bootstrap HVMlite guest

On Tue, Jan 26, 2016 at 11:00 AM, Boris Ostrovsky
<boris.ostrovsky@oracle.com> wrote:
> On 01/26/2016 01:46 PM, Andy Lutomirski wrote:
>>
>> On Tue, Jan 26, 2016 at 10:34 AM, Luis R. Rodriguez <mcgrof@suse.com>
>> wrote:
>>>
>>> On Mon, Jan 25, 2016 at 05:28:08PM -0500, Boris Ostrovsky wrote:
>>>>
>>>> On 01/25/2016 04:21 PM, H. Peter Anvin wrote:
>>>>>
>>>>> On 01/25/16 13:12, Luis R. Rodriguez wrote:
>>>>>>>
>>>>>>> Perhaps, but someone would still have to set hardware_subarch. And
>>>>>>> it's hvmlite_bootparams() that does it.
>>>>>>
>>>>>> No, Xen would do it as well, essentially all of hvmlite_bootparams()
>>>>>> could be
>>>>>> done in Xen.
>>>>>>
>>>>> Or a stub code.
>>>>
>>>> This patch in fact is the stub for Xen HVMlite guests, after we are
>>>> done with it we jump to bare-metal startup code (i.e startup_32|64)
>>>
>>> Right the point is the stub need not be in Linux, I'll explain in the
>>> other
>>> thread where I provided more details on the different known approaches.
>>>
>> ISTM if the Xen ABI-specified entry point has a different convention
>> than the Linux native entry, then the stub should live in Linux.  It
>> would be just a couple if lines of code, right?
>
>
> It's not exactly a couple of lines but it's not large neither. It mainly
> sets up boot_params (similar to what make_boot_params() does for EFI). Plus,
> for 64-bit, it loads identity page tables and switches to long mode. And
> then jumps to bare-meta startup code.

This terse summary provides an example of the issue I'm highlighting.
Even though the stub is small we undermine its impact!

Although the stub is small the different entry point also enables
subtle additions which are required, although they are minimal they
are important and if not considered for new x86 features causes
regressions. I don't care what people tell me about "this should have
been caught by code review, and no one CC'd me -- whaa!" -- this is a
silly expectation and I think we should do better.

Case in point:

xen_start_kernel() has seen regressions now on both cr4_init_shadow()
which you forgot to add to Xen Andy -- and later Boris fixed. Another
example: the latest one is a kasan init -- which to this day remains
fucked up -- a Kasan enabled PV guest crashes, and fixing is no where
in sight. I flagged this a while ago, and I think we should put a
proactive measure in place for that.

The linker table stuff I'm doing was not for kicks -- its a means to
an end here, and although I can't yet read subarch at early init, if
we had it, then things link xen_start_kernel() could just be an x86
init stub, it'd be the first stub Xen runs. You can keep whatever
boot_params setup stub it does today, even though I think its cleaner
done in Xen, but at the very least it could at least instead just
*read* the Xen custom generic structure to parse and set boot_params
by using subarch_data pointer.

I also have been reading the history of code changes on the other
entry points in Linux and even though the code is small I see tons of
commits in there for minor but critical fixes all over, likewise
comments and recommendations and visions to unify / share code. I
refuse to accept that we should undermine the issues of a new entry
point or leaving the situation as-is with dual entry points for x86_64
 / xen if we can instead just cleanly unify even asm entry points.

>> The issue that caused headaches in the past isn't that there's code
>> that's executed only on native, it's that there are whole big
>> functions that are executed only on native for no good reason and that
>> aren't clearly marked.

Its more than that, as I noted.

> This won't happen with HVMlite.

And that's fine, its just I think we can avoid yet-another entry --
even if we still are coming into a common C entry later.

>> If we had native_start_kernel and xen_start_kernel, and they both
>> called very quickly in to common_start_kernel, it would be very clear
>> what's going on.
>
> What is now xen_start_kernel() is no longer the entry point for HVMlite. It
> is called as x86_init.oem.arch_setup() (or I may even move it to
> x86_hyper_xen.init_platform() or something like that).

And that's a huge win! Yet I invite us to consider other prospects to
even merge more and simplify more.

 Luis

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.kernel

csiph-web