Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1630360 > unrolled thread

Re: about CPU QoS in KVM

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: about CPU QoS in KVM Juri Lelli <juri.lelli@arm.com> - 2017-04-25 12:20 +0200
    Re: about CPU QoS in KVM Luca Abeni <luca.abeni@santannapisa.it> - 2017-04-26 09:50 +0200

#1630360 — Re: about CPU QoS in KVM

Hi,

just noticed this and thought I might just jump in. :)

On 20/04/17 16:00, Paolo Bonzini wrote:
> Adding Rik.
> 
> Paolo
> 
> On 20/04/2017 15:32, Gonglei (Arei) wrote:
> > Hi all,
> > 
> > Currently, KVM do the CPU resource reservation by the cgroup mechanism
> > which can't do entire accurate separation because the capacity of the Linux
> > scheduler. Take the public cloud as an example, some customers rent one vm
> > with 8 CPUs paid by enough money, they want to get enough response
> > speed on CPU scheduling. So we (the cloud platform providers ) reserve 1GHz
> > CPU resources by cgroup for those VM's vcpu/pcpu. 
> > 
> > But the actual effects can't meet those requirements because the cgroup is
> > limiting share usage of other processes in order to attach the reservation proportion,
> > but the scheduler can't assure that. This mechanism is different with Xen,
> > We can directly change the CPU weight on Xen hypervisor so that we can
> > get entire accurate control on CPU resources based on accurate capacity (upper limit),
> > share (weight) and reservation.
> > 
> > So my question is do we have a good method to do CPU reservation in KVM?
> > 
> > Thanks,
> > -Gonglei

Not entirely sure what your particular requirements are Gonglei, but you
might be interested to know that there has been research work [1,2,3,
just to name a few] that used a mainline real-time scheduling policy
(SCHED_DEADLINE) to provide Qos support to virtual machines (KVM).

I won't enter in too much detail, but the basic idea is to use
reservation based scheduling mechanisms to enforce temporal isolation and
guaranteed CPU bandwidth to VM's vcpu(s).

I'm Cc-ing Tommaso, Luca and Carlo, whom can provide more information as
needed.

Best,

- Juri

[1] - http://retis.sssup.it/~nino/publication/rtlws14sdnnfs.pdf
[2] - http://retis.sssup.it/~tommaso/publications/VHPC-2010.pdf
[3] - http://retis.sssup.it/~tommaso/publications/RTSOAA-2009-RTV.pdf

Skimming through Tommaso and Luca publications might be interesting as
well. I'm pretty sure I missed the most importants papers. :)
http://retis.sssup.it/~tommaso/eng/publications.html
https://scholar.google.co.uk/citations?user=C3a6glEAAAAJ&hl=en

[toc] | [next] | [standalone]

#1631215

Hi all,

On Tue, 25 Apr 2017 11:13:03 +0100
Juri Lelli <juri.lelli@arm.com> wrote:
[...]
> > > Currently, KVM do the CPU resource reservation by the cgroup
> > > mechanism which can't do entire accurate separation because the
> > > capacity of the Linux scheduler. Take the public cloud as an
> > > example, some customers rent one vm with 8 CPUs paid by enough
> > > money, they want to get enough response speed on CPU scheduling.
> > > So we (the cloud platform providers ) reserve 1GHz CPU resources
> > > by cgroup for those VM's vcpu/pcpu. 
> > > 
> > > But the actual effects can't meet those requirements because the
> > > cgroup is limiting share usage of other processes in order to
> > > attach the reservation proportion, but the scheduler can't assure
> > > that. This mechanism is different with Xen, We can directly
> > > change the CPU weight on Xen hypervisor so that we can get entire
> > > accurate control on CPU resources based on accurate capacity
> > > (upper limit), share (weight) and reservation.
> > > 
> > > So my question is do we have a good method to do CPU reservation
> > > in KVM?
> > > 
> > > Thanks,
> > > -Gonglei  
> 
> Not entirely sure what your particular requirements are Gonglei, but
> you might be interested to know that there has been research work
> [1,2,3, just to name a few] that used a mainline real-time scheduling
> policy (SCHED_DEADLINE) to provide Qos support to virtual machines
> (KVM).
> 
> I won't enter in too much detail, but the basic idea is to use
> reservation based scheduling mechanisms to enforce temporal isolation
> and guaranteed CPU bandwidth to VM's vcpu(s).

Since I've been CC-ed, I add some details to what Juri wrote:
if the guest workload is mostly CPU intensive, just scheduling the vCPU
threads with SCHED_DEADLINE allows deterministic CPU allocation.

I recently showed some experiments about this to some students: if some
periodic real-time tasks are executed in the guest and KVM's vCPU
threads are scheduled with SCHED_DEADLINE, then the number of missed
deadlines measured in the guest matches with the number of missed
deadlines expected from theoretical analysis (using real-time
hierarchical scheduling analysis).
[I can provide more details if needed: here is a quick and incomplete
summary I wrote for the students:
http://retis.sssup.it/~luca/CBSD/h-sched_experiments.txt
up to item 4 it is just theoretical schedulability analysis; the
interesting parts start from item 5]

I think this shows that (for CPU-intensive workloads) it is possible to
deterministically control the QoS of a KVM-based VM by using
SCHED_DEADLINE.


If your guest workload includes more I/O, then you have to schedule
more threads with SCHED_DEADLINE (for example, I have some experiments
with a lot of network traffic in the guest; I used vhost-net and I've
been able to control the network throughput by scheduling the vhost-net
kernel thread with SCHED_DEADLINE and changing the runtime and period
associated to thread).



				Luca

> 
> I'm Cc-ing Tommaso, Luca and Carlo, whom can provide more information
> as needed.
> 
> Best,
> 
> - Juri
> 
> [1] - http://retis.sssup.it/~nino/publication/rtlws14sdnnfs.pdf
> [2] - http://retis.sssup.it/~tommaso/publications/VHPC-2010.pdf
> [3] - http://retis.sssup.it/~tommaso/publications/RTSOAA-2009-RTV.pdf
> 
> Skimming through Tommaso and Luca publications might be interesting as
> well. I'm pretty sure I missed the most importants papers. :)
> http://retis.sssup.it/~tommaso/eng/publications.html
> https://scholar.google.co.uk/citations?user=C3a6glEAAAAJ&hl=en

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web