Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1537987

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

Show all headers | View raw

Hello,

On Wed, Dec 07, 2016 at 08:57:01AM -0800, Dave Hansen wrote:
> It is more space-efficient.  We're fitting the order into 6 bits, which
> would allows the full 2^64 address space to be represented in one entry,

Very large order is the same as very large len, 6 bits of order or 8
bytes of len won't really move the needle here, simpler code is
preferable.

The main benefit of "len" is that it can be more granular, plus it's
simpler than the bitmap too. Eventually all this stuff has to end up
into a madvisev (not yet upstream but somebody posted it for jemalloc
and should get merged eventually).

So the bitmap shall be demuxed to a addr,len array anyway, the bitmap
won't ever be sent to the madvise syscall, which makes the
intermediate representation with the bitmap a complication with
basically no benefits compared to a (N, [addr1,len1], .., [addrN,
lenN]) representation.

If you prefer 1 byte of order (not just 6 bits) instead 8bytes of len
that's possible too, I wouldn't be against that, the conversion before
calling madvise would be pretty efficient too.

> and leaves room for the bitmap size to be encoded as well, if we decide
> we need a bitmap in the future.

How would a bitmap ever be useful with very large page-order?

> If that was purely a length, we'd be limited to 64*4k pages per entry,
> which isn't even a full large page.

I don't follow here.

What we suggest is to send the data down represented as (N,
[addr1,len1], ..., [addrN, lenN]) which allows infinite ranges each
one of maximum length 2^64, so 2^64 multiplied infinite times if you
wish. Simplifying the code and not having any bitmap at all and no :6
:6 bits either.

The high order to low order loop of allocations is the interesting part
here, not the bitmap, and the fact of doing a single vmexit to send
the large ranges.

Once we pull out the largest order regions, we just add them to the
array as [addr,1UL<<order], when the array reaches a maximum N number
of entries or we fail a order 0 allocation, we flush all those entries
down to qemu. Qemu then builds the iov for madvisev and it's pretty
much a 1:1 conversion, not a decoding operation converting the bitmap
in the (N, [addr1,len1], ..., [addrN, lenN]) for madvisev (or a flood
of madvise MADV_DONTNEED with current kernels).

Considering the loop that allocates starting from MAX_ORDER..1, the
chance the bitmap is actually getting filled with more than one bit at
page_shift of PAGE_SHIFT should be very low after some uptime.

By the very nature of this loop, if we already exacerbates all high
order buddies, the page-order 0 pages obtained are going to be fairly
fragmented reducing the usefulness of the bitmap and potentially only
wasting CPU/memory.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration Liang Li <liang.z.li@intel.com> - 2016-11-30 10:00 +0100
  [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info Liang Li <liang.z.li@intel.com> - 2016-11-30 10:00 +0100
    Re: [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page  info Dave Hansen <dave.hansen@intel.com> - 2016-11-30 20:20 +0100
      RE: [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused  page info "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-04 14:20 +0100
        Re: [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page  info Dave Hansen <dave.hansen@intel.com> - 2016-12-05 18:30 +0100
          RE: [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused  page info "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-06 05:50 +0100
  [PATCH kernel v5 2/5] virtio-balloon: define new feature bit and head struct Liang Li <liang.z.li@intel.com> - 2016-11-30 10:00 +0100
  [PATCH kernel v5 4/5] virtio-balloon: define flags and head for host request vq Liang Li <liang.z.li@intel.com> - 2016-11-30 10:00 +0100
  [PATCH kernel v5 1/5] virtio-balloon: rework deflate to add page to a list Liang Li <liang.z.li@intel.com> - 2016-11-30 10:00 +0100
  Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration David Hildenbrand <david@redhat.com> - 2016-12-06 09:50 +0100
    RE: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-07 14:40 +0100
      Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-07 16:40 +0100
        RE: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-09 04:10 +0100
      Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-07 16:50 +0100
        Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration David Hildenbrand <david@redhat.com> - 2016-12-07 17:30 +0100
          Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-07 18:00 +0100
            Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration Andrea Arcangeli <aarcange@redhat.com> - 2016-12-07 19:40 +0100
              Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-07 20:00 +0100
                Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration Andrea Arcangeli <aarcange@redhat.com> - 2016-12-07 20:00 +0100
              Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-07 21:00 +0100
                Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration Andrea Arcangeli <aarcange@redhat.com> - 2016-12-07 21:30 +0100
                RE: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-09 05:50 +0100
                Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration Dave Hansen <dave.hansen@intel.com> - 2016-12-09 06:00 +0100
                RE: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration "Li, Liang Z" <liang.z.li@intel.com> - 2016-12-09 06:40 +0100
                Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for  fast (de)inflating & fast live migration Andrea Arcangeli <aarcange@redhat.com> - 2016-12-09 17:50 +0100
      Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast  (de)inflating & fast live migration David Hildenbrand <david@redhat.com> - 2016-12-07 16:50 +0100

csiph-web