Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1090248 > unrolled thread

Bug#989462: About bumping Vagrant box disk image to 1TB

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#989462: About bumping Vagrant box disk image to 1TB Emmanuel Kasper <manu@debian.org> - 2022-01-20 19:30 +0100
    Bug#989462: About bumping Vagrant box disk image to 1TB Hans-Christoph Steiner <hans@eds.org> - 2022-01-21 12:20 +0100
      Bug#989462: Info received (About bumping Vagrant box disk image to 1TB) Hans-Christoph Steiner <hans@eds.org> - 2022-02-03 23:10 +0100
        Bug#989462: Info received (About bumping Vagrant box disk image to 1TB) Emmanuel Kasper <manu@debian.org> - 2022-02-05 14:10 +0100

#1090248 — Bug#989462: About bumping Vagrant box disk image to 1TB

I did some testing around
https://salsa.debian.org/cloud-team/debian-vagrant-images/-/tree/1TBv2
(not merged in master yet) and I am still reluctant to merge the branch.
I am OK to bump the default disk size to something like 40GB but not to 1TB.

The problem with the disk size of 1TB is as such: when you do a lot of
write / erase cycles, the deletion of blocks is not propagated to the
qcow2 backing disk image, so even though the OS in the VM reports only
2GB of block usage, the disk image could grow to 1TB, without the user
knowing it.
I could reproduce this behavior running `fio` in the guest in a loop.
I find this behavior dangerous.

At that point I see three possibilities:
- you add to your pull request a change of the virtualized disk
controller from virtio-blk to virtio-scsi and to the default libvirt
vagrantfile the "unmap" option so that deletion of blocks in the guest
are propagated om host storage
- you're fine with a disk image size of 40, or let's say 80GB
- you use a shared folder for the builds. I just noticed vagrant-libvirt
has also support for virtio-fs which according to its author has native
host performance. If they are security concerns, let's discuss that in
details and involve upstream if needed. virtio-fs is mature enough that
it's use in production for Kata Containers in Kubernetes and OpenShift
Sandboxed containers in the Red Hat Kubernetes offering.

What do you think ?

[toc] | [next] | [standalone]

#1090319

Emmanuel Kasper:
> I did some testing around
> https://salsa.debian.org/cloud-team/debian-vagrant-images/-/tree/1TBv2
> (not merged in master yet) and I am still reluctant to merge the branch.
> I am OK to bump the default disk size to something like 40GB but not to 1TB.
> 
> The problem with the disk size of 1TB is as such: when you do a lot of
> write / erase cycles, the deletion of blocks is not propagated to the
> qcow2 backing disk image, so even though the OS in the VM reports only
> 2GB of block usage, the disk image could grow to 1TB, without the user
> knowing it.

Yeah, I also thought about that.  Would it be possible to ship the images with a 
disk/partition size of 1TB but keep a filesystem size of 20GB?  It is easy to 
expand the filesystem as needed.  The hard part is expanding the disk/partition 
size.

> I could reproduce this behavior running `fio` in the guest in a loop.
> I find this behavior dangerous.
> 
> At that point I see three possibilities:
> - you add to your pull request a change of the virtualized disk
> controller from virtio-blk to virtio-scsi and to the default libvirt
> vagrantfile the "unmap" option so that deletion of blocks in the guest
> are propagated om host storage

This sounds like the ideal solution.  I have no idea how much work it would be. 
  Do you?

> - you're fine with a disk image size of 40, or let's say 80GB

Chromium builds can take more than 100GB, so either of those would mean we still 
need to make our own basebox.

> - you use a shared folder for the builds. I just noticed vagrant-libvirt
> has also support for virtio-fs which according to its author has native
> host performance. If they are security concerns, let's discuss that in
> details and involve upstream if needed. virtio-fs is mature enough that
> it's use in production for Kata Containers in Kubernetes and OpenShift
> Sandboxed containers in the Red Hat Kubernetes offering.

We like the security isolation of throwing away all things that the build 
process has written to, so this option is less appealing, though perhaps 
workable.  It there was a host-controlled method of resetting the virtio-fs to a 
previous snapshot, then it could work.

.hc

[toc] | [prev] | [next] | [standalone]

#1091978 — Bug#989462: Info received (About bumping Vagrant box disk image to 1TB)

 > - you add to your pull request a change of the virtualized disk
 > controller from virtio-blk to virtio-scsi and to the default libvirt
 > vagrantfile the "unmap" option so that deletion of blocks in the guest
 > are propagated om host storage

I looked into this a bit more.  These feature weren't added to vagrant-libvirt 
until 0.4.0, so no stable Debian package of vagrant-libvirt could support this 
yet.  Debian/testing has 0.7.0, and a backport to stable would be possible.  I 
guess a lot of people just use `vagrant plugin install ...`. It might be 
possible to make it work with older vagrant-libvirt versions by using a hack 
like this:

https://github.com/vagrant-libvirt/vagrant-libvirt/pull/692#issuecomment-922329049


There is a little more info here:
https://github.com/vagrant-libvirt/vagrant-libvirt/issues/999#issuecomment-487728207


```
ENV['VAGRANT_EXPERIMENTAL'] = 'typed_triggers'

require 'open3'

Vagrant.configure('2') do |config|
   ...
   config.vm.provider :libvirt do |lv, config|
     lv.storage :file, :size => '3G', :device => 'sda', :bus => 'scsi', :discard 
=> 'unmap', :cache => 'unsafe'
     config.trigger.before 
:'VagrantPlugins::ProviderLibvirt::Action::StartDomain', type: :action do |trigger|
       trigger.ruby do |env, machine|
         stdout, stderr, status = Open3.capture3(
           'virt-xml', machine.id,
           '--edit', 'type=scsi',
           '--controller', 'model=virtio-scsi')
         if status.exitstatus != 0
           raise "failed to run virt-xml to modify the controller model. 
status=#{status.exitstatus} stdout=#{stdout} stderr=#{stderr}"
         end
       end
     end
     ...
   end
end
```

[toc] | [prev] | [next] | [standalone]

#1092132 — Bug#989462: Info received (About bumping Vagrant box disk image to 1TB)

On 2/3/22 22:58, Hans-Christoph Steiner wrote:
>  > - you add to your pull request a change of the virtualized disk
>  > controller from virtio-blk to virtio-scsi and to the default libvirt
>  > vagrantfile the "unmap" option so that deletion of blocks in the guest
>  > are propagated om host storage
> 
> I looked into this a bit more.  These feature weren't added to 
> vagrant-libvirt until 0.4.0, so no stable Debian package of 
> vagrant-libvirt could support this yet.  Debian/testing has 0.7.0, and a 
> backport to stable would be possible.  I guess a lot of people just use 
> `vagrant plugin install ...`. It might be possible to make it work with 
> older vagrant-libvirt versions by using a hack like this:
> 
> https://github.com/vagrant-libvirt/vagrant-libvirt/pull/692#issuecomment-922329049 
> 
> 
> 
> There is a little more info here:
> https://github.com/vagrant-libvirt/vagrant-libvirt/issues/999#issuecomment-487728207 
> 
> 
> 
> ```
> ENV['VAGRANT_EXPERIMENTAL'] = 'typed_triggers'
> 
> require 'open3'
> 
> Vagrant.configure('2') do |config|
>    ...
>    config.vm.provider :libvirt do |lv, config|
>      lv.storage :file, :size => '3G', :device => 'sda', :bus => 'scsi', 
> :discard => 'unmap', :cache => 'unsafe'
>      config.trigger.before 
> :'VagrantPlugins::ProviderLibvirt::Action::StartDomain', type: :action 
> do |trigger|
>        trigger.ruby do |env, machine|
>          stdout, stderr, status = Open3.capture3(
>            'virt-xml', machine.id,
>            '--edit', 'type=scsi',
>            '--controller', 'model=virtio-scsi')
>          if status.exitstatus != 0
>            raise "failed to run virt-xml to modify the controller model. 
> status=#{status.exitstatus} stdout=#{stdout} stderr=#{stderr}"
>          end
>        end
>      end
>      ...
>    end
> end
> ```

Hi Hans-Christoph

I had a look at this and it looks like although we can now set via the 
`disk_driver` options to use `discard` when a SCSI controller is 
choosed, it is still not possible to set the SCSI controller `model` to 
virtio-scsci via a vagrant-libvirt option. Without a virtio-scsi SCSI 
controller I/O will be 10 times slower.

We could actually force the controller model and the discard option 
manually in the Vagrantfile shipped by the box using the hack/workaround 
you pasted above. But the :action trigger used there is an experimental 
feature, requiring VAGRANT_EXPERIMENTAL="typed_triggers" to be set.
So this is too much of an over stretch here at the momment.

When time allows, I will now try to use a box with virtio-fs and see how 
it works against NFS as this seem to be way the way forward (maybe use 
virtio-fs instead of NFS too by default for libvirt too)

Emmanuel




-- 
You know an upstream is nice when they even accept m68k patches.
   - John Paul Adrian Glaubitz, Debian OpenJDK maintainer

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web