Please reduce KVM image size

Development
1

Hi Patrick I am enthusiastic about trying the KVM image, but the problem is that its size is massive and exceeds my system hard disk space. Is there any setting you can tweak to reduce it to a couple of gigs and reupload it again?

2

We cannot have it both ways. Or can we?

KVM I/O slowness on RHEL 6 says, without using metadata preallocation, images will be slow.

Since we’re using metadata preallocation, the image seems no longer to be a sparse files (Sparse file - Wikipedia), hence the waste of disk space.

Can get metadata preallocation and sparse files at the same time?

This is implemented here:
https://github.com/Whonix/Whonix/blob/master/build-steps.d/2400_convert-img-to-qcow2

The only command that needs to be improved is this one:

qemu-img \ convert \ -p \ -O qcow2 \ -o cluster_size=2M \ -o preallocation=metadata \ "$WHONIX_BINARY/$VMNAME.img" \ "$WHONIX_BINARY/$VMNAME-$whonix_build_whonix_version_new.qcow2"

3

How bad is the performance without this setting? Maybe if someone less busy than you can take a crack at benchmarks this could be known. Does decreasing the cluster size to a minimum bring it down to a reasonable size? If I had enough disk space I would have experimented with this.

4

Cluster size has a negligible effect on the image. (Dev/KVM - Whonix)

5

So it only affects size but not really performance then?

6

Unfortunately we lost contents of our old forum. If I remember right, without that option the image was so slow, that it was unbearable.

You can always convert these images back and forth.

I think your best bet would be getting enough hdd space and then testing how much performance penalty there is. Or finding out how to combine spare files with metadata allocation.

7

From the dev wiki.page I saw an idea about using QED as a format for KVM, do you mind trying this? The only good thing about QCOW was that it was the standard qemu format for so long.

8

Man page of qemu-img about qed doesn’t make me optimistic.

Image format with support for backing files and compact image files (when your filesystem or transport medium does not support holes). Good performance due to less metadata than the more featureful qcow2 format, especially with cache=writethrough or cache=directsync. Consider using qcow2 which will soon have a similar optimization and is most actively developed.

Looks like it would require performance tests as well. Seems like qed doesn’t support snapshots. And when moving to qed, next thing most likely will happen is someone demanding in the forum to fix the qcow2 issue, so snapshots are working again.

I have another solution in mind…

9

Can you run the following commands please and post the results in here?

[code]du -h Whonix-Gateway-8.tar.gz

du -h --apparent-size Whonix-Gateway-8.tar.gz

du -h Whonix-Gateway-8.qcow2

du -h --apparent-size Whonix-Gateway-8.qcow2[/code]

10

I have an hypothesis. What I did:

  1. run “df -h”
  2. extract Whonix-Gateway-8.tar.gz
  3. run “df -h”
  4. compare output of these two invocations of “df -h”

After extracting Whonix-Gateway-8.tar.gz, I only had ~3 GB less free space than before. This matches output of “du -h Whonix-Gateway-8.qcow2”.

Therefore I think Whonix-Gateway-8.qcow2 already is a sparse file. The question is, if your file system supports sparse files? Mind telling, what file system you are using? Anyway, I can tell you, that with ext4 (what I am using), this is a non-issue. Therefore changing to another file system might solve your problem.

11

my filesystem is EXT4. These are the results for the commands I ran on the images, The files seem to not be sparse unfortunately:

[root@localhost Downloads]# du -h Whonix-Gateway-8.tar.gz
1.2G Whonix-Gateway-8.tar.gz
[root@localhost Downloads]# du -h --apparent-size Whonix-Gateway-8.tar.gz
1.2G Whonix-Gateway-8.tar.gz
[root@localhost Downloads]# du -h Whonix-Gateway-8.qcow2
101G Whonix-Gateway-8.qcow2
[root@localhost Downloads]# du -h --apparent-size Whonix-Gateway-8.qcow2
101G Whonix-Gateway-8.qcow2

12

Here are my results.

du -h Whonix-Gateway-8.tar.gz 1.2G Whonix-Gateway-8.tar.gz du -h --apparent-size Whonix-Gateway-8.tar.gz 1.2G Whonix-Gateway-8.tar.gz

du -h Whonix-Gateway-8.qcow2 2.5G Whonix-Gateway-8.qcow2 du -h --apparent-size Whonix-Gateway-8.qcow2 101G Whonix-Gateway-8.qcow2

Seems like sparse files to me. At least on my system. Using Debian Wheezy / ext4. Also the “df -h” test indicated this.

I guess the issue isn’t with the image itself, but with the file system, host operating system and/or kernel.

Would you be willing to tell us your host operating system? (You don’t have to if that is private.) (Would be useful for other users having this issue / reference.)

Whatever your host operating system is, you could try with Debian Wheezy and see if the issue is solved there. If we’re already providing sparse images, then there is probably nothing we can do from Whonix side. Except telling you, what host operating systems / kernels support sparse files.

For further debugging of this issue, you could download one of our .ova images. And extract it (or alternatively import it into VirtualBox). The .vmdk image you get to see then should be a sparse image.

13

I’d prefer not to reveal my OS for privacy reasons. I trust you personally of course, but I don’t want that known publically. I managed to find some interesting info on KVM snapshots and image formats:

http://wiki.qemu.org/Features/Snapshots

The snapshot image will have to be in a format which support backing files, ie QCOW2 and QED, however the original image can be of any supported format. Ie. it is possible to make a QCOW2 snapshot of a RAW image, or a QED snapshot of a QED image.

What I want to try is to find a way to enable sparse files on my host - if possible. Try conversion to QED and see if this improves.

14

I ran:

qemu-img convert -p -O qed -S 50G ~/Whonix-Gateway-8.qcow2 ~/Whonix-Gateway-8.QED

And I do get a real sparse file in QED format. Its actual size is 2.3 GB while it can potentially grow to 50. I think for one reason or another the image file needs an explicit sparse parameter to make it so.

15

Great finding. Could you try with

as well please? If that works, I would add a

to the conversion command. Would be a simple and quick fix. I hope this works.

16

As well as metadata allocation.

17

I updated https://github.com/Whonix/Whonix/blob/master/build-steps.d/2400_convert-img-to-qcow2 already. Nevermind. (Not that important, forget that I said it maybe.)

Can you help out here again please?

Could you run the following command?

qemu-img \ convert \ -p \ -O qcow2 \ -S "50G" \ -o cluster_size=2M \ -o preallocation=metadata \ original.qcow \ yours.qcow2

That should do what the build script will do in next major Whonix version. (Maybe I do that manually for the next Whonix maintenance update version as well.)

And then check if you end up with a sparse file as well? That would be awesome!

18

After running your commands exactly on the original image, I get a 2.8G image from du -h and a 101G from running du -h --apparent-size. I think that does the trick!

19

Update: I always copy the disk images in case something goes wrong and they become corrupted for any reason. When I tried to copy them, I got an error message that there is no disk space left even though their actual size is no where near the apparent. What is the reason for this? It seems the filesystem is treating these files only based on their apparent sizes.

20

Image size.

You can use “qemu-img convert -S size”, or “qemu-img resize size” after conversion.

After converting a Whonix-Gateway.vmdk image to a raw Whonix-Gateway.img, I had a 100GB image size. Resized it to 4GB.