How to decrease 100GB Whonix image size

KVM
1

I’m following the Wiki to setup Whonix for KVM.
After decompression (downloaded the latest LXqt stable) I find myself with two 100GB Whonix image files. Is it possible to shrink them tho they real size?

2

It’s normal, is the size it come out of the box.
but the reality is that any of the machines exceed the 20GB initially. I suppose they do this to doesn’t have space problems but 50GB would be enough in my opinion.

The subject here is that though the system recognize the files are 100GB size, they aren’t actually this size. It’s known as sparse files, ls -lh/du --apparent-size -h * read the metadata of the files and show the allocated blocks, but du -h/ls -lsh show the real size/blocks of the file skipping the unallocated regions filled with zeros.

Sparse files use holes of zeros. When you read from these holes, the filesystem transparently returns zero bytes without ever having stored those zeros. This is part of how the qcow2 works.

I think there are two options in this case:

  1. Change the size of the disk into the guest system and shrink the disk file in host.
  2. Reclaim space from the sparse file, what should liberate the rest of space and be able to use it for other apps.

I think the option 2 is the best, because from what I’ve read, qcow2 files grow whenever the internal disk grow, the disk file should grow with it.

sudo -i && TMPDIR=/dir-where-files-are virt-sparsify --compress your_disk.qcow2 compact_disk.qcow2

The command above have worked for me, for example converting a kicksecure disk file of apparently 108G but with a real size of 11G, to 3,2GB in both cases.
I’ve installed some packets without issues and the disk file’s size have increased correctly to 5GB, and the disk of the guest continue being 100GB.

1 Like
3

Thanks @spartack, there is an option on the Wiki to decompress the package using tar -xvf it the fs didn’t support sparse files (I instead used tar -xSvf).

I tried but it seems it just brought me in sudo -i mode. The file’s size didn’t change. I tried one more time without sudo -i but nothing.

[user /mnt/shared/w]% sudo -i && TMPDIR=/mnt/shared/w virt-sparsify --compress Whonix-Gateway-LXQt-18.1.4.2.Intel_AMD64.qcow2  Whonix-Gateway-LXQt.qcow2
[root ~]# TMPDIR=/mnt/shared/w virt-sparsify --compress Whonix-Gateway-LXQt-18.1.4.2.Intel_AMD64.qcow2  Whonix-Gateway-LXQt.qcow2
zsh: command not found: virt-sparsify
zsh: exit 127   TMPDIR=/mnt/shared/w virt-sparsify --compress  Whonix-Gateway-LXQt.qcow2

Would tar -xvf have decompress the qcow2 showing their actual size?

If so I can decompress them again with the right command.

4

To be clear, does the filesystem in use support sparse files? If it does, then the 100 GB “size” can probably be safely ignored.

There isn’t a way to do that with tar. The files themselves are 100 GB (at the “file structure” level), but some filesystems are smart enough to realize that they consist of mostly 0 bytes and just stores some info about where the 0s are rather than storing them directly. If you want to shrink the files themselves to be smaller than 100 GB, you need to use qemu-img to rewrite them so that they are small and don’t contain massive zones of 0s. See:

Keep in mind that because the file will end up truly smaller, the virtualizer will have to do the work of growing the file and reorganizing data within it as you write data to it, which could result in lower I/O performance. That’s why the files are large and sparse by default.

1 Like
5

you have to install the package

sudo apt update && sudo apt install libguestfs-tools

But like @arraybolt3 if the system support sparse files, like debian and kicksecure do, there is no issues because the system only use the real block used by the files and leave out the zero holes.

1 Like
6

Thanks @arraybolt3,

This helped egregiously!!! I couldn’t make zerofree work on the VM partition as instructed in the Wiki but I skipped it and shrunk the files anyway, they boot perfectly.

Thanks I’ll try it out.

1 Like
7

I think that back when I initially tried whonix, before using Virtualbox-Whonix, I was trying KVM-Whonix. I ran into the same issue, and I think I remember that the problem was that ext4 did not support sparse files or something. Maybe if you reinstall your distribution and choose a different file system, it would work.

I may very well end up doing this and switch to using KVM/Whonix, due to the shared clipboard issue which Virtualbox-Whonix is currently experiencing.

8

Hmm, every ext4 system I’ve used supported sparse files, and it seems that it’s had that feature since 2007 at least (https://www.kernel.org/doc/ols/2007/ols2007v2-pages-21-34.pdf). Maybe it’s an optional feature? Not sure.

1 Like