[HOME] [DOWNLOAD] [DOCS] [NEWS] [SUPPORT] [TIPS] [ISSUES] [DONATE]

scripts to shrink vbox and kvm images

i have played with a couple scripts in order to keep the virtual hd images in both virtualbox and kvm from ballooning out of control. the virtualbox script was included in the last short-lived version of the guide i work on. the kvm script is still experimental. i’m posting both here if anyone would like to review or tweak. basically, they search a directory for hd images that have been modified in the past hour, and apply routines to reduce the size. both rely on “zerofree” in some respects. however, the kvm script also has an option to use the qemu compression option. both scripts have resulted in the freeing of gigabytes of hard drive space on the host when used. feedback welcomed.

vbox script

this script requires a user to boot into recovery mode, mount the hd as readonly, and run zerofree on it first.

from a terminal, type
echo "alias vbcompact='find ~/VirtualBox\ VMs/ \ -type f -mtime -1 -size +10M -name *.vdi \ -exec vboxmanage modifymedium --compact {} t {} \;'" >> ~/.bashrc

typing “vbcompact” later will search through the default virtualbox directory and attempt to compact any “vdi” file modified in the past hour.

kvm script

this script will also see advantages from booting into recovery mode, mounting the hds readonly, and running zerofree. however, qemu compression may negate that, but may take longer. i ran into a couple issues with snapshots not working here and there after this process, which i’m exploring.

this one is a little more involved. it’s to be appended to a user’s .bashrc file.

function vmshrink() { list=$(sudo find /var/lib/libvirt/images/ -type f -mtime -1 -size +10M -name *.qcow2)
	for f in $list
	do
	# ask if user wants to shrink image
	read -e -p "Shrink $f? [y/N] " choice
	if [[ "$choice" == [Yy]* ]]
	   then
	    #"yes" choice. show file size and move original to a back up file.
	    ls -lh $f
	    sudo mv $f $f.backup
	    # ask if user wants to use qemu compression
	    read -e -p "Do you want to enable disk compression on $f? [y/N] " choice2
            if [[ "$choice2" == [Yy]* ]]
              then 
                #"yes" choice. apply qemu compression.
		sudo qemu-img convert -O qcow2 -p -c $f.backup $f
	      else
	        #"no" choice. convert to remove space changed by zerofree only.
		sudo qemu-img convert -O qcow2 -p $f.backup $f
	    fi
	    ls -lh $f
	    # ask if user wants to delete the back up copy.
	    read -e -p "Please run the vm to confirm it still works. If it works, you can remove the back up. Remove $f.backup? [y/N] " choice3
		if [[ "$choice3" == [Yy]* ]]
	          then
		    #"yes" choice. delete backup
		    sudo rm $f.backup
		elif [[ "$choice3" != [Yy]* ]]
		  then
		    #"no" choice. ask if user wants to restore backup.
		    read -e -p "Restore backup? [y/N] " choice4
		    if [[ "$choice4" == [Yy]* ]]
                       then
			  #restore backup
                          sudo mv $f.backup $f
		fi
	fi
fi
done }

the goal of both scripts was to make the process of shrinking hd images after a dist-upgrade a little simpler, rather than having to type the individual commands for each individual file.

1 Like

Might be easier for some users to boot into live mode rather than
recovery mode:

But without setting the disk to read-only, i.e. without this:

Since disk is mounted as read-only it should be possible to run zerofree
on it.

We have documentation on how to in increase virtual disk size.

But we don’t have it for shrinking virtual disks?

1 Like

i tried that. unfortunately, i have not been able to get it to work. however, i did not pursue it too hard. when you boot into the whonix desktop outside of recovery mode as you describe, zerofree will not recognize the file system of /dev/sda1. if you try “sudo mount -o remount,ro /dev/sda1 /” you will get a “/ is busy” error.

We have documentation on how to in increase virtual disk size.

https://www.whonix.org/wiki/Grow_Virtual_Harddisk

But we don’t have it for shrinking virtual disks?

just so we are clear, this is not shrinking the max capacity of the disk. the actual allotted size for the disk is left untouched. what this does is remove otherwise garbage data that is still present in a sparse image. for example, after running “sudo apt-get-update-plus dist-upgrade && sudo apt-get clean,” sometimes the data stored on the virtual disk will grow by 1-2 gigabytes. a fix for this implements the zerofree program.

disable readonly flag in vbox or qemu-kvm.
boot in recovery mode.
type “mount -o remount,ro /dev/sda1 /”
type “zerofree -v /dev/sda1”
shutdown system.

then, from the host, depending on the hypervisor, use the command in the scripts above. the processed disk images then tend to shrink down closer to the size they were before any additional software was installed, thus saving gigabytes of space.

so, this does not actually shrink the capacity of the virtual drives. it just removes the data that causes them to balloon. i’ve found this method necessary on usb sticks, where disk space is much more limited, and on machines where i may have greater than 10 virtual machines installed at once.

1 Like

At least for KVM images there already is an official tool: http://libguestfs.org/virt-sparsify.1.html

2 Likes

will play with that. does it allow for test and restore of backup?

1 Like

It automatically creates a new disk and keeps the old one. You can skip that with the --in-place option

2 Likes

Might even work for VirtualBox! :slight_smile:

Supported and known-working output formats are: raw , qcow2 , vdi .

1 Like

does virt-sparsify take care of zerofree runs? the libguestfs-tools package installs a ton of unneeded software on the host, including exim4 and some other email related tools.

Does sudo apt --no-install-recommends install libguestfs-tools help
and install fewer packages?

1 Like

ah. i’d over looked that since they weren’t appearing as '“suggested” packages. yes, the --no-install-recommends flag makes this a lot more manageable and doesn’t install exim4. thanks.

1 Like

so, i’m running into a number of issues at this point and i am not sure where i made a mistake. in some instances, after a zerofree run on the image and then a convert with qemu-img, the virtual disk size almost doubles. the same is happening with virt-sparsify. running the “compression” flag with either reduces the size. but, i am not sure why that is needed. this is with kvm based images.

1 Like

When run manually or by script? If by script, echo actually executed commands. Otherwise share which commands were run either for analysis / reproduction?

it’s with both script or manual. it’s occurring with both “qemu-img convert” and “virt-sparsify.” my guess, at the moment, is because these images may have had the “qemu-img convert --compress” option used at one point. i am going to experiment with it more. i have not yet been able to figure this one out.

this was a false alarm. it looks like i had enabled “compression” on the test images at one point. thus, it appears that the increased file size after a “qemu-img convert” or “virt-sparisfy” copy method created a new image without compression enabled and, therefore, was larger.

on a side note, aside from the potential performance hits, is anyone aware of any security issues when using a compressed image with kvm?

[Imprint] [Privacy Policy] [Cookie Policy] [Terms of Use] [E-Sign Consent] [DMCA] [Investors] [Priority Support] [Professional Support]