Let’s stay still away from virt-sparsify. At the moment it may be ok, but we don’t know how the implementation of virt-sparsify changes over the years.
This is because we don’t have safeguards in place for sanity testing. That is automated checking if modifications to the image are made, Verifiable Builds - Whonix, deterministic builds.
Just a little update: the new functions/scripts worked fine with the Whonix-Gateway, I am trying with the Workstation (building from scratch). It takes time. I’ll keep you updated.
OK, seems to work now, except that my build attempts ALWAYS fail during the unmount-raw function of script 1700 (I have to reboot the machine and launch the remaining scripts manually)… But this is most probably unrelated to this topic.
Should I do a few more tries (maybe on a ext4 partition, not NTFS), or I can already commit the new scripts?
I did a few more tries, both on NTFS and ext4 partitions.
I confirm zeroing works now! .ova files are shrunk as expected. I will update the code on git later today. Please test and review!
As usual, the 1700 script fails during unmounting (device busy)… Since it happens before the new 2350 script, and nothing else was changed, I am assuming it is not related to the zeroing function. Last time it happened I did a check with lsof and it showed that a single process, gpg-agent had actually some files still open in the Workstation image (see screenshot below). Has anyone already seen that?
I put everything on GitHub. Please check and review the code.
mount-raw-no-chroot shouldn’t have
+main (){
+ root_check
+ if [ "$ANON_BUILD_INSTALL_TO_ROOT" = "1" ]; then
+ true "${green}INFO: Skipping script, because
ANON_BUILD_INSTALL_TO_ROOT=1: $BASH_SOURCE${reset}"
+ exit 0
+ else
+ mount_raw_nochroot
+ fi
+}
+
+main "$@"
The main point of sourceing a file in most cases is to import a bash
function which is then used from multiple bash scripts.
(Except when using a logic "if sourced, don’t execute (up to script that
sources it. if executed (not sourced), execute it. (We have examples of
this in Whonix source code.))
So just source mount_raw_nochroot and then use it from
2350_zerofree-raw.
The duplicate source code issue is not solved yet. mount-raw /
unmount-raw still have the full copy of the source code.
Maybe the easiest would be if the mount-raw / unmount-raw would also
source mount-raw-no-chroot / unmount-raw-no-chroot and then run their
functions.
Thanks for your feedback. Unfortunately I have trouble understanding what you mean here. I am afraid this is beyond my technical skills yet. I don’t know how I can help here.
onion_knight:
As usual, the 1700 script fails during unmounting (device busy)… Since it happens before the new 2350 script, and nothing else was changed, I am assuming it is not related to the zeroing function. Last time it happened I did a check with
lsofand it showed that a single process,gpg-agenthad actually some files still open in the Workstation image (see screenshot below). Has anyone already seen that?
Attempting to fix this.
https://github.com/Whonix/Whonix/commit/51acda6fcd40ff06cc9a0c66231c2cb2c62c198e
Included in 14.0.0.7.0-developers-only but haven’t finished a build
myself yet.
Above works.
It works around an interesting maybe heisenbug.
+ /home/user/Whonix/help-steps/umount_kill.sh /home/user/whonix_binary/Whonix-Gateway_image/
-> Attempting to kill any processes still running in '/home/user/whonix_binary/Whonix-Gateway_image/' before un-mounting
Okay, the following pids are still running inside '/home/user/whonix_binary/Whonix-Gateway_image/', which will now be killed.
PID TTY STAT TIME COMMAND
8486 ? S 0:00 /bin/bash /etc/X11/Xsession.d/20power_savings_disable_in_vms
8488 ? S 0:00 sleep 60
un-mounting /home/user/whonix_binary/Whonix-Gateway_image/dev
And also attempted to really fix the bug.
zerofree is now implemented.
Please test building:
14.0.0.7.3-developers-only
Maybe this bug below is related?
In Qubes R4, with sys-whonix set as updateVM, when you run sudo qubes-dom-update in dom0, you see:
/usr/lib/qubes/qubes-rpc-multiplexer: 14: /etc/profile.d/20_power_savings_disable_in_vms.sh: shopt: not found
(repeats twice)
dom0 update procedure still works though.
Has been deployed.
Fantastic! @Patrick thanks a lot for making this real! Sorry that I could not take it through the end. I’ll make sure to have a look on the code and see how it looks now, hopefully with time my technical skills will progress and I’ll be able to contribute more thoroughly 
Please have a look.
Qubes uses fstrim.
qubes-linux-template-builder/cleanup_image at master · QubesOS/qubes-linux-template-builder · GitHub
If we run that before and/or after zerofree (or vice versa) could we further reduce Whonix image sizes?
Hi Patrick. I didn’t know this method. I can give it a try.
Cheers
Hello
Back to the forum after a long summer pause…
I did a try with a Whonix-Gateway (current, 14) build and did not notice any difference.
I proceeded as follows:
home/user/Whonix/help-steps/mount-raw
sudo /sbin/fstrim -v /home/user/Whonix-Gateway_image
The final .ova size is 937 MB, exactly the same size as without the fstrim command.
By the way, I notice that using the current github version results in a 14.0.0.8.1-19 version, whereas the official download version is currently 14.0.0.7.4, and only weighs 850 MB. Is there any substantial difference between the two versions, and why this size increase?
One more little thing: the values for .ova files must be updated here according to their new value: https://www.whonix.org/wiki/Template:DownloadTable#