Running Whonix on Proxmox

I’ve been running Whonix on Proxmox for a while now (about 18 months), for reasons I won’t go into because my use-cases tend to be fairly niche and a bit oddball.

I recently raised a thread about issues with clipboard sharing failing to work since I upgraded my “gold” images to Whonix 18 (TL;DR: spice-vdagent doesn’t currently support Wayland, check the other thread for details), and it was suggested in there that some coverage of the hows of running Whonix on Proxmox might be appreciated by the community.

My solution originally started out as tech-demo built in nested-virt under VirtualBox in a VMWare ESXi VM, unfortunately whilst functional, it wasn’t particularly performant and didn’t really fulfil our needs as a result. Fast forward a bit and Broadcom did “the thing” which lead to us migrating all of our Hypervisors over to Proxmox, and given that Proxmox uses KVM as it’s VM environment I thought it was worth another try, this is the result of that.

First off, it pretty much "just works"™ and I’m sufficiently comfortable with the way it works that I’m happy to employ it in most of our use-cases and with our threat model but YMMV (I’ve also done some other things outside of the norm which are specific to our environment and as such I won’t be going into them in detail).

First off, you’ll need to create a new “Linux Bridge” on your Proxmox host, I arbitrarily used vmbr9, and set the comment to “Whonix”, no further configuration is needed (or desirable), this will be your “Internal” network (referred to as “WhonixBridge” below).

Next you’ll need to grab the libvirt images for the current version of Whonix, I grab both the CLI and GUI versions because I prefer to run my Whonix-Gateway as a CLI only box (I don’t see the point of the extra resource consumption of a DE on a box I basically only interact with to install updates).

Extract the images somewhere sensible on your Proxmox server (I use /root/Whonix/CLI and /root/Whonix/LXQt);

cd /root/Whonix/CLI
tar -xvf /path/to/<CLI Image>.libvirt.xz
cd /root/Whonix/LXQt
tar -xvf /path/to/<GUI Image>.libvirt.xz

Next you’ll need to create the Whonix-Gateway VM (all the set commands CAN be combined into a single command but this is more readable. If you don’t care about SecureBoot you can omit a lot of these commands, see notes.);

cd /root/Whonix/CLI
qm create <VMID> --memory 2048 --name Whonix-Gateway --net0 virtio,bridge=<OutboundBridge> --net1 virtio,bridge=<WhonixBridge> # OutboundBridge is whatever bridge you want the Gateway to use to connect to the internet.
qm importdisk <VMID> <Whonix-Gateway...qcow2> <ProxmoxStorage>
qm set <VMID> --virtio0 <ProxmoxStorage>:<imported_disk>
qm set <VMID> --boot c --bootdisk virtio0
# The stuff below is mostly only needed if you want UEFI/SecureBoot
qm set <VMID> --machine q35		# My preference for standardisation, omit for i440bx default
qm set <VMID> --cores 4			# Realistically it's probably fine with the default of 1 for most purposes
qm set <VMID> --agent 1			# Potential information leakage vector but I value clean shutdowns and restarts
qm set <VMID> --bios ovmf		# Turns on UEFI
qm set <VMID> --rng0 /dev/urandom	# Required for some of the UEFI/SecureBoot stuff to behave
qm set <VMID> --efidisk0 <ProxmoxStorage>:0	# Required for UEFI settings to stick
qm set <VMID> --tpmstate0 <ProxmoxStorage>:0,version=v2.0	# Required for SecureBoot

Boot the VM and log in as user, break out of systemcheck with Ctrl-C because it will not succeed unless you happen to use 10.0.2.0/24 for your network and your gateway happens to be 10.0.2.2, then you’ll need to edit /etc/network/interfaces.d/30_non-qubes-whonix. Find the iface eth0 inet static stanza and update the address, netmask and gateway to something appropriate for your network. You CAN use DHCP here by commenting out that whole stanza and replacing it with iface eth0 inet dhcp but you’ll need to install the isc-dhcp-client package to get DHCP to actually work (Logistics of doing so are left as an exercise for the reader. Your threat model may vary, I run it on my Workstations because I have Workstations but my Gateway’s “WAN” interface is static). Reboot, do the normal “systemcheck/upgrade if necessary” dance. Optionally install qemu-guest-agent if you want nice shutdown behaviour and the potential information leakage is compatible with your threat model.

Once your Gateway is up and working you can move onto the Workstation;

cd /root/Whonix/LXQt
qm create <VMID> --memory 2048 --name Whonix-Workstation --net0 virtio,bridge=vmbr<WhonixBridge>
qm importdisk <VMID> <Whonix-Workstation...qcow2> <ProxmoxStorage>
qm set <VMID> --virtiofs0 WhonixShare,direct-io=1	# We get files in and out of workstation via VirtioFS (there's another VM which has access to this share which is external to the Whonix ecosystem) you'll probably want to add an `/etc/fstab` entry to automount this at boot.
qm set <VMID> --vga qxl,clipboard=vnc	# Required to have any chance of getting a working clipboard
qm set <VMID> --virtio0 <ProxmoxStorage>:<imported_disk>
qm set <VMID> --boot c --bootdisk virtio0
# The stuff below is mostly only needed if you want UEFI/SecureBoot
qm set <VMID> --machine q35		# My preference for standardisation, omit for i440bx default
qm set <VMID> --cores 4			# I'd suggest at least 2 cores for your Workstation, I have plenty so I give them 4
qm set <VMID> --agent 1			# Potential information leakage vector but I value clean shutdowns and restarts
qm set <VMID> --bios ovmf		# Turns on UEFI
qm set <VMID> --rng0 /dev/urandom	# Required for some of the UEFI/SecureBoot stuff to behave
qm set <VMID> --efidisk0 <ProxmoxStorage>:0	# Required for UEFI settings to stick
qm set <VMID> --tpmstate0 <ProxmoxStorage>:0,version=v2.0	# Required for SecureBoot

Boot and test connectivity, assuming all is well, reboot into a sysmaint session and install the qemu-guest-agent package (assuming that’s within your threat model), while you’re there do the “systemcheck/update if necessary” dance, reboot in to a user session and you should be all good.

Our primary interface surface to the Workstation is a noVNC client (FWIW you can still use noVNC to access the console in the Proxmox web UI even with the display type set to QXL (Spice), you just have to click the down-arrow on the “Console” button and explicitly select it).

NOTES:

  • As noted in the other thread, clipboard support won’t work because spice-vdagent doesn’t currently support wayland (worked fine in Whonix 17 though).
  • Resolution changes DO NOT work (and didn’t even with the older x11 based images. I haven’t had the time or inclination to run that down yet but it might happen after the wayland stuff gets merged into spice-vdagent) at least not with the noVNC client. Haven’t tested with virt-viewer and a direct SPICE connection because that doesn’t work for our use-case (and the HTML5 SPICE client has never been a particularly happy camper when I’ve tested it).
  • Can’t confirm if file-sharing works or not for the same reason as above (as noted we just have a virtiofs share mounted in the Workstation which we use for getting files in and out via another VM, no network connectivity between Whonix and that VM, just the virtiofs share provided by the Proxmox host).
  • If you want multiple workstations you can either;
    • Install them as separate Gateway/Workstation pairs with a separate bridge for each pair.
    • Manually update the IP address of all your Workstations to make them unique.
    • Insert a DHCP server onto the bridge by a means you are comfortable with and which is compatible with your threat model. Set it up to hand out addresses in 10.152.152.0/26 with a gateway of 10.152.152.10, install the isc-dhcp-client package in your Workstations and edit /etc/network/interfaces.d/30_non-qubes-whonix appropriately.
1 Like

Oh, forgot one thing, you may have issues with sdwdate because the guests will automatically insert the kvmclock module which will force time sync to the host, to fix that (both Gateway and Workstation, for Workstation you’ll need to reboot to sysmaint);

Create a file /etc/defaults/grub.d/50_no_kvmclock.cfg containing;

GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX no-kvmclock"

Then run sudo update-grub.

To validate (after a reboot);

cat /sys/devices/system/clocksource/clocksource0/current_clocksource

Should NOT return kvmclock (generally should return tsc). If it does return kvmclock double check the file you created and re-run sudo update-grub, you can also check /proc/cmdline to confirm that the no-kvmclock parameter was passed to the kernel at boot.

1 Like