VMs do no start until the CPU configuration is set to “Copy host CPU configuration”

@onion_knight in Whonix Desktop Installer with Calamares - field report - #109 by onion_knight

Did this also happen for you if you used KVM “normally” “as a normal user” as per the usual instructions on Whonix ™ for KVM on Debian buster hosts?

No, this only happens using the Whonix-Host system.

1 Like

OK, I think I found what is the problem :slight_smile:

First, at least one of the kernel hardening boot parameters somehow messes with the CPU detection on the host (in my case, a quadcore showing only two physical cores, the virtual cores being ignored), which heavily decreases performances. I need to further investigate this issue to find out which specific boot parameter(s) causes that.

But the main problem is that currently the VMs domain types in the host are set as qemu and not kvm. Indeed, this is done on purpose during building as a workaround to configure the VMs:

## workaround to replace the 'kvm' domain type with 'qemu' otherwise libvirtd service will fail to start in chroot
sed -i "1 s/^.*$/<domain type='qemu'>/" "$temp_dir/xml/Whonix-Gateway.xml"
sed -i "1 s/^.*$/<domain type='qemu'>/" "$temp_dir/xml/Whonix-Workstation.xml"

The problem is that this temporary change must be reverted back once the VMs have been configured, which was initially taken care of, but these lines were (probably mistakenly) removed from the same file:

## now we can replace back 'qemu' with 'kvm' domain type
sed -i "8 s/^.*$/<domain type='kvm'>/" $CHROOT_DIRECTORY/etc/libvirt/qemu/Whonix-Gateway.xml
sed -i "8 s/^.*$/<domain type='kvm'>/" $CHROOT_DIRECTORY/etc/libvirt/qemu/Whonix-Workstation.xml

See my pull request :slight_smile: (I hope I changed the right file! but you get the idea…)

1 Like

When Whonix Host starts in a VM… To make developer builds more easy…
I could write a systemd unit file that does “something” only when detecting virtualization. Even only if some forms of virtualization are detected. Perhaps (not) do something when detected running inside VirtualBox / Xen (Qubes) but (not) do something when running inside KVM.
“something” could be such as switching from KVM to QEMU.
At that stage the VMs are already imported so probably another command is needed to change from KVM to QEMU?
I just need to know the conditions (which virtualization or any virtualization) and which command to run.

Or better… Does Nested KVM Virtualization work for you? I see it documented here:
Whonix for KVM

Can you enable Nested KVM Virtualization on the physical host as well as inside Whonix Host (running in VM)?

It it sane to enable Nested KVM Virtualization on Whonix Host by default?

Otherwise we get minimal QEMU support for developer testing purposes? @HulaHoop

That is only if we cannot get Nested KVM Virtualization functional?

Copied here: Kernel Hardening - security-misc - #15 by HulaHoop

EDIT: forget what I just wrote and erased, I missed the part where you said you were talking about “When Whonix Host starts in a VM”.
Will try nested KVM virtualization and report back.

1 Like
  1. I confirm that running (manually in my case)
sed -i "8 s/^.*$/<domain type='kvm'>/" $CHROOT_DIRECTORY/etc/libvirt/qemu/Whonix-Gateway.xml
sed -i "8 s/^.*$/<domain type='kvm'>/" $CHROOT_DIRECTORY/etc/libvirt/qemu/Whonix-Workstation.xml

solves the initial problem, as expected.

  1. I confirm that once enabled on host, nested KVM virtualization works, both with ISO and on install target!

Actually, I am currently writing this post from a Whonix-Host install running in KVM in live-mode, with both Whonix-VM in live-mode too! Isn’t that amazing? :slight_smile: It’s not even that slow! Perfectly fine for testing!

2 Likes

See screenshot below:

Host running Whonix-Host install (booted in live mode) on KVM with virt-manager -> Whonix-Host running Whonix-Gateway and Whonix-Workstation VMs in live-mode (nested KVM virtualization)

2 Likes

Emulation should always be a last resort because performance.

@onion_knight Are there changes I need to do to the libvirt files to make things easier?

1 Like

@HulaHoop not that I know of.

2 Likes

What (simply) virsh define

virsh -c qemu:///system define /usr/share/whonix-libvirt/xml/Whonix-Gateway.xml

is does is simply copy /usr/share/whonix-libvirt/xml/Whonix-Gateway.xml to /etc/libvirt/qemu/Whonix-Gateway.xml?

(The file name in name Whonix-Gateway.xml folder /etc/libvirt/qemu/ is not extracted from the path /usr/share/whonix-libvirt/xml/Whonix-Gateway.xml but from <name>Whonix-Gateway</name>.

I.e. if you replace <name>Whonix-Gateway</name> with <name>Whonix-Gateway</name> then /etc/libvirt/qemu/Whonix-Gateway-test.xml is created.

Does (simply) virsh define do something else too on top of that?

I am asking because rather than (simply) virsh define we could avoid using (simply) virsh define and copy over the file manually instead. That would spare us all the creation of a temporary file and all the sed calls.

Are both QEMU and KVM xml files inside the same folder /etc/libvirt/qemu?

Do you have folder /etc/libvirt/kvm anywhere on any “usual” (KVM VM version as of now) on any of your hosts?

It does or maybe “it might”. Unrelated command (⚓ T914 Whonix Host Live - enable KVM readonly mode - virt-xml vm-name --edit --disk readonly=on)

sudo virt-xml Whonix-Workstation --edit --disk readonly=on

changes on top of what is expected additionally domain type from kvm to qubes on my Qubes test machine.

Did you need to enable Nested KVM Virtualization inside the Whonix-Host build which was running inside a VM?

Merged. And modified/fixed.

In conclusion, this should be fixed in whonix-libvirt git master.

Yay!

Terrific! :slight_smile: :partying_face:

As far as I remember (but would need testing anew to confirm), virsh define command was necessary as “simply copying” the file would not would not persist after reboot or something like that… Or maybe the following virsh commands would not work otherwise…

On my debian buster host: yes, everything is inside this directory, and I have no /etc/libvirt/kvm directory.

No, only on the physical host (as the virtualized host probably “doesn’t know” it is virtualized?).

1 Like