Whonix-Host Operating System (OS) ISO

Nope has nothing to do with baremetal’s IP management and everything to do with NAT network config for hypervisors. So it has to stay this way to avoid creating different settings for KVM vs Vbox.

1 Like

Whonix-Host (currently running inside VirtualBox) is now networking is now by network manager. eth0 got automatically assigned 10.0.2.15.
Now Whonix-Host networking (currently running inside VirtualBox) is functional and both gateway and workstation are starting.
Speed in nested virtualization is too slow to test if networking inside VMs is functional but I will assign more RAM and try again. Can also say more when I build a new image that doesn’t require all the manual changes I did during experimentation.
However, maybe likely this will work?


For QEMU support both of these shouldbe removed?

gateway (actually works without removing this):

<vcpu placement='static' cpuset='0'>1</vcpu>

workstation:

<vcpu placement='static' cpuset='1'>1</vcpu>

Removal was required for me inside VirtualBox. Otherwise I was getting cannot set CPU affinity on process error when trying to start workstation.

Whonix tag 15.0.1.3.2-developers-only should build.

Though, only tested with --remote-derivative-packages true. And had a minor hiccup which hopefully was just a one time thing.

dpkg-deb (subprocess): decompressing archive member: lzma error: compressed data is corrupt
dpkg-deb: error: <decompress> subprocess returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/calamares_3.2.4-3_amd64.deb (--unpack):
 cannot copy extracted data for './usr/lib/x86_64-linux-gnu/calamares/modules/keyboard/libcalamares_viewmodule_keyboard.so' to '/usr/lib/x86_64-linux-gnu/calamares/modules/keyboard/libcalamares_viewmodule_keyboard.so.dpkg-new': unexpected end of file or stream
Errors were encountered while processing:
 /var/cache/apt/archives/calamares_3.2.4-3_amd64.deb

Managed to make Whonix-Host in VirtualBox coexist with KVM networking file Whonix-External.xml.

nmcli con show

NAME UUID TYPE DEVICE
virbr0 ecc61543-fa15-4a43-9732-fa99065ff801 bridge virbr0
virbr2 7b784b6c-44e4-4589-8fba-9ab6f322e810 bridge virbr2
Wired connection 1 2ed217e7-866f-3760-926a-11dedcca3182 ethernet eth0

nmcli con down Wired\ connection\ 1 

Connection ‘Wired connection 1’ successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/1)

virsh -c qemu:///system net-start Whonix-External

Network Whonix-External started

nmcli con up Wired\ connection\ 1 

Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)

I.e. the trick is to let libvirt (whonix-libvirt-install.service) (virsh -c qemu:///system net-start Whonix-External) run before network manager.
Network manager can deal with it but libvirt cannot.
I don’t see any solution for this with systemd drop-ins in this case because:

/lib/systemd/system/network-manager.service has Before=network.target
while
/lib/systemd/system/libvirtd.service has After=network.target.

1 Like

Yes can be safely ignored/removed.

1 Like

A lot of progress has been done lately on the Whonix-Host project.

I think we are quite close to be release-ready, at least in beta state. I also thought a nice sum-up of what blockers/issues remain would be nice to set priorities :slight_smile:

Whonix-Host to-do list as of May 29, 2020
We currently have 21 open issues on phabricator:
https://phabricator.whonix.org/project/view/26/

Let’s break them down (subjective triage by myself):

1. Severity: blocker
needs to be addressed before official release

2. Severity: not a blocker
small issue/might be implemented in a future release

3. Severity: unknown (needs triage)
Maybe a blocker, maybe not, maybe a non-issue. Needs advising

To sum-up, as of today, and pending decision on “need triage” issues, there are only 2 identified blocking issues:

  • Implementation of Whonix-Host firewall
  • Fixing EFI booting problem on installed Whonix-Host

Alternatively, to speed things up, we could decide that the inability of installing Whonix-Host on a EFI machine is not a blocker (as the main innovation we offer is the ability to boot an GRUB/EFI ISO with Whonix VMs pre-installed and pre-configured).

Of course, many other issues will probably pop up once we will have more test time…

2 Likes

Doesn’t need to be blocked. Forensics testing would be OK for beta.

Blocker indeed. It needs to be developed.

Right.

Yes, also does not need to be a blocker. In beta testing can be called a known issue. Perhaps by then we find out or someone else finds out.

Same as above. As long as efi booting is broken, this is not too bad either.

Indeed.

Actually blocker. Without Tor host config, sdwdate and apt is broken by default.

Blocker since this needs to be documented.

Yes.

1 Like

Greetings!
Coming back from a looooong pause… :slight_smile:
But still alive and willing to complete this project.

I just built Whonix-Host from source, from stable branch (15.0.1.3.4).

Good news: what did work still works apparently (didn’t run extensive tests, but on the surface looks good):

  • Boot ISO in BIOS mode: OK
  • Boot ISO in EFI mode: OK
  • Run VM machines in ISO live mode: OK
  • Install in BIOS mode: OK
  • Run VM machines in host persistent (post-installation) mode: OK

“Bad” news (but expected): what didn’t work, still doesn’t work:

  • Whonix-Host (post-installation) fails to boot in EFI, although installation reports no error
  • Whonix-Host Firewall and Whonix-Host Tor configuration and anon-connection-wizard are still not implemented (as far as I know)

I have no idea how to work out the Firewall and Tor configuration issues. I hoped someone more knowledgeable would take up the challenge :slight_smile:

Meanwhile, I will try to solve the EFI issue. I will contact the debian-live team on Gitlab, see if they can help. I will post the link here once it’s posted.
EDIT: I cannot post any issue there, only pull requests.

In the meantime, if we want the EFI installation to work, we must add the modules
sources-media
and
sources-media-unmount
in /etc/calamares/settings.conf otherwise it will fail to download the package grub-efi-amd64 during the installation process and the installation will abort.
See pull request: Update settings.conf.dist by onions-knight · Pull Request #4 · Kicksecure/live-config-dist · GitHub

2 Likes

Welcome back! We missed you around here :smiley:

Yeah grub-efi is definitely a hard dep AFAICT.

1 Like

Yay! :slight_smile: Welcome back!

Whonix-Host Firewall is rather difficult to invent. → Whonix-Host KVM Firewall

Maybe now after working on other things I’ll have new inspiration.

They use mailing lists.

https://lists.debian.org/debian-live/

Merged. Now included in 15.0.1.5.2-developers-only.

A bit non-ideal to download during installation but we can perfect that later (somehow cache the package during build process).

EFI booting might be broken because of this. In a VM with grub-pc installed:

sudo apt install grub-efi-amd64

The following NEW packages will be installed:
efibootmgr grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed mokutil shim-helpers-amd64-signed shim-signed shim-signed-common
shim-unsigned

Most of these packages seem to be Recommends:.

When using

sudo apt install grub-efi-amd64 --no-install-recommends

The following NEW packages will be installed:
grub-efi-amd64 grub-efi-amd64-bin

These Recommends: in Debian packaging terms might actually be required for successful EFI booting.

Since Whonix source code doesn’t mention grub-efi or other packages in the build script or in anon-meta-packages, I guess that could be why EFI booting is broken.

Thank for the warm welcoming back :slight_smile:

Great suggestion.
I installed grub-efi-amd64-bin (wouldn’t allow me to install grub-efi-amd64) and all its dependencies (7 additional packages, forgot to write down which ones exactly…) in the Host raw file and reburnt the ISO.

Still stuck at “Minimal BASH-like editing…” / “grub>” prompt after installation.

1 Like

It’s something really small and stupid that is messing (missing?) somewhere. We will eventually find out.

Installation works, and the Host EFI installed machine actually boots into GRUB if repaired using grub> commands or if it is booted with the ISO still in (I know, strange. Same behavior last time I checked in May).

Of course, all these tests are performed in QEMU/KVM (virt-manager) for now, but if it doesn’t work here at this level it won’t work on real hardware.

1 Like

New functionality added for overlayfs may be useful for live mode:

1 Like

Hello and happy new year!
GOOD NEWS for EFI install mode! :grinning:
I think I have FINALLY found the culprit!

-> Just tried it with version 15.0.1.5.4 (built myself).

It appears that unless specified otherwise, Calamares installation in EFI mode creates a directory on the EFI partition using the default branding name, i.e. in our case “Whonix-Host”:

user@host:/mnt$ ls -lR
.:
total 4
drwxr-xr-x 4 root root 4096 Jan 22 14:05 EFI

./EFI:
total 8
drwxr-xr-x 2 root root 4096 Jan 22 14:05 boot
drwxr-xr-x 2 root root 4096 Jan 22 14:05 Whonix-Host

./EFI/boot:
total 0

./EFI/Whonix-Host:
total 5212
-rwxr-xr-x 1 root root     108 Jan 22 14:05 BOOTX64.CSV
-rwxr-xr-x 1 root root 1206824 Jan 22 14:05 fbx64.efi
-rwxr-xr-x 1 root root     209 Jan 22 14:05 grub.cfg
-rwxr-xr-x 1 root root 1533296 Jan 22 14:05 grubx64.efi
-rwxr-xr-x 1 root root 1261192 Jan 22 14:05 mmx64.efi
-rwxr-xr-x 1 root root 1322936 Jan 22 14:05 shimx64.ef

But for some reason, the installed machine is unable to boot if the directory containing the bootloader files is not named “Debian”.

It seems to be an obscure bug on which few relevant information is readily available online.

-> renaming /EFI/Whonix-Host into /EFI/Debian solves the problem.

I will upload a pull request on gitlab in a few moments.

Apart from that, didn’t run extensive tries, but everything else seemed to work as expected.

All tested on KVM so far (not on real hardware).

2 Likes

https://gitlab.com/whonix/live-config-dist/-/merge_requests/1/commits

2 Likes

Happy New Year to you too. I always love the moments of epiphany when finally hunting down an annoying bug. This is great progress :slight_smile:

2 Likes

Yes, exactly. I wasn’t very actively working on it for months, and decided to give it a try with a fresh head. Then I tried to change the name of this directory and bingo! :slight_smile:

I am currently testing (and writing this post) using Whonix-Workstation on Whonix-Host installed on real hardware (high capacity USB stick)! Works fine.

Only thing, had to change back the domain name from qemu to kvm, as it was set to qemu for testing purposes in KVM (for nested virtualization). Just a reminder that we need to it back in the release candidate. Otherwise the VMs are unbearably slow.

See post 237 by @Patrick:

EDIT: I see that per use qemu if kvm is unavailable for easier Whonix development using ne… · Kicksecure/libvirt-dist@b77262f · GitHub there should be a check during the first installed Host boot for kvm capabilities (If I understand correctly). As I used KVM with virt-manager to install Whonix-Host on the USB stick (easier to debug), that might explain why the VMs xml files were set to qemu on first boot (also in KVM). I should try to run the entire installation process on real hardware to see if kvm domain is correctly set. Then we wouldn’t need to change anything.

2 Likes

So now as per our roadmap we “only” lack Whonix-Host Firewall. Whonix-Host Tor configuration and anon-connection-wizard and we are good to go.

Any news on the development of these features? How can it be helped?

1 Like

https://gitlab.com/whonix/live-config-dist/-/merge_requests/1/commits

Yay! :slight_smile: :slight_smile: :slight_smile:

Welcome back again and happy new year!

whonix-libvirt /usr/lib/whonix-libvirt/install is started by /lib/systemd/system/whonix-libvirt-install.service

https://github.com/Whonix/whonix-libvirt/blob/master/usr/lib/whonix-libvirt/install

That script creates a .done file:

/var/lib/whonix-libvirt/install.done

whonix-libvirt-install.service won’t run /usr/lib/whonix-libvirt/install again if the .done file exists.

ConditionPathExists=!/var/lib/whonix-libvirt/install.done

If you have suggestions on how to improve that please let me know. Would be good to support this use case somehow. Why not. Installation of Whonix-Host in a VM on USB can be a good idea how beginners/testers since that guarantees really installation to USB. Internal boot disk which the user currently booted from remains unchanged. We also need to keep things comfortable for developers/testers.

( There is also a use case of sometimes running an operating system from hardware and sometimes inside a VM. Suppose you just made a full dd backup and now want to try if the backup is functional. Assign the USB drive to a VM and boot it.)

What would be a sane way to implement this? At every boot iterate over all installed libvirt XLM’s (multiple gateway’s, workstation’s) and change back/forth from qemu to kvm? That seems surprising / intrusive? Users who set up some non-Whonix VM to qemu to notice that these where just changed to kvm after reboot.

A bit hard to script. kvm to qemu includes “remove <pvspinlock state='on'/> from XML file” but qemu to kvm would mean “re-add <pvspinlock state='on'/>”. Re-adding is more difficult because it needs to be added in the correct position.

XML files don’t support comments so comment in/out isn’t possible either.

Neither we should delete the user’s local version in /etc/ and copy over the original from /usr/share because then user modifications would be lost.

Yes.

Unfortunately not.

If anyone could implement any of that, that would help.

As for Whonix-Host KVM Firewall I never got any idea how to filter traffic by “VM name”. Simplified: How do we allow Whonix-Gateway to use the internet but prohibit everyone else?

Ideally the user could configure a list of VM names which have networking permitted. The default list would only include Whonix-Gateway.

More advanced:

  • allow Whonix-Gateway (default)
  • allow Whonix-Gateway 2 (custom, multiple Whonix-Gateway)
  • allow Kicksecure VM
  • allow debian-tor user (host operating system Tor process)

After the simplified question is solved, implementing the advanced stuff might actually be easy.

1 Like