Whonix host operating system

[onion_knight]

EDIT:
I forgot this one
https://phabricator.whonix.org/T914
Maybe not a blocker, but should definitely be taken care of, as it is inconvenient (as currently needs editing on first boot after Whonix-Host persistent installation in order to make it work).

[Patrick]

@onion_knight

This may however be a problem (not a blocker to me anyway) if swap is really a concern to you, as Calamares installer creates a swap partition by default:
⚓ T904 Whonix-Host Swap Considerations - Swap Partition vs Swap File vs No Swap By default

Yes, ideally there would be a swap file but it’s not super critical as its encrypted swap partition.

@onion_knight

Solutions:

• Don’t care (the installation should be on a encrypted disk anyway, swap is encrypted too)

Yes, that’s it for now since calamares does not support swap file support yet.

@onion_knight

See how we can change Calamares settings so as to not provide a swap partition by default during partition

Not sure that is a good idea. For users with 4 GB RAM and more that might be OK but with less I don’t know if we should care and/or if swap would help.

[onion_knight]

Currently building 15.0.1.0.2-developers-only from scratch, to see how it goes.

[Patrick]

Whonix ™

Jos haluat nähdä sisällön, kirjaudu sisään tai rekisteröidy

[Patrick]

• Should Whonix host be fully torified by default?
• Should Whonix-Host have any features besides hosting Whonix VMs in a secure way?
• Whonix-Host Firewall

[Patrick]

This is a search for project whonix-host, whonix-15 with status open.

https://phabricator.whonix.org/maniphest/query/_Obk7yld9FTN/#R

These tasks I think we should get done before the first release aka Whonix-Host release blocker.

All open Whonix-Host tasks (also lower priority, non-blockers):

https://phabricator.whonix.org/maniphest/query/tJZ0eiJ0CrLl/#R

[onion_knight]

@Patrick

This is a search for project whonix-host , whonix-15 with status open .

⚓ Query: Advanced Search

These tasks I think we should get done before the first release aka Whonix-Host release blocker.

All open Whonix-Host tasks (also lower priority, non-blockers):

⚓ Query: Advanced Search

Great, I can take
https://phabricator.whonix.org/T910
https://phabricator.whonix.org/T969
I’ll first start with a new build, see if everything works fine, and test it further.

[onion_knight]

Whonix-Host-15.0.1.0.7-developers-only: field report

I build a new Whonix-Host 15.0.1.0.7, as well as Whonix-Gateway and Whonix-Workstation VMs and tested it out on real hardware. Not an extensive test, just to see if everything works as expected.

First of all, there is one thing important to consider during Whonix-Host build. If gw and ws VMs are not compressed first, the Calamares installation will take hours, and likely fail if not installing on a disk with less than 250 GB capacity. This is because Calamares installer seems to “think” that Whonix VMs really use 100 GB of space each. Compressing the gw and ws disk images BEFORE building Whonix-Host using qemu-img convert solves the problem:

qemu-img convert -f qcow2 -O qcow2

This is just an observation for builders, it is done manually at this stage, but could be added in the whonix-host build process.

Results:

Whonix-Host ISO/Calamares

• Consider changing /etc/motd: “Whonix GNU/Linux” → “Whonix-Host GNU/Linux”

• BIOS installation works fine. Installed and tested on real hardware. → great!

• EFI installation is broken. The installation cannot proceed (more on this later).

• EFI ISO doesn’t use the current kernel flags, see my pull requests (also replaced “Whonix Desktop” by “Whonix-Host” in both grub.cfg and isolinux.cfg files):
  https://github.com/Whonix/Whonix/pull/432
  https://github.com/Whonix/Whonix/pull/433

• Whonix-Host currently does not have ntfs filesystem support. Quite annoying. I suggest adding ntfs-3g package. Maybe also consider adding dosfstools package. Both packages are very lightweight. (“dosfstools consists of the programs mkfs.fat, fsck.fat and fatlabel to create, check and label file systems of the FAT family.” GitHub - dosfstools/dosfstools: dosfstools consists of the programs mkfs.fat, fsck.fat and fatlabel to create, check and label file systems of the FAT family.)

• Some functionalities to fully support desktop integration of mounting encrypted and non-encrypted disks appear to be missing: this means encrypted partitions must be decrypted and mounted manually (likely related to some polkit configuration nightmare…).

Whonix gw/ws VMs integration

• Whonix-VM resolution size is fixed (automatic 1920x1080 on first boot) → great!
• Whonix-VMs desktop background colors are currently wrong (inverted: gw uses ws color and ws uses gw color). Couldn’t do a pull request as I don’t remember what files take care of this

Whonix-Host installed version

• Installed Whonix-Host GRUB menuentry is currently “Whonix GNU/Linux” → shouldn’t it be changed to “Whonix-Host GNU/Linux”? Same thing for /etc/motd

• persistent-mode-to-read-write half works: after installation, Whonix VMs are correctly set to -disk readonly=off but their file permissions are still read-only (0440). They will fail to boot in virt-manager (Error starting domain: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/var/lib/libvirt/images/Whonix-Workstation.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Block node is read-only)

• nfts filesystem, mounting disks in thunar: see above

In general, what further tests would you suggest me to perform on an installed Whonix-Host system?

[Patrick]

Awesome!

Packages: yes, please add here anon-meta-packages/control at master · Whonix/anon-meta-packages · GitHub

@onion_knight

First of all, there is one thing important to consider during Whonix-Host build. If gw and ws VMs are not compressed first, the Calamares installation will take hours, and likely fail if not installing on a disk with less than 250 GB capacity. This is because Calamares installer seems to “think” that Whonix VMs really use 100 GB of space each. Compressing the gw and ws disk images BEFORE building Whonix-Host using qemu-img convert solves the problem:

qemu-img convert -f qcow2 -O qcow2

This is just an observation for builders, it is done manually at this stage, but could be added in the whonix-host build process.

Strange.

Don’t we create a sparse qcow2 image here? //cc @HulaHoop

https://github.com/Whonix/Whonix/blob/master/build-steps.d/2400_convert-raw-to-qcow2#L37

And then don’t we copy the sparse qcow2 image as sparse qcow2 image into the Whonix-Host raw image?

https://github.com/Whonix/Whonix/blob/master/build-steps.d/1800_copy_vms_into_raw#L32

Anything that needs to be fixed to make it sparse indeed?

Related:

Sparse files

@onion_knight

qemu-img convert -f qcow2 -O qcow2

I don’t understand this command.

• -f filename
• -O output format

Possible to use long instead of short qemu-img command line parameters?

We convert qcow2 to qcow2? At what point that command says “use compression”?

Does that mean we have to settle for compressed qcow2 images for Whonix-Host ISO and Whonix-Host installed? Performance degradation? Unpack on installed Whonix-Host?

@onion_knight

Consider changing /etc/motd : “Whonix GNU/Linux” → “Whonix-Host GNU/Linux”

Yeah. There’s /etc/motd.d and /etc/issue.d nowadays. I’d like to port to that.

@onion_knight

Some functionalities to fully support desktop integration of mounting encrypted and non-encrypted disks appear to be missing: this means encrypted partitions must be decrypted and mounted manually (likely related to some polkit configuration nightmare…).

Hopefully fixed when working on these:

• ⚓ T965 install gvfs by default / fix access LUKS encrypted USB drive with Thunar
• ⚓ T961 fix USB auto mounting bug / document
• ⚓ T966 fix pkexec

Need to make special note to test Whonix-Host too.

@onion_knight

Whonix-VMs desktop background colors are currently wrong (inverted: gw uses ws color and ws uses gw color). Couldn’t do a pull request as I don’t remember what files take care of this

I thought the blue color is most friendly, pretty and workstation the place to spend most time. Therefore added there.

@onion_knight

Installed Whonix-Host GRUB menuentry is currently “Whonix GNU/Linux” → shouldn’t it be changed to “Whonix-Host GNU/Linux”? Same thing for /etc/motd

Should be doable. That, /etc/issue, /etc/motd could use a ticket.

@onion_knight

persistent-mode-to-read-write half works: after installation, Whonix VMs are correctly set to -disk readonly=off but their file permissions are still read-only (0440). They will fail to boot in virt-manager ( Error starting domain: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/var/lib/libvirt/images/Whonix-Workstation.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Block node is read-only )

Please add comment to the ticket.

@onion_knight

In general, what further tests would you suggest me to perform on an installed Whonix-Host system?

I guess ⚓ T910 anti-forensics / amnesia testing of Whonix-Host in Live mode is the only major test for now that comes to mind that you’re already aware off. Everything else you seem to have covered already.

[onion_knight]

@Patrick

I don’t understand this command.

• -f filename
• -O output format

Possible to use long instead of short qemu-img command line parameters?

We convert qcow2 to qcow2? At what point that command says “use compression”?

Does that mean we have to settle for compressed qcow2 images for Whonix-Host ISO and Whonix-Host installed? Performance degradation? Unpack on installed Whonix-Host?

It’s just an empiric finding. The term “compress” is probably not appropriate. When using qemu-img convert command, the .qcow2 size seems to match its real size, not its virtual potential one:

user@host:~$ sudo ls -lh /var/lib/libvirt/images/
total 5.3G
-rwxr-xr-x 1 libvirt-qemu libvirt-qemu 2.5G Mar 24 14:34 Whonix-Gateway.qcow2
-r--r----- 1 root         root         2.9G Mar 24 14:34 Whonix-Workstation.qcow2

(instead of 100G before qemu-img).

I didn’t notice any performance degradation. Once again, “compress” is probably not the right term.

Full command:

qemu-img convert -f qcow2 -O qcow2 /path_to_whonix_vms/whonix_vms.qcow2 /path_to_whonix_binary/whonix_vms.qcow2

@Patrick

Don’t we create a sparse qcow2 image here? //cc @HulaHoop

https://github.com/Whonix/Whonix/blob/master/build-steps.d/2400_convert-raw-to-qcow2#L37

And then don’t we copy the sparse qcow2 image as sparse qcow2 image into the Whonix-Host raw image?

https://github.com/Whonix/Whonix/blob/master/build-steps.d/1800_copy_vms_into_raw#L32

I guess we could probably replace the cp --sparse=always command by qemu-img convert. In my experience, the result is the same, only .qcow2 images “advert” their real size when using qemu-img convert.

@Patrick

Hopefully fixed when working on these:

• ⚓ T965 install gvfs by default / fix access LUKS encrypted USB drive with Thunar
• ⚓ T961 fix USB auto mounting bug / document
• ⚓ T966 fix pkexec

Need to make special note to test Whonix-Host too.

OK!

@onion_knight

Whonix-VMs desktop background colors are currently wrong (inverted: gw uses ws color and ws uses gw color). Couldn’t do a pull request as I don’t remember what files take care of this

I thought the blue color is most friendly, pretty and workstation the place to spend most time. Therefore added there.

OK, so not an issue. Although I prefer it the other way personnally

@onion_knight

persistent-mode-to-read-write half works: after installation, Whonix VMs are correctly set to -disk readonly=off but their file permissions are still read-only (0440). They will fail to boot in virt-manager ( Error starting domain: internal error: process exited while connecting to monitor: qemu-system-x86_64: -drive file=/var/lib/libvirt/images/Whonix-Workstation.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Block node is read-only )

Please add comment to the ticket.

OK

@Patrick

I guess ⚓ T910 anti-forensics / amnesia testing of Whonix-Host in Live mode is the only major test for now that comes to mind that you’re already aware off. Everything else you seem to have covered already.

Ok, will test later

I’ll try to find out what’s happening with this EFI installation failure. More details on that later.

[HulaHoop]

@Patrick

Don’t we create a sparse qcow2 image here? //cc @HulaHoop

Yes

Maybe the problem onion_knight ran into was caused by incorrect order of operations?

@Patrick

And then don’t we copy the sparse qcow2 image as sparse qcow2 image into the Whonix-Host raw image?

Yes

@Patrick

We convert qcow2 to qcow2? At what point that command says “use compression”?

Seems redundant. qcow2 does the optimal compression by default IIRC. I will read more.

EDIT:

We already do the best we can in the compression department.

Tip: compress raw disk images using qcow2

Of course the degree of compression you get depends on the amount of zeroed free space in the image, and the amount by which qcow2 is able to compress the other blocks containing data.

qcow2 uses zlib for compression, so the compression won’t be that spectacular. It’s better to keep the filesystems “sparse” in the first place, by ensuring unused disk blocks are zeroed.

For ext2/3 filesystems, Fedora ships a utility called zerofree, which you can either run inside the guest, or run offline from guestfish. This turns unused filesystem blocks into zeroes, which will make outside compression eg with qcow2 much more efficient. For other filesystems, the usual trick is to create a large file of all zeroes until you fill up the free space, then delete it.

[onion_knight]

As I said I don’t think it’s a compression issue, I think it is related to how Calamares “perceives” the gw/ws VM size when it rsync the ISO squashfs filesystem into the install target. When copying .qcow2 files created by cp --sparse it interprets their virtual 101G size as a real size, whereas when copying .qcow2 files obtained with qemu-img convert, it sticks to their actual size.

See also ls command output:

user@host:~$ ls -lh Whonix-Gateway-15.0.1.0.7.qcow2 
-rw-r--r--  1 user user 101G Mar 22 18:30 Whonix-Gateway-15.0.1.0.7.qcow2
user@host:~$ cp --sparse=always Whonix-Gateway-15.0.1.0.7.qcow2 Whonix-Gateway-cp-sparse.qcow2     
user@host:~$ qemu-img convert -f qcow2 -O qcow2 Whonix-Gateway-15.0.1.0.7.qcow2 Whonix-Gateway-qemu-img.qcow2
user@host:~$ ls -lh Whonix-Gateway-*
-rw-r--r-- 1 user user 101G Mar 22 18:30 Whonix-Gateway-15.0.1.0.7.qcow2
-rw-r--r-- 1 user user 101G Mar 25 18:31 Whonix-Gateway-cp-sparse.qcow2
-rw-r--r-- 1 user user 2.5G Mar 25 18:32 Whonix-Gateway-qemu-img.qcow2

[onion_knight]

@Patrick

Awesome!

Packages: yes, please add here anon-meta-packages/control at master · Whonix/anon-meta-packages · GitHub

See pull request. Added packages as dependencies for whonix-host-xfce-kvm-freedom. Not sure it’s the right place to have them?
https://github.com/Whonix/anon-meta-packages/pull/22

@Patrick

Don’t we create a sparse qcow2 image here? //cc @HulaHoop

Yes

Maybe the problem onion_knight ran into was caused by incorrect order of operations?

And then don’t we copy the sparse qcow2 image as sparse qcow2 image into the Whonix-Host raw image?

Yes

It’s definitely weird, as during the build qcow2 images are actually created using qemu-img convert
https://github.com/Whonix/Whonix/blob/master/build-steps.d/2400_convert-raw-to-qcow2#L37

But this is an empiric observation that I already did in the past: when using qemu-img convert in Whonix script, size is kept at 101G (although it’s a virtual size). When using qemu-img convert manually outside the script on the same files, the resulting qcow2 images are shown as using their real size, i.e. approximately 2.5G. I don’t have an explanation for that. Tested again just now:

user@host:~$ ls -lh Whonix-Gateway-XFCE-15.0.1.0.7.raw
-rw-r--r-- 1 user user 100G Mar 26 22:17 Whonix-Gateway-XFCE-15.0.1.0.7.raw
user@host:~$ qemu-img convert -f raw -O qcow2 Whonix-Gateway-XFCE-15.0.1.0.7.raw Whonix-Gateway-XFCE-15.0.1.0.7.test-qemu-img.qcow2
user@host:~$ ls -lh Whonix-Gateway-XFCE-15.0.1.0.7.*
-rw-r--r-- 1 user user 100G Mar 26 22:17 Whonix-Gateway-XFCE-15.0.1.0.7.raw
-rw-r--r-- 1 user user 2.5G Mar 26 22:20 Whonix-Gateway-XFCE-15.0.1.0.7.test-qemu-img.qcow2

@onion_knight

Installed Whonix-Host GRUB menuentry is currently “Whonix GNU/Linux” → shouldn’t it be changed to “Whonix-Host GNU/Linux”? Same thing for /etc/motd

Should be doable. That, /etc/issue, /etc/motd could use a ticket.

Do you want me to create a ticket on phabricator?

[Patrick]

onion_knight via Whonix Forum:

@Patrick

Awesome!

Packages: yes, please add here anon-meta-packages/control at master · Whonix/anon-meta-packages · GitHub

See pull request.

Merged.

Added packages as dependencies for whonix-host-xfce-kvm-freedom. Not sure it’s the right place to have them?

Not sure if perfect but good enough for now. Could be split into a
different package later on that makes clear that it’s non-essential.
Supposing these packages should only be there on an Whonix-Host XFCE
build but not inside VMs.

Do you want me to create a ticket on phabricator?

Yes.

[Patrick]

add 'grub-live | grub-live-boot | boot-live' to kicksecure-xfce

^ Kicksecure-Hosts and Whonix-Hosts will now have grub-live installed by default. I.e. installed Whonix-Host should have the grub live boot menu entries.

---

When booting the host for the first time it might be interesting to boot into recovery mode. In Whonix VMs I could see that /home/user does not exist yet at that time (no pam mkhomedir run yet), which is expected. Might be interesting for installed Whonix-Host.

[Patrick]

Using qemu-img now instead of cp --sparse=always to hopefully fix sparse file issue.

https://github.com/Whonix/Whonix/commit/7cb5ca4e7f52de5cfd7557655fc05e629e565dd1

See TODO in commit above.

Should we use these qemu-img command line parameters?

• -p - progress - that one shouldn’t hurt.
• -o cluster_size=2M
• -o preallocation=metadata

Or would these cause the issue?

[Patrick]

15.0.1.0.8-developers-only includes above fix as well as grub, /etc/issue and /etc/motd branding specific to host, gateway, workstation. Untested.

[onion_knight]

I’ve just built 15.0.1.0.8-developers-only

/etc/motd and /etc/issue are empty (no content inside, applies to Whonix-Host and gw/ws).

Besides, something went completely wrong, VMs were unbootable (“Boot failed: not a bootable disk”).

It seems that this was caused by a faulty qemu-img convert command in 1800_copy_vms_into_raw

Changing -f raw to -f qcow2 and rebuilding Whonix-Host seemed to fix the issue.

Se my pull request:
https://github.com/Whonix/Whonix/pull/434

More testing later.

[Patrick]

@onion_knight

Update 1800_copy_vms_into_raw

Merged, thanks!

@onion_knight

/etc/motd and /etc/issue are empty (no content inside, applies to Whonix-Host and gw/ws).

This is expected.
Changed files: https://github.com/Whonix/Whonix/commit/f59f94616188d1c3d3fa69b60ec62cdc2ea6aa19
Implementation:

---

• anon-gw-base-files/30_whonix-gateway.issue at master · Whonix/anon-gw-base-files · GitHub
• https://github.com/Whonix/anon-gw-base-files/blob/master/usr/lib/motd.d/30_whonix-gateway.motd
• anon-gw-base-files/30_whonix-gateway.cfg at master · Whonix/anon-gw-base-files · GitHub

---

• anon-ws-base-files/30_whonix-workstation.issue at master · Whonix/anon-ws-base-files · GitHub
• https://github.com/Whonix/anon-ws-base-files/blob/master/usr/lib/motd.d/30_whonix-workstation.motd
• anon-ws-base-files/30_whonix-workstation.cfg at master · Whonix/anon-ws-base-files · GitHub

---

• https://github.com/Whonix/whonix-libvirt/blob/master/etc/issue.d/30_whonix-host.issue
• https://github.com/Whonix/whonix-libvirt/blob/master/etc/default/grub.d/30_whonix-host.cfg
• https://github.com/Whonix/whonix-libvirt/blob/master/usr/lib/motd.d/30_whonix-host.motd

---

• motd - Debian Wiki on motd.d
• pam_motd man page - pam - System Administration | ManKier on motd.d
• getty(8) — util-linux — Debian buster — Debian Manpages on /etc/issue.d

---

motd and issue should now be dynamically created from drop-in configuration folder. Any idea why this isn’t working?