Whonix-Host Operating System (OS) ISO

onion_knight · March 10, 2020, 9:05am

Ok, so we want a minimalist install. Just copy the ISO content, disk creation, grub/bootloader and initramf stuff.

I tried again on 15.0.0.9.4 leaving only the following modules:

sequence:

- show:
  - welcome
  - partition
  - summary

- exec:
  - partition
  - mount
  - unpackfs
  - fstab
  - bootloader-config
  - grubcfg
  - bootloader
  - packages
  - luksbootkeyfile
  - initramfscfg
  - initramfs
  - umount

- show:
  - finished

Seems to work well.
(tested in a KVM VM with msdos partition only, both with encrypted and non encrypted settings)

Install went smoothly, newly installed host returns following values:

user@host:~$ locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

user@host:~$ timedatectl
               Local time: Tue 2020-03-10 08:34:55 UTC
           Universal time: Tue 2020-03-10 08:34:55 UTC
                 RTC time: Tue 2020-03-10 08:35:40
                Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: no
              NTP service: inactive
          RTC in local TZ: no



user@host:~$ sudo apt update
Hit:1 tor+https://deb.debian.org/debian-security buster/updates InRelease      
Hit:2 tor+https://deb.debian.org/debian buster InRelease                       
Hit:3 tor+https://deb.whonix.org buster InRelease                              
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.

I am not sure I understand what you mean exactly to achieve here. Do you want the same machine ID on the Host as the Whonix (debootstrap) default (26ada0c0-1165-4098-884d-aafd2220c2c6)? If yes, then I am not sure it is even possible. I removed the machine-id module but still got a new UUID on the installed host:

Non encrypted install:
/dev/vda1: UUID="2417b79c-e29f-4ae8-ae3d-8f697859c251" TYPE=“ext4” PARTUUID=“30036c34-01”
Encrypted install:
/dev/vda1: UUID=“4a885924-ff5e-4649-8f57-f0a830f16a41” TYPE=“crypto_LUKS” PARTUUID=“d93a7116-01”
/dev/vda2: UUID=“f210179a-0c07-4fab-8a9b-589f3878565f” TYPE=“crypto_LUKS” PARTUUID=“d93a7116-02”
/dev/mapper/luks-4a885924-ff5e-4649-8f57-f0a830f16a41: UUID=“0e95d35d-1510-464e-a28b-95f572660c4f” TYPE=“ext4”
/dev/mapper/luks-f210179a-0c07-4fab-8a9b-589f3878565f: UUID=“4f76d8c5-9705-4ed4-9a7c-715486bb4621” TYPE=“swap”

If you want I could share the whole log of the installation. It’s not too long.

Patrick · March 10, 2020, 10:34am

Awesome!

Yes, your module list already looks very nice and minimal.

Probably good. Could you compare please with workstation VM? Should be probably the same, which would be good.

Looks good. Similar to above.

These (UUID="2417b79c-e29f-4ae8-ae3d-8f697859c251" etc.) are disk uuids. These are not considered yet if these should be shared among all Whonix users. These are not considered machine-id. Terminology. Maybe “an ID” or “a machine specific ID” or something but not what I refer to when saying machine-id which refers only to /etc/machine-id and /var/lib/dbus/machine-id. Quite certain on terminology but not super certain.

/etc/machine-d should be same as dist-base-files/etc/machine-id at master · Kicksecure/dist-base-files · GitHub in Whonix gateway, workstation and host. Same thing for dist-base-files/var/lib/dbus/machine-id at master · Kicksecure/dist-base-files · GitHub.

By not using calamares module machineid we probably will have that result.

onion_knight · March 10, 2020, 10:45am

Same result on Whonix-Workstation.

Ok now I understand.

Same result here on installed Host:

user@host:~$ cat /etc/machine-id 
b08dfa6083e7567a1921a715000001fb
user@host:~$ cat /var/lib/dbus/machine-id 
b08dfa6083e7567a1921a715000001fb

onion_knight · March 10, 2020, 10:51am

Pull request

Patrick · March 10, 2020, 11:03am

Merged.

I think it was a good decision to use calamares-settings-debian since Debian sorted out a few Debian specifics with their configs already. Building on top of that is better.

Until now my impression of calamares is really good.

There are various calamares upstream config files for modules that we still require:

^ these need be be understood. We need to check if these defaults are OK or if tweaking is required / advisable.

Similar to above the same applies to the following calamares settings files by Debian package calamares-settings-debian:

A lot configs will overlap calamares upstream vs calamares-settings-debian. I guess calamares-settings-debian will be mostly OK but some tweaks might be needed for Whonix.

onion_knight · March 10, 2020, 11:14am

My understanding is that calamares-settings-debian overrides upstream calamares settings, no?
I can have a look at these files but I will probably not understand all of them I assume we would only look closely at the modules we are actually using, right?

Anything else I could test in the meantime? I was thinking testing again why setting of ro mode fails when booting the ISO.

onion_knight · March 10, 2020, 11:20am

Do we have specific kernel boot flags for Kicksecure (I assume yes)? Then we would probably have to modify/have a look at bootloader.config/grub.cfg modules.

Patrick · March 10, 2020, 12:03pm

Yes, please. I will also look through them but it’s much better if it’s not only me.

Yes. Did I post configs of some modules we’d not be using? If so… Indeed. No need to understand configs of modules we are not going to use. Maybe a superficial understanding to make sure we’re not missing something.

This is correct.
However, I guess it would make sense to understand both upstream calamares and Debian (for the configs we are going to use). And then see if any tweaks are needed.

Good.

I don’t have a roadmap forward.

Actually a lot…

security-misc/etc/default/grub.d at master · Kicksecure/security-misc · GitHub

Good point.

Two things to distinguish:

Whonix ISO live boot
boot of Whonix from HDD after installation

boot of Whonix from HDD after installation: if /etc/grub.d gets copied over as expected and update-grub runs then grub boot config should already correctly pass all linux kernel boot parameters. This again is defined in packages. Therefore we probably don’t need help of calamares to set some linux kernel boot parameters.

Whonix ISO live boot: I don’t know what kernel boot parameters are being used. Same as security-misc sets should be used. Should use the standard way of /etc/default/grub.d / /boot grub.cfg etc. Therefore it might be better to have Whonix ISO live boot use grub instead of syslinux/isolinux if you could make that work?

Could you compare please Whonix VM /proc/cmdline with Whonix Host Hard Drive /proc/cmdline?

Could you compare please Whonix VM /proc/cmdline with Whonix Live ISO /proc/cmdline?

(Not sure about terminology yet. We need easy/consistent use of Live ISO vs Whonix Host Hard Drive Installed. The latter is a bit of a mouthful / a lot to type.)

onion_knight · March 10, 2020, 12:21pm

All clear. I’ll have a look into this grub stuff first.

cat /proc/cmdline on Whonix-Workstation-15.0.0.8.7 (KVM):

BOOT_IMAGE=/boot/vmlinuz-4.19.0-8-amd64 root=UUID=26ada0c0-1165-4098-884d-aafd2220c2c6 ro console=tty0 console=ttyS0,115200n8 random.trust_cpu=off intel_iommu=on amd_iommu=on slab_nomerge slub_debug=FZP page_poison=1 mce=0 pti=on spectre_v2=on spec_store_bypass_disable=on tsx=off tsx_async_abort=full,nosmt mds=full,nosmt l1tf=full,force nosmt=force kvm.nx_huge_pages=force vsyscall=none quiet

cat /proc/cmdline on Whonix-Host-15.0.0.9.4 (KVM, installed on encrypted VM HDD):

BOOT_IMAGE=/boot/vmlinuz-4.19.0-8-amd64 root=UUID=a7a069b1-b644-420a-84ee-230b4b0623b6 ro spectre_v2=on spec_store_bypass_disable=on tsx=off tsx_async_abort=full,nosmt mds=full,nosmt l1tf=full,force nosmt=force kvm.nx_huge_pages=force random.trust_cpu=off intel_iommu=on amd_iommu=on efi=disable_early_pci_dma slab_nomerge slub_debug=FZP page_poison=1 mce=0 pti=on vsyscall=none extra_latent_entropy quiet cryptdevice=UUID=1ab01497-2615-42b6-891b-650a2c15c06a:luks-1ab01497-2615-42b6-891b-650a2c15c06a root=/dev/mapper/luks-1ab01497-2615-42b6-891b-650a2c15c06a resume=/dev/mapper/luks-1ab01497-2615-42b6-891b-650a2c15c06a

There is actually more stuff in the Host version. Maybe some new kernel boot flags were added since 15.0.0.8.7?
But at first glance it seems that config from /etc/grub.d/ has been copied over as expected, therefore your assumption

seems correct.

cat /proc/cmdline Whonix-Host-iso-15.0.0.9.4:

BOOT_IMAGE=/live/vmlinuz initrd=/live/initrd boot=live live-config.user-default-groups=libvirt,kvm apparmor=1 security=apparmor ip=frommedia

This is expected, as these values are taken from here: https://github.com/Whonix/Whonix/blob/master/iso/isolinux.cfg

label Whonix Desktop Live VM Live

menu label ^Whonix Desktop Live VM Live

menu default

kernel /live/vmlinuz

append initrd=/live/initrd boot=live live-config.user-default-groups=libvirt,kvm apparmor=1 security=apparmor ip=frommedia

text help

Boot Whonix Desktop Live VM Live image

endtext

Two solutions I see:

Modify isolinux.cfg
Replace isolinux by grub (probably better?)

I will try to see how to use grub instead of isolinux.

Patrick · March 10, 2020, 12:57pm

No, I don’t think so.

The only extras seems related to host version. Probably added by calamares to /etc/default/grub. cryptdevice=, root= and resume=. These are probably needed and OK.

Makes sense.

Might be better.
Unless isolinux is somehow better nowadays for something still?
Seems like a somewhat complex topic for research.

@HulaHoop any input on isolinux / syslinux vs grub2?

Patrick · March 10, 2020, 1:06pm

Features we care about syslinux vs grub2:

BIOS boot support
EFI boot support
future Secure Boot support
boot from a real DVD (iso)
write an iso to USB and be able too boot it
“Hybrid UEFI GPT + BIOS GPT/MBR boot”
hybrid DVD / USB (didn’t find exact term)
and then it is also possible to chain various bootloaders if that is of any use

Dunno. Any best practice? Maybe easier to follow a role model? Any iso image by any (popular) Linux distribution that is highly compatible with most/all of this?

onion_knight · March 10, 2020, 7:46pm

Good point. Worth looking at how popular distributions (I am thinking Tails, Ubuntu) solve this. No need to reinvent the wheel. By the way kernel boot flags can be as well modified here, no need for grub at all costs.
https://github.com/Whonix/Whonix/blob/master/iso/isolinux.cfg

HulaHoop · March 10, 2020, 10:22pm

Ubuntu seems to bolt syslinux on with Grub using the latter in EFI environments. There’s no apparent advantage I can see with syslinux unless I am missing something. Both can coexist on the same system if one has a great feature that requires it.

onion_knight · March 10, 2020, 11:47pm

At first glance Tails also seems to use a hybrid syslinux (BIOS) and grub (EFI) system. Maybe we shouldn’t over-complicate this thing and keep that also (with modified kernel boot flags)?

Patrick_mobile · March 11, 2020, 4:06am

Alright. Boot parameters:

We can update isolinux.cfg boot parameters manually for now.

(Use a newline (\n) to separate standard boot options (security related) from iso-only boot options for better readability if possible?)

Later there could be a script (that maybe runs during build process as a sanity check) which verifies that boot parameters are up to date. It could copy a grub.cfg from workstation VM image, extract boot parameters and compare with isolinux.cfg.

onion_knight · March 11, 2020, 11:16am

cat /proc/cmdline on Whonix-Host iso with modified isolinux.cfg :

BOOT_IMAGE=/live/vmlinuz initrd=/live/initrd boot=live spectre_v2=on spec_store_bypass_disable=on tsx=off tsx_async_abort=full,nosmt mds=full,nosmt l1tf=full,force nosmt=force kvm.nx_huge_pages=force random.trust_cpu=off intel_iommu=on amd_iommu=on efi=disable_early_pci_dma slab_nomerge slub_debug=FZP page_poison=1 mce=0 pti=on vsyscall=none extra_latent_entropy

I don’t think it’s possible. The only iso-only boot option remaining now is boot=live.

I have removed the ip=frommedia option as we already use network-manager (we have dhcp out of the box, no static ip, thus no need to keep networking from the iso file).

live-config.user-default-groups=libvirt,kvm and apparmor=1 security=apparmor were also removed. Reason: we don’t use live-config anymore and pre-installed user “user” is already a member of libvirt and kvm groups. Apparmor is now enabled by default in debian 10, no need to add a kernel boot flag.

Pull request:
https://github.com/Whonix/Whonix/pull/430

Patrick · March 11, 2020, 1:11pm

Merged.

Patrick · March 12, 2020, 8:02am

https://github.com/Whonix/whonix-libvirt/blob/master/etc/calamares/modules/users.conf.whonix

calamares/users.conf at calamares · calamares/calamares · GitHub
Could you try please setRootPassword: false? We don’t want to ask for or set a root password. That is because root account is locked by Whonix default. (Details: Strong Linux User Account Isolation)
I would like to not set defaultGroups there too (minimal use of calamares) and leave that to some package too. Though, not too important. I wonder if anon-base-files gets installed before whonix-libvirt. If so, that group config could be done in anon-base-files config too.

defaultGroups:
- libvirt
- qemu

Patrick · March 12, 2020, 8:41am

Re roadmap… Actually there is. We used to keep good track of open tasks:

Do you get (e-mail) notifications from Whonix phabricator? Some discussions would be good to have there.

Please see: ⚓ T914 Whonix Host Live - enable KVM readonly mode - virt-xml vm-name --edit --disk readonly=on

Patrick_mobile · March 12, 2020, 8:57am

Created new git tag that contains all the latest branding related changes just now.