My question is, is it possible to mount an iso image similar to it being possible mounting a raw image for read/write? Once a bootable (raw, iso, anything) “base” (just Debian) image exists, it is easy for the other Whonix build steps to take over. [which can then install a Whonix meta package]
Or has an iso to be created in on run? I.e. to we have to leave it all to live-boot and once the iso is finalized we cannot mount it, chroot it, modify it, unchroot it, unmount it, deploy it?
More answers to come later.
Looks amazing!
For sure!
Certainly not a blocker. I find without LVM even easier to grasp and recover from boot medium.
Great!
I really think that we are close to be production ready, we have an actual proof of work, we “only” need to make sure everything works well and of course integrate everything in the host build process in a clean and professional way (I did everything by hand with dirty bash chroot scripts or directly on the cli).
I think it shouldn’t be a problem at all. live-build can be run integrally (lb build) or can be broken down into different steps that can be run separately, all of them completely customizable. It also allows for the creation of disk images. The ISO creation comes last in the build process. As quoted from man live-build:
LIVE-BUILD COMMANDS
We divide live-build into high level ("porcelain") commands and low
level ("plumbing") commands.
Here is the complete list of all available live-build commands. See
their man pages for additional documentation.
HIGH-LEVEL COMMANDS (PORCELAIN)
We separate the porcelain commands into the main commands and some
ancillary user utilities.
Main porcelain commands
lb config(1)
creates configuration for live-build
lb bootstrap(1)
executes the first build stage, creating (bootstraping) a basic
Debian root filesystem
lb chroot(1)
executes the second build stage, building the live OS filesystem
lb installer(1)
executes the third build stage, obtaining installer components
(optional)
lb binary(1)
executes the fourth build stage, generating a binary image
lb source(1)
executes the fifth build stage, generating a source image
(optional)
lb clean(1)
cleans up system build directories
Documentation is quite good and abundant, see for example
But the question is do we need live-build? What for? In your opinion, what specific needs would cover live-build tools that we don’t currently cover with the current Whonix Host build-steps?
We already have a bootable raw image, that can be converted into a bootable BIOS/UEFI (“burnable”) ISO file with mksquashfs and xorriso commands (see my script above). What would be the purpose of live-build?
In my experience, the only real problem I was faced with was avoiding the copy of the existing user and his home directory into the target during the install part with Calamares. My solution was simply to suppress the user and his directory and let the live-config tools take care of the rest, which automatically creates a live-user. Quite basic, but functional.
If it’s not acceptable, maybe we can simply avoid creating the user during the Hardened Debian VM build? Or maybe we could play with the live-config scripts so they would ignore the existing user and his home directory and login with the live-user at boot time (just thinking out loud, I don’t know if it’s possible)? Or maybe even easier to somehow configure the Calamares installer so it would ignore the existing user and its home directory during the unpack step of the install process?
This being said, there might be a ton of things that I am missing. Such as making sure that the live system AND the installed system do honor the various hardening settings that are the main purpose of the “hardened” Whonix version. Also making sure that we don’t copy useless stuff into the target (I am thinking about logs, files owned by the initial user, etc.).
I could easily verify that with my current build, if you or anyone knowledgeable enough would provide me with a check-list of the hardened settings that we expect to be working (apparmor stuff, etc.). I must say that I didn’t take the time to real dig into what makes the hardened version special compared to a stock debian version.
#create_squashfs_ok is currently out commented. Why’s that?
Also other functions are out commented currently which is a bit confusing.
[not sure yet if help-step or build-steps.d but it is easily moved later on]
Once it is in Whonix/help-steps I can easily make it use variables rather than hardcoded strings such as for CHROOT_FOLDER, ISO_FILE, and whatnot. Also re-use code for (un)mount-raw (if possible, sensible, probably yes).
The would be step one. A hardened debian iso which can be live booted.
Would look probably like this: sudo ~/Whonix/whonix_build --flavor hardened-debian --target iso --build
But that’s not a Whonix-Duo Installer ISO yet. So step two would be to implement --flavor whonix-duo-xfce-kvm which would install a different meta package whonix-duo-xfce-kvm. (1700_install-packages)
Therefore a new package whonix-duo-xfce-kvm needs to be added to anon-meta-packages. What packages whonix-duo-xfce-qcow2 should depend on? Could you send a pull request for anon-meta-packages please or let me know which packages it should depend on?
I guess the usual KVM dependencies qemu-kvm libvirt-daemon-system libvirt-clients virt-manager,
and calamares?
Anything else?
Then we’d have a bootable Whonix-Duo Live Installer ISO with calamares installer, KVM dependencies, but not Whonix KVM images yet. I am not sure yet how to best get the Whonix KVM images installed on Whonix-Duo (host) during the build process. See my idea to package Whonix KVM images as [Help Welcome] KVM Development - staying the course - #289 by Patrickdeb packages. sudo apt-mark hold whonix-gateway-xfce-qcow2 sudo apt-mark hold whonix-workstation-xfce-qcow2
could skip the whole anon-base-files.postinst maintainer script.
Yes.
Would not worry much. Could be called “out of scope” for now. Hardened Debian still is underdeveloped. As long as package hardened-debian-xfce is installed, the job of Whonix host is done. Anything missing in hardened debian can be called “future work”.
That would be up to Calamares. But if that happens, that is not that bad. It’s non-deterministic contents created on the user’s machine. Not during the build process of the iso.
What’s really bad is creating let’s say a random seed or Tor entry guards and shipping that in an iso which then gets shared by thousands of users.
But there might be one unsupported use case:
Boot Live; then install; and assume the iso live session activities did not leak into the install, right?
Great, I’ll try to do that when I have sometime (maybe not today).
Yes, the script definitely needs some cleaning, the uncommented step is a mistake. Also the -e boot options has to be removed (it excludes the /boot directory from the copied squashfs file, Calamares needs it to make the target system bootable (it won’t reinstall the kernel by default).
I will make a list of all added packages for the “Whonix Desktop” image. Basically, it’s KVM/virt-manager dependencies + free/non-free firmware (needed for wifi support) + network manager related packages (user-friendly wifi setup) + Calamares dependencies + some theming.
I did not install these additional packages with the --no-install-recommends option, didn’t want anything to break and spend hours to debug what recommended package was missing for the sake of sparing a few hundreds MB.
On a side note, I think ideally the Calamares installer should have an option asking the user whether he would like to install non-free packages or not on the target system. If yes = keep it as it is, if no = remove the non-free packages (this is taken care of during the packages step of the installer).
I am sure it is easily doable, though I don’t know how yet.
OK
Yes
I ran userdel -r user with root
OK, seems the easiest way to me right now as I have no idea how to implement all the other options I was thinking of (or if it is even possible).
Indeed, that would be really bad. But as far as I know, the live session activities do not leak as the installer only copies the filesystem.squashfs file which does not get modified by the file session.This file is just a squashfs’ed image of the .raw disk image packaged into an ISO.
So my understanding right now is that if the master .raw image is clean, then its filesystem.squashfs image should be clean too, and so should be the newly installed system. But I will do some more testing to verify this claim.
On reflection hardened debian based builds should not even have anon-base-files installed. It’s not a dependency in anon-meta-packages. Hardened debian and Whonix host is going to need its own base files package.
Freedom Software purists won’t be satisfied by that. They don’t want to download any iso including anything nonfreedom in the first place let alone boot anything nonfreedom including. Therefore would be futile. Instead I’ll be providing two meta packages and two build flavors pure and impure. Relatively minor extra effort for me. [However, I probably won’t be uploading purist builds. That should be done by someone else or self-made builds.]
Anything else on there should be (no) free vs nonfree discussion, please move here:
No. You’ll be missing the volume snapshotting feature that LVM provides, but that’s it. This can be compensated by having a filesystem that support this instead.
/usr/lib/whonix-libvirt/install is currently not (yet?) idempotent, meaning
it cannot be re-run without error
if it breaks in the middle, it cannot recover when run again
Not yet added to postinst but soon.
I am not sure what’s best. It could be easily made idempotent but then we would keep re-running its commands on each time whonix-libvirt gets upgraded. I guess the best solution is to make it idempotent but run it only at initial installation. That’s what I’ll be going for unless there are better suggestions.