Is FS-VERITY any good for the desktop?
This would be useful for verified boot and to prevent malware persistence similar to dm-verity.
2 Likes
I don’t think any (Debian based) Linux desktop distribution can/will implement this unfortunately in mid term future. Happy to be proven wrong. Also because non-determinism and unclean separation of system and user folders makes it even harder to provide a signed base image (or filesystem).
dm-verify / fs-verity require a distributor to create and sign and image (or filesystem). Which the user cannot modify (at least not without superroot) (android calls that “rooting” or after market firmware image flashing). Then the package management works differently - it does not write to the root image. Is therefore more limited - unless extensive development is being done. Android added a lot of the missing functionality to make this work. But then users require the distributor (in case of mobile phones their mobile carrier or mobile phone producer) to provide upgrades - which we know - is a very broken process. For way too many phones there aren’t upgrades delivered (and worse so, freedom is denied to users to update their own devices due to locked bootloaders).
Perhaps root (or superroot) could upgrade and then a key generated on the user’s machine could sign the file system or all binaries. Then when booting using multiple boot modes for better security: persistent user | live user | persistent secureadmin | persistent superadmin | persistent recovery mode with user, any modifications to the root file system could be detected. But that seems superuser since Linux file permission prevent that anyhow. Any attacker compromise / escalate these permissions could also subvert dm-verify / fs-verity, I think.
2 Likes
What if we install all user applications into a separate chroot?
So we have the base system image that’s verified with dm-verity. We then create the /apps directory in a separate non-verified image that’s mounted during boot which then has the base system files mounted.
We create an apt wrapper so whenever the user installs a program, we chroot into /apps and install the program.
Whenever the user executes their app it chroots and executes it.
mount /path/to/unverified_image /apps
for dir in bin sbin usr lib lib64 var etc
do
mkdir "/apps/${dir}"
mount -o bind "/${dir}" "/apps/${dir}"
done
mkdir /apps/{proc,sys,dev}
mount proc /apps/proc -t proc
mount sysfs /apps/sys -t sysfs
mount devtmpfs /apps/dev -t devtmpfs
mount devpts /apps/dev/pts -t devpts
apt-get install -o Dir=/apps $program
chroot /apps $program
The only issue with this is all the directories are mounted read-write so any modification to e.g. /apps/bin would modify /bin too. There should be a way to mount the base system as read-only but then everything else in the chroot as read-write.
chroot’s aren’t secure. Research “chroot escape”.
A lot more of that. I don’t think chroots were invented with security in
mind. I guess if chroot’s were secure, nobody would bother with VMs /
containers.
Maybe that’s why CLIP OS uses containers?
madaidan via Whonix Forum:
What if we install all user applications into a separate chroot?
User runs “sudo apt install pkg-name”. How to make install that to chroot?
Another stackable wrappers issue?
At least the initial implementation could surely live without “sudo apt
install pkg-name” replacement and this being an opt-in.
How to make install that to chroot? Maybe it shouldn’t?
Maybe after we have
user user can’t use APT anymore. Maybe after implementing that it
would be a good time to invent new commands such as apt-chroot?
So we have the base system image that’s verified with dm-verity.
Sounds like a lot development work since nobody publicly implemented
that with Debian yet or any Linux desktop distribution.
He has some examples that don’t look too difficult for non-encrypted
(VM) setups.
But then integrating that into the build process is non-trivial.
Also how would we upgrade the base system image?
Whenever the user executes their app it chroots and executes it.
Every app the same chroot or different chroots? Could be same chroot but
home folder isolation? And yet a way to share files among apps through
shared folders? Similar to android?
sudo chroot /apps $program
That would execute the program as root. Even if chroot, we ought to run
as little as possible as root.
The only issue with this is all the directories are mounted read-write so any modification to e.g.
/apps/binwould modify/bintoo. There should be a way to mount the base system as read-only but the everything else in the chroot as read-write.
Qubes TemplateBasedAppVMs can write to the shared root image (including
/usr/bin etc.) of TemplateVM but these changes aren’t actually written
to actual TemplateVM root image. These are written to volatile.img and
discarded after shutdown. However, all applications can write to the
root image. They won’t “notice” anything. Similar to a Live ISO overlayfs?
2 Likes
I know, I’ve looked a lot into chroot escapes for other projects. That’s why I didn’t suggest to use chroots for isolating processes. It’s only to have the base system and app data on 2 different images.
sandbox-app-launcher is what would be used for isolating apps. We’d chroot, then run the app in sandbox-app-launcher.
Create a wrapper that runs sudo apt install -o Dir=/apps pkg-name or maybe we can add an apt configuration file.
It should, hence the -o Dir=/apps.
I think it might be good to allow the user to install their own programs but Debian has so much generic system stuff in the repos other than actual apps that might make this problematic.
I haven’t seen full system MAC policies on desktop Linux either yet we have apparmor-profile-everything.
We should probably look into the update_engine thing Daniel Micay talked about.
The same chroot.
Yes.
That would be trivial to implement. We just need to create /shared-storage or similar.
sandbox-app-launcher would drop privileges (sudo -H -u "${app_user}").
Nice blog post on usefulness of dm-verity generally:
madaidan via Whonix Forum:
I know, I’ve looked a lot into chroot escapes for other projects. That’s why I didn’t suggest to use chroots for isolating processes. It’s only to have the base system and app data on 2 different images.
sandbox-app-launcher is what would be used for isolating apps. We’d chroot, then run the app in sandbox-app-launcher.
Ok, good.
Create a wrapper that runs
sudo apt install -o Dir=/apps pkg-nameor maybe we can add an apt configuration file.
sudo apt install -o Dir=/apps pkg-name doesn’t work. Maye if /apps is
already a complete chroot. But then better use the inside-chroot’s APT?
(Not the outside-chroot’s APT.)
But then again, if user runs outside-chroot “sudo apt” how to translate
that to sandboxed chrooted APT? And should this be done?
We should probably look into the update_engine thing Daniel Micay talked about.
Seems difficult as I couldn’t find anyone who made use of this outside
of Android yet, you?
Quote A/B (seamless) system updates | Android Open Source Project
OEMs not using Google’s OTA infrastructure will be able to reuse the
AOSP system code but will need to supply their own client.
(Fedora) CoreOS / Container Linux might be an alternative, had and
likely still has dm-verity as per:
CoreOS is also (somewhat)
stateless? /etc? But
CoreOS might not work for hosts? (Whonix-Host) (I.e. both Whonix-Host
and Whonix VMs couldn’t be both CoreOS based?)
Then there’s also Fedora Silverblue? Stateless?
While CoreOS and Fedora Silverblue are similar (rpm-ostree based), I
haven’t found if Fedora Silverblue supports dm-verity yet. But since
CoreOS had/has dm-verity, I guess it’s doable for Fedora Silverblue too
and/or even coming somewhat soon supported by upstream Fedora?
Fedora seems more interesting now. Kinda the “holy grail” I am looking
for. Stateless (at least more than Debian) + dm-verity (or fs-verity)
within reach. Though, I am not sure of the other criteria for choosing a
base distribution for Whonix. Wiki page for Fedora notes:
1 Like
It does work as /apps is a complete chroot due to the mounting.
I don’t think there’s any real difference.
A wrapper executes chroot /apps apt $@.
CoreOS has an “update_engine” but I don’t know if it’s the same.
Someone mentioned Silverblue to Daniel and he responded:
no
it’s less bad
it doesn’t do it the way that I’m saying and it probably makes systemd much harder to avoid
I don’t think any of the existing OSes is a useful starting point beyond for use as a way to generate templates
i.e. we have to do it ourselves.
1 Like
Make as few outside-chroot programs process any data inside-chroot for better isolation?
I guess that needs the stackable wrappers?
A wrapper executes
chroot /apps apt $@.
That’s also better since it uses chroot’s apt.
Android update_engine changed license from BSD to Apache
Re-license update_engine to Apache2 · AOSiP/platform_system_update_engine@aea4c1c · GitHub and Copyright (c) 2010 The Chromium OS Authors.
And CoreOS update_engine also has Copyright (c) 2010 The Chromium OS Authors..
At the moment I am not trying to replace Debian / Fedora with a more-secure-by-default base distribution that packages its own wayland, firefox, and whatnot. That would require a complete re-focus. Totally different project. It’s also probably more of an organizational, social challenge. Would need to get a sufficiently big team to join, i.e. maintainers. And then somehow vet these maintainers. Have trusted people that likely won’t be adding backdoors. Before that I also probably would have to try to change Debian / Fedora from the inside first by contributing to them. Also talking to more people from there to see how such an organization works to learn how to build one.
2 Likes
Apt isn’t going to be run inside sandbox-app-launcher as that’d break it so the only isolation would be the chroot which isn’t much isolation at all, especially for root processes.
We can harden chroots to make them give proper isolation like grsecurity’s chroot hardening. I could probably extract that and send it to linux-hardened but I don’t think there’d be much advantage.
There is apparmor-profile-everything’s apt-get profile (it needs more hardening though) for actual isolation but that’s not related to this.
Yes.
FYI AOSiP is a custom ROM, not official AOSP.
What if we were to package a few “core” packages ourselves while still being based on Debian? i.e. whonix.org can provide the base system compiled with clang for CFI, musl etc. with proper security updates.
We don’t need to replicate all of Debian’s packages but just a few main ones.
It’s a compromise for better security but not the best as that’s not currently maintainable.
1 Like
Yes, I took a lazy shortcut not looking at AOSP update_engine source code and assumed AOSiP wouldn’t be much different. Yes interesting to see the connection of Chromium OS with update_engine.
How few are a few?
Also requires a solution for Selecting Secure Packages from packages.debian.org - i.e. objective, doable criteria for judging secure vs non-secure.
How’s that possible?
Use newer software from upstream as they release and package? Well, that’s not going to fly well due to dependency issues. New versions don’t work with Debian stable. Would be similar to mixing Debian testing with Debian stable which doesn’t work great.
- One could also argue in theory (but not in practice) Debian testing is more secure than Debian stable since Debian testing sticks closer to upstream releases. (That however doesn’t hold true during periods of Debian freeze.)
- One could also argue in theory (but again not in practice) that Debian sid is more secure than Debian stable since Debian sid sticks closer to upstream releases. (Ignoring that sid is unstable indeed.)
- Debian rolling was an idea but it didn’t materialize.
A rolling distribution - not “really” rolling but mostly rolling at long periods distribution - Debian testing - was unsuitable as base for Whonix, see Why is Whonix ™ based on Debian Stable, not Debian Testing?
Therefore I don’t think this is feasible with the current project resources.
1 Like
It could be anything. We can make a small list of commonly used packages. Especially packages that parse untrusted input e.g. a document viewer.
We can package the dependencies too but that might not work out well.
Even without the proper security updates, things like CFI alone would be great.
1 Like
For now, this is infeasible:
1 Like
Compiling our own packages or dm-verity? dm-verity sounds pretty feasible.
1 Like
That’s why my last post was about, yes.
That one might be doable indeed.
1 Like
Can we use ostree in Debian to accomplish the same?
@madaidan can you focus on getting a firejail alternative off the ground instead of extending scope too much? Perfect is enemy of the good.
What do you think sandbox-app-launcher is?
Good stuff. I thought from this discussion that progress is contingent on getting ostree to work on Debian which can take a while.
1 Like
Instead of update_engine, we can look into the CLIP OS system updater.
2 Likes