Kernel Hardening - security-misc

BOOT_IMAGE=/boot/vmlinuz-5.3.0-0.bpo.2-amd64 root=/dev/xvda3 ro xen_scrub_pages=0 root=/dev/mapper/dmroot console=hvc0 console=tty0 swiotlb=8192 noresume intel_iommu=on amd_iommu=on slab_nomerge slub_debug=FZ init_on_alloc=1 init_on_free=1 mce=0 pti=on mds=full,nosmt vsyscall=none page_alloc.shuffle=1

But I am experiencing two other Qubes kernel panics and I don’t know yet what is causing them. (Possible causes: remount-secure.service, lkrg, tirdad or any combination maybe (with above kernel parameters).)

madaidan · December 23, 2019, 2:20pm

There might be something in dmesg.

We can change it so there’s only one parameter if you want.

Patrick · December 23, 2019, 2:31pm

madaidan via Whonix Forum:

Patrick:

In other words: Is the new effective value still FZP ? How to test that?

There might be something in dmesg.

Nothing.

We can change it so there’s only one parameter if you want.

That would be the easiest, and safest.

madaidan · December 23, 2019, 7:46pm

Patrick · December 24, 2019, 11:36am

Merged.

madaidan · December 27, 2019, 3:07pm

We might want to set our sysctl values in earlier boot to better defend the system against rootkits that run before systemd-sysctl is executed.

For example: rootkit executes before kptr_restrict is set → gets access to kernel symbols → can now use those for kernel exploits

Or, even worse: rootkit executes before kexec_load_disabled is set → replaces the running kernel with a malicious kernel

apparmor-profile-everything will prevent the attacker from changing the sysctls themselves so that’s not a problem. The problem is if the attacker manages to launch attacks before they are changed to our safer settings.

The best approach to this would be to hard-code the sysctl values with a kernel patch and make them read-only with __ro_after_init (kernel/git/torvalds/linux.git - Linux kernel source tree) so even arbitrary write kernel exploits can’t overwrite the values.

Or, we can set our sysctl values in the initramfs which is better than systemd-sysctl, but not as good as above.

torjunkie · December 27, 2019, 1:15pm

Anything useful for your kernel work? (have no idea, so don’t want to pollute that thread)

https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project/Recommended_Settings

https://www.kernel.org/doc/html/latest/security/self-protection.html

madaidan · December 27, 2019, 2:32pm

We’ve applied most of the KSPP recommendations already. The only things we haven’t applied is disabling CONFIG_INET_DIAG and CONFIG_BINFMT_MISC which I need to look into.

Patrick · December 28, 2019, 9:49am

Could you implement set sysctl using kernel command line parameter? upstream at kernel.org?

How could that happen?

I don’t see how that would be possible. Would require installing a systemd unit file or initramfs hook which is already prevented by apparmor-profile-everything.

That sounds really inflexible. Maybe a better idea…

Can you submit a patch to kernel.org or linux-hardened which adds a kernel boot parameter ldsysctl “Lock Down Sysctl” (or better name) which makes sysctl read-only after having set those? I.e. a kernel boot parameter which enforces that sysctl can be initially set but the not changed again?

madaidan · December 28, 2019, 2:02pm

I wouldn’t know how to implement that.

If an attacker manages to compromise systemd through a vulnerability (which aren’t exactly rare Freedesktop Systemd : Security vulnerabilities, CVEs).

Since apparmor-profile-everything confines even systemd, our threat model would include preventing a compromised init from attacking the kernel.

I probably could, but sysctls are sprawled all over the kernel source and making all of them read-only would be hard to do and maintaining it would also be hard.

Scripting it may be possible though.

Patrick · December 30, 2019, 7:50am

Patrick:

How could that happen?

If an attacker manages to compromise systemd through a vulnerability (which aren’t exactly rare Freedesktop Systemd : Security vulnerabilities, CVEs).

Since apparmor-profile-everything confines even systemd, our threat model would include preventing a compromised init from attacking the kernel.

I see.

Seems a challenge usability wise. Possible issues:

Hardcoded, hard to change sysctl settings, inflexible.
Unexpected by users that there are sysctl settings in initramfs.
Difficult to change sysctl settings in initramfs.
Different settings in initramfs sysctl vs rootfs sysctl creating
strange, hard to debug situations.

Solvable.

This could consist of various components.

sysctl-to-initrd (or so): An initramfs hook which copies
/etc/sysctl and /etc/sysctl.d (/etc/sysctl(.d)) to initramfs. The
rootfs /etc/sysctl(.d) would be considered the prime source. initramfs
/etc/sysctl(.d) would be cleanly created from /etc/sysctl(.d) at
every run.
a dpkg trigger that runs when dpkg modified any files in
/etc/sysctl(.d) (activate-noawait update-initramfs) (See existing
Whonix source code for that.) (I can probably do that.)
a systemd unit file that runs on shutdown which re-runs
update-initramfs -u. But only if needed to save re-creating initramfs
every time. Re-creating initramfs every time non-conditionally increases
the risk of unclean shutdown of breaking the system and increases
shutdown/reboot time.
maybe better than a systemd unit file would be a inotifywait based
solution that monitors changes to /etc/sysctl(.d)? Less chance of
unclean shutdown and destroying initramfs. See whonix-firewall
/usr/lib/whonix-firewall/firewall-restarter for example use of
inotifywait. (I can probably do that.) (inotifywait must not
conflict with the dpkg trigger.)

Initially initramfs-tools based only. Keeping it generic enough so later
support for dracut can be added.

What do you think?

While we are at it, before creating more and more initramfs integration,
are there good reasons to port from initramfs-tools to dracut anyhow? It
would be better to do this sooner than later if there are good reasons
to avoid having to re-implement things later. See also:

Patrick:

Can you submit a patch to kernel.org or linux-hardened which adds a kernel boot parameter ldsysctl “Lock Down Sysctl” (or better name) which makes sysctl read-only after having set those? I.e. a kernel boot parameter which enforces that sysctl can be initially set but the not changed again?

I probably could, but sysctls are sprawled all over the kernel source and making all of them read-only would be hard to do and maintaining it would also be hard.

Shouldn’t setting a sysctl be done through an API / function? Then
perhaps only the relevant function would need to be patched for ldsysctl?

madaidan · December 30, 2019, 11:03pm

Sounds good, but instead of continuously regenerating the initramfs, maybe we can apply the sysctl settings after the real rootfs has been mounted e.g.

sysctl -p /new_root/etc/sysctl.d/*.conf

Sysctls are just variables in the kernel that are configurable from userspace. e.g. dmesg_restrict is set with:

int dmesg_restrict = IS_ENABLED(CONFIG_SECURITY_DMESG_RESTRICT);

linux/kernel/sysctl.c at master · torvalds/linux · GitHub allows you to change certain variables with sysctl but isn’t what actually sets them. The variables are set in various other files.

e.g. dmesg_restrict is set in linux/kernel/printk/printk.c at master · torvalds/linux · GitHub

Maybe I could disable changing sysctls with our own proc_handler that does something like

if (write)
      return -EPERM;

and then change all sysctls to use this but then a write flaw could still modify the variables themselves as they weren’t made read-only, but they just weren’t permitted to change them via sysctl.

Patrick · December 31, 2019, 5:21am

madaidan via Whonix Forum:

Patrick:

What do you think?

Sounds good, but instead of continuously regenerating the initramfs, maybe we can apply the sysctl settings after the real rootfs has been mounted e.g.
sysctl -p /new_root/etc/sysctl.d/*.conf

This is of course A LOT better. The implementation stays contained
inside initramfs, no re-generation of initramfs, faster, no unclean
shutdown issue, no de-sync of rootfs sysctl vs initramfs sysctl.
Superadmin could still change sysctl and these would be consistent both,
rootfs sysctl and initramfs sysctl. No usability issue since users don’t
need to remember this when changing sysctl. Awesome! Please implement.

madaidan · January 12, 2020, 8:56pm

Now that we’re working on untrusted root, we might want to set ptrace_scope to 3 but this will break a ton of useful debugging tools like strace.

madaidan · January 12, 2020, 9:44pm

https://github.com/Whonix/security-misc/pull/53

Should we disable the systemd-sysctl service and apparmor profile in apparmor-profile-everything now?

Patrick · January 13, 2020, 5:07pm

madaidan via Whonix Forum:

Now that we’re working on untrusted root, we might want to set ptrace_scope to 3 but this will break a ton of useful debugging tools like strace.

https://github.com/Whonix/security-misc/blob/master/etc/sysctl.d/ptrace_scope.conf
already breaks strace. I had to use sudo sysctl -w kernel.yama.ptrace_scope=0 so I could use strace.

If something is not useful on it’s own for security-misc package (?),
add to apparmor-profile-everything package?

Maybe we should invent a package debug (not installed by default)
which ships config snippets which resets to normal debug
kernel.yama.ptrace_scope=0, kernel.sysrq=1 and what not?

madaidan · January 13, 2020, 5:15pm

Only for non-root users. Run strace as root and it works fine.

Setting ptrace_scope to 2 restricts ptrace to root only, setting it to 3 denies access to ptrace entirely.

This would still be useful regardless if apparmor-profile-everything is used.

It would be preferable if it sets ptrace_scope to 2 as that still allows ptrace but only for root. ptrace is really dangerous is it gives malware an easy way to compromise any program in a user’s session. Disabling ptrace_scope wouldn’t be the best.

kernel.sysrq=1 is not the default. Dunno what Debian sets it to though.