Kernel Hardening - security-misc

[Patrick_mobile]

Have to disable kernel lockdown. Unfortunately. Because that enforces kernel module signature verification. Which we don’t have yet.

[madaidan]

Disable lockdown

[Patrick]

@madaidan

More kernel hardening

old:

slub_debug=FZP

new:

slub_debug=FZ slub_debug=P

Does the second parameter overwrite the first?

In other words: Is the new effective value still FZP? How to test that?

[Patrick]

kver="$(uname -r)"

That does not work on kernel upgrade because by then the running kernel is still the old one.

security-misc/40_kernel_hardening.cfg at master · Kicksecure/security-misc · GitHub

https://phabricator.whonix.org/T945

[Patrick]

@madaidan

More kernel hardening

Also, I need someone to test if init_on_free=1 breaks in Qubes like page_poison=1 does.

Working.

cat /proc/version

Linux version 5.3.0-0.bpo.2-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 5.3.9-2~bpo10+1 (2019-11-13)

cat /proc/cmdline 

BOOT_IMAGE=/boot/vmlinuz-5.3.0-0.bpo.2-amd64 root=/dev/xvda3 ro xen_scrub_pages=0 root=/dev/mapper/dmroot console=hvc0 console=tty0 swiotlb=8192 noresume intel_iommu=on amd_iommu=on slab_nomerge slub_debug=FZ init_on_alloc=1 init_on_free=1 mce=0 pti=on mds=full,nosmt vsyscall=none page_alloc.shuffle=1

But I am experiencing two other Qubes kernel panics and I don’t know yet what is causing them. (Possible causes: remount-secure.service, lkrg, tirdad or any combination maybe (with above kernel parameters).)

[madaidan]

@Patrick

In other words: Is the new effective value still FZP ? How to test that?

There might be something in dmesg.

We can change it so there’s only one parameter if you want.

[Patrick]

madaidan via Whonix Forum:

@Patrick

In other words: Is the new effective value still FZP ? How to test that?

There might be something in dmesg.

Nothing.

We can change it so there’s only one parameter if you want.

That would be the easiest, and safest.

[madaidan]

Use only one slub_debug parameter

[Patrick]

@madaidan

Use only one slub_debug parameter

Merged.

[madaidan]

We might want to set our sysctl values in earlier boot to better defend the system against rootkits that run before systemd-sysctl is executed.

For example: rootkit executes before kptr_restrict is set → gets access to kernel symbols → can now use those for kernel exploits

Or, even worse: rootkit executes before kexec_load_disabled is set → replaces the running kernel with a malicious kernel

apparmor-profile-everything will prevent the attacker from changing the sysctls themselves so that’s not a problem. The problem is if the attacker manages to launch attacks before they are changed to our safer settings.

The best approach to this would be to hard-code the sysctl values with a kernel patch and make them read-only with __ro_after_init (kernel/git/torvalds/linux.git - Linux kernel source tree) so even arbitrary write kernel exploits can’t overwrite the values.

Or, we can set our sysctl values in the initramfs which is better than systemd-sysctl, but not as good as above.

[torjunkie]

Anything useful for your kernel work? (have no idea, so don’t want to pollute that thread)

https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project/Recommended_Settings

https://www.kernel.org/doc/html/latest/security/self-protection.html

[madaidan]

We’ve applied most of the KSPP recommendations already. The only things we haven’t applied is disabling CONFIG_INET_DIAG and CONFIG_BINFMT_MISC which I need to look into.

[Patrick]

Could you implement set sysctl using kernel command line parameter? upstream at kernel.org?

@madaidan

We might want to set our sysctl values in earlier boot to better defend the system against rootkits that run before systemd-sysctl is executed.

How could that happen?

@madaidan

The problem is if the attacker manages to launch attacks before they are changed to our safer settings.

I don’t see how that would be possible. Would require installing a systemd unit file or initramfs hook which is already prevented by apparmor-profile-everything.

@madaidan

hard-code the sysctl values with a kernel patch

That sounds really inflexible. Maybe a better idea…

@madaidan

[…] sysctl […] make them read-only with __ro_after_init (arch: Introduce post-init read-only memory - kernel/git/torvalds/linux.git - Linux kernel source tree) so even arbitrary write kernel exploits can’t overwrite the values.

Can you submit a patch to kernel.org or linux-hardened which adds a kernel boot parameter ldsysctl “Lock Down Sysctl” (or better name) which makes sysctl read-only after having set those? I.e. a kernel boot parameter which enforces that sysctl can be initially set but the not changed again?

[madaidan]

@Patrick

Could you implement set sysctl using kernel command line parameter? upstream at kernel.org?

I wouldn’t know how to implement that.

@Patrick

How could that happen?

If an attacker manages to compromise systemd through a vulnerability (which aren’t exactly rare Freedesktop Systemd : Security vulnerabilities, CVEs).

Since apparmor-profile-everything confines even systemd, our threat model would include preventing a compromised init from attacking the kernel.

@Patrick

Can you submit a patch to kernel.org or linux-hardened which adds a kernel boot parameter ldsysctl “Lock Down Sysctl” (or better name) which makes sysctl read-only after having set those? I.e. a kernel boot parameter which enforces that sysctl can be initially set but the not changed again?

I probably could, but sysctls are sprawled all over the kernel source and making all of them read-only would be hard to do and maintaining it would also be hard.

Scripting it may be possible though.

[Patrick]

@Patrick

How could that happen?

If an attacker manages to compromise systemd through a vulnerability (which aren’t exactly rare Freedesktop Systemd : Security vulnerabilities, CVEs).

Since apparmor-profile-everything confines even systemd, our threat model would include preventing a compromised init from attacking the kernel.

I see.

Seems a challenge usability wise. Possible issues:

• Hardcoded, hard to change sysctl settings, inflexible.
• Unexpected by users that there are sysctl settings in initramfs.
• Difficult to change sysctl settings in initramfs.
• Different settings in initramfs sysctl vs rootfs sysctl creating
  strange, hard to debug situations.

Solvable.

This could consist of various components.

• sysctl-to-initrd (or so): An initramfs hook which copies
  /etc/sysctl and /etc/sysctl.d (/etc/sysctl(.d)) to initramfs. The
  rootfs /etc/sysctl(.d) would be considered the prime source. initramfs
  /etc/sysctl(.d) would be cleanly created from /etc/sysctl(.d) at
  every run.

• a dpkg trigger that runs when dpkg modified any files in
  /etc/sysctl(.d) (activate-noawait update-initramfs) (See existing
  Whonix source code for that.) (I can probably do that.)

• a systemd unit file that runs on shutdown which re-runs
  update-initramfs -u. But only if needed to save re-creating initramfs
  every time. Re-creating initramfs every time non-conditionally increases
  the risk of unclean shutdown of breaking the system and increases
  shutdown/reboot time.

• maybe better than a systemd unit file would be a inotifywait based
  solution that monitors changes to /etc/sysctl(.d)? Less chance of
  unclean shutdown and destroying initramfs. See whonix-firewall
  /usr/lib/whonix-firewall/firewall-restarter for example use of
  inotifywait. (I can probably do that.) (inotifywait must not
  conflict with the dpkg trigger.)

Initially initramfs-tools based only. Keeping it generic enough so later
support for dracut can be added.

What do you think?

While we are at it, before creating more and more initramfs integration,
are there good reasons to port from initramfs-tools to dracut anyhow? It
would be better to do this sooner than later if there are good reasons
to avoid having to re-implement things later. See also:

replacing initramfs-tools with dracut

If we want to implement a live boot option for Whonix we maybe need to change the default tool for generating the initramfs from the current initramfs-tools to dracut. Therefore, also some minor changes to the build scripts are required. Dracut is mostly used by everything Fedora based (RedHat, CentOS, Qubes) and OpenSuse. Initramfs-tools is used by Debian and derivatives by default. I couldn’t find any important differences between both, which doesn’t mean they don’t exist. Dracut is also in …

@Patrick

Can you submit a patch to kernel.org or linux-hardened which adds a kernel boot parameter ldsysctl “Lock Down Sysctl” (or better name) which makes sysctl read-only after having set those? I.e. a kernel boot parameter which enforces that sysctl can be initially set but the not changed again?

I probably could, but sysctls are sprawled all over the kernel source and making all of them read-only would be hard to do and maintaining it would also be hard.

Shouldn’t setting a sysctl be done through an API / function? Then
perhaps only the relevant function would need to be patched for ldsysctl?

[madaidan]

@Patrick

What do you think?

Sounds good, but instead of continuously regenerating the initramfs, maybe we can apply the sysctl settings after the real rootfs has been mounted e.g.

sysctl -p /new_root/etc/sysctl.d/*.conf

@Patrick

Shouldn’t setting a sysctl be done through an API / function? Then
perhaps only the relevant function would need to be patched for ldsysctl ?

Sysctls are just variables in the kernel that are configurable from userspace. e.g. dmesg_restrict is set with:

int dmesg_restrict = IS_ENABLED(CONFIG_SECURITY_DMESG_RESTRICT);

linux/kernel/sysctl.c at master · torvalds/linux · GitHub allows you to change certain variables with sysctl but isn’t what actually sets them. The variables are set in various other files.

e.g. dmesg_restrict is set in linux/kernel/printk/printk.c at master · torvalds/linux · GitHub

Maybe I could disable changing sysctls with our own proc_handler that does something like

if (write)
      return -EPERM;

and then change all sysctls to use this but then a write flaw could still modify the variables themselves as they weren’t made read-only, but they just weren’t permitted to change them via sysctl.

[Patrick]

madaidan via Whonix Forum:

@Patrick

What do you think?

Sounds good, but instead of continuously regenerating the initramfs, maybe we can apply the sysctl settings after the real rootfs has been mounted e.g.

sysctl -p /new_root/etc/sysctl.d/*.conf

This is of course A LOT better. The implementation stays contained
inside initramfs, no re-generation of initramfs, faster, no unclean
shutdown issue, no de-sync of rootfs sysctl vs initramfs sysctl.
Superadmin could still change sysctl and these would be consistent both,
rootfs sysctl and initramfs sysctl. No usability issue since users don’t
need to remember this when changing sysctl. Awesome! Please implement.

[madaidan]

Now that we’re working on untrusted root, we might want to set ptrace_scope to 3 but this will break a ton of useful debugging tools like strace.

[madaidan]

https://github.com/Whonix/security-misc/pull/53

Should we disable the systemd-sysctl service and apparmor profile in apparmor-profile-everything now?