Kernel Hardening - security-misc

[Patrick]

madaidan via Whonix Forum:

Now that we’re working on untrusted root, we might want to set ptrace_scope to 3 but this will break a ton of useful debugging tools like strace.

https://github.com/Whonix/security-misc/blob/master/etc/sysctl.d/ptrace_scope.conf
already breaks strace. I had to use sudo sysctl -w kernel.yama.ptrace_scope=0 so I could use strace.

If something is not useful on it’s own for security-misc package (?),
add to apparmor-profile-everything package?

Maybe we should invent a package debug (not installed by default)
which ships config snippets which resets to normal debug
kernel.yama.ptrace_scope=0, kernel.sysrq=1 and what not?

[madaidan]

@Patrick

already breaks strace.

Only for non-root users. Run strace as root and it works fine.

Setting ptrace_scope to 2 restricts ptrace to root only, setting it to 3 denies access to ptrace entirely.

@Patrick

If something is not useful on it’s own for security-misc package (?), add to apparmor-profile-everything package?

This would still be useful regardless if apparmor-profile-everything is used.

@Patrick

Maybe we should invent a package debug (not installed by default) which ships config snippets which resets to normal debug kernel.yama.ptrace_scope=0 , kernel.sysrq=1 and what not?

It would be preferable if it sets ptrace_scope to 2 as that still allows ptrace but only for root. ptrace is really dangerous is it gives malware an easy way to compromise any program in a user’s session. Disabling ptrace_scope wouldn’t be the best.

kernel.sysrq=1 is not the default. Dunno what Debian sets it to though.

[madaidan]

https://github.com/Whonix/security-misc/pull/54

[Patrick]

@madaidan

Maybe we should invent a package debug (not installed by default) which ships config snippets which resets to normal debug kernel.yama.ptrace_scope=0 , kernel.sysrq=1 and what not?

It would be preferable if it sets ptrace_scope to 2 as that still allows ptrace but only for root.

A package debug which isn’t installed by default is a tool which must be as useful as possible for me. In past there was a situation where I needed to use strace to debug an application which should not be run as root (GUI application). Installing a debug package and then still needing to fiddle more (relax more settings) seems not useful for me. In Linux Kernel Runtime Guard (LKRG) - Linux Kernel Runtime Integrity Checking and Exploit Detection - #26 by Patrick I experienced a hard to debug issue which required serial console and SysRq. It wasn’t easy to set up a debug environment to gather any useful output. If it’s already hard for me as a developer, than I can only imagine how hard it would be for advanced users or new developers trying to debug something. Therefore there need to be strong development tools. A trigger someone can pull (superroot) sudo apt install debug to have access to unrestricted debugging. To debug something isn’t the most visible, rewarding task anyhow but having roadblocks on the way makes it even less likely someone could be bothered to look.

@madaidan

kernel.sysrq=1 is not the default.

Indeed.

[Patrick]

madaidan via Whonix Forum:

Document panic_on_oops by madaidan · Pull Request #54 · Kicksecure/security-misc · GitHub

Merged.

[Patrick]

madaidan via Whonix Forum:

Set sysctl values in initramfs by madaidan · Pull Request #53 · Kicksecure/security-misc · GitHub

Merged.

Could you please also process /etc/sysctl.conf? I personally don’t
like /etc/sysctl.conf file since there is /etc/sysctl.d folder but
this is required for consistency. /etc/sysctl.conf is still used by
many sysadmins and in many security guides.

Should we disable the systemd-sysctl service and apparmor profile in apparmor-profile-everything now?

I don’t know. Does that that work? The less code, the better. I guess
it’s no longer needed since no sysctl would be changed anyhow since
these get applied even already in initial ramdisk? systemd-sysctl trying
to re-enforce already existing settings wouldn’t require any permissions?

On the other hand, is it OK to say apparmor-profile-everything depends
on security-misc? Would that be the case?

[madaidan]

@Patrick

Does that that work?

Yes. I’ve tested it.

The main reason for this is so we can get rid of the systemd-sysctl profile so even if systemd-sysctl is compromised, it can’t change sysctls.

@Patrick

On the other hand, is it OK to say apparmor-profile-everything depends on security-misc?

Yes. Most apparmor-profile-everything users will be using security-misc anyway.

[madaidan]

@Patrick

Could you please also process /etc/sysctl.conf ? I personally don’t
like /etc/sysctl.conf file since there is /etc/sysctl.d folder but
this is required for consistency. /etc/sysctl.conf is still used by
many sysadmins and in many security guides.

Process sysctl.conf in initramfs

[Patrick]

madaidan via Whonix Forum:

@Patrick

Could you please also process /etc/sysctl.conf ? I personally don’t
like /etc/sysctl.conf file since there is /etc/sysctl.d folder but
this is required for consistency. /etc/sysctl.conf is still used by
many sysadmins and in many security guides.

Process sysctl.conf in initramfs by madaidan · Pull Request #55 · Kicksecure/security-misc · GitHub

Merged.

Dunno what initramfs does if a script fails? If sysctl fails, we should
probably not break the boot? To be on safe side, I added this:

error handling

(unless you have ideas for more explicit error handling)

[madaidan]

If one script fails, it shouldn’t break boot.

[Patrick]

add update initramfs trigger

[Patrick]

This wasn’t conclusive debug output.

sudo dmesg | grep lkrg

[   89.832261] p_lkrg: loading out-of-tree module taints kernel.
[   89.850921] p_lkrg: module verification failed: signature and/or required key missing - tainting kernel
[   89.852290] [p_lkrg] Loading LKRG...
[   91.952994] [p_lkrg] LKRG initialized successfully!
[   92.017905] [p_lkrg] Disabling MSRs verification during CI.
[   92.047093] [p_lkrg] [ED] New pCFI configuration => 1 (No stackwalk (weak))
[  510.949628] [p_lkrg] <Exploit Detection> !!! BLOCKING UMH !!!
[  510.949632] [p_lkrg] <Exploit Detection> Someone is trying to execute file: [/bin/false]
[  510.949633] [p_lkrg] <Exploit Detection> --- . ---

sudo sysctl -a | grep lkrg

lkrg.block_modules = 0
lkrg.ci_panic = 0
lkrg.clean_message = 0
lkrg.enforce_msr = 0
lkrg.enforce_pcfi = 1
lkrg.force_run = 0
lkrg.hide = 0
lkrg.log_level = 1
lkrg.random_events = 1
lkrg.smep_panic = 1
lkrg.timestamp = 15
lkrg.umh_lock = 0

grep -r /bin/false /etc/modprobe.d

/etc/modprobe.d/vivid.conf:install vivid /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install dccp /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install sctp /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install rds /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install tipc /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install n-hdlc /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install ax25 /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install netrom /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install x25 /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install rose /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install decnet /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install econet /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install af_802154 /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install ipx /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install appletalk /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install psnap /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install p8023 /bin/false
/etc/modprobe.d/uncommon-network-protocols.conf:install p8022 /bin/false
/etc/modprobe.d/blacklist-dma.conf:install firewire-core /bin/false
/etc/modprobe.d/blacklist-dma.conf:install thunderbolt /bin/false
/etc/modprobe.d/msr.conf:install msr /bin/false
/etc/modprobe.d/blacklist-bluetooth.conf:install bluetooth /bin/false
/etc/modprobe.d/blacklist-bluetooth.conf:install btusb /bin/false

At least I’d like to know which module specifically was refused loading.

Instead of install vivid /bin/false I tried install vivid /bin/false vivid. I.e. passing a parameter to /bin/false which would be ignored by /bin/false but useful to see in logs. That didn’t work. LKRG won’t show it.

It was security-misc kernel.core_pattern=|/bin/false

Therefore what do you think about changing /etc/modprobe.d/vivid.conf from

install vivid /bin/false

to

install vivid /bin/false_vivid

To a non-existing binary.

Any reason against that?

[Patrick]

hide stdout (but not stderr) by sysctl during initramfs

---

I’ve renamed / merged some files in security-misc to make these easier to overrule from debug-misc opt-in package to easily enable debugging.

[madaidan]

@Patrick

Any reason against that?

If an attacker somehow controls /bin/false_vivid, they can execute arbitrary code as root.

Dunno if an attacker can reasonably control that file though.

[Patrick]

Attackers have exactly as much or as little control over /bin/false too?

---

The sysctl early loading inspired me. Would it make sense to load some kernel modules more early in initramfs too? initramfs supports force_load.

force_load

adds a module (and its dependencies) to the initramfs image and also
unconditionally loads the module during boot. Also supports passing arguments to
the module by listing them after the module name.

Useful kernel modules come to mind could be jitterentropy_rng and LKRG [1]. Others?

---

[1] Though LKRG latter needs to be discussed here Linux Kernel Runtime Guard (LKRG) - Linux Kernel Runtime Integrity Checking and Exploit Detection and/or with upstream due to complexities.)

[madaidan]

@Patrick

Attackers have exactly as much or as little control over /bin/false too?

/bin/false already exists and has its permissions correctly set.

I don’t really like the idea of it pointing to an unknown file. What if /bin/false_vivid already exists but with 777 perms? An attacker can write whatever they want there and it will be regularly executed as root.

@Patrick

Useful kernel modules come to mind could be jitterentropy_rng and LKRG [1]. Others?

jitterentropy should be built-in, not a module. We already do that with hardened-kernel though.

LKRG and tirdad should be loaded as early as possible. Preferably, they would also be built-in but that’s not supported (yet?).

[Patrick]

@Patrick

Attackers have exactly as much or as little control over /bin/false too?

/bin/false already exists and has its permissions correctly set.

I don’t really like the idea of it pointing to an unknown file. What if /bin/false_vivid already exists but with 777 perms?

Nothing in /bin does ever exist (without previous system compromise or created by system administrator) that has permissions 777 (i.e. writeable by others than root). Any argument against /bin/false_vivid could also be made against /bin/false.

An attacker can write whatever they want there and it will be regularly executed as root.

@Patrick

Useful kernel modules come to mind could be jitterentropy_rng and LKRG [1]. Others?

jitterentropy should be built-in, not a module. We already do that with hardened-kernel though.

Alright.

However, as long as we’re not using hardened-kernel by default and for non-users of hardened-kernel, does it make sense the load jitterentropy_rng more early?

See also:

load jitterentropy_rng kernel module using the kernel command line / kernel built-in jitterentropy_rng?

jitterentropy_rng is currently only used by the in-kernel DRBG.

What uses the in-kernel DRBG? Asked in above ticket too.

LKRG and tirdad should be loaded as early as possible. Preferably, they would also be built-in but that’s not supported (yet?).

tirdad statically in kernel, asked upstream, link here:

TCP ISN CPU Information Leak Protection - tirdad - #7 by Patrick

Also:

TCP ISN CPU Information Leak Protection - tirdad

Why should tirdad be loaded as early as possible? Currently tirdad is loaded before networking comes up through systemd-modules-load.service.

Also:

Linux Kernel Runtime Guard (LKRG) - Linux Kernel Runtime Integrity Checking and Exploit Detection

As for LKRG loading it as early as possible is non-trivial. Currently loading LKRG as early as possible would cause some issues. For example it would detect changes by tirdad as malicious and show a lot false positive messages. Asked upstream about it: Compiling LKRG static into the Kernel / Loading LKRG kernel module as early as possible or after other modules?

[Patrick]

Another kernel module as candidate for early loading inside initrd: virtio-rng
Questions: would that break VirtualBox or cause issues on the host?

Or rather than maybe re-inventing a lot things, see if systemd has already overtaken initrd too?

Initrd Interface mentions dracut.

Perhaps better to port Whonix to dracut instead?

replacing initramfs-tools with dracut

If we want to implement a live boot option for Whonix we maybe need to change the default tool for generating the initramfs from the current initramfs-tools to dracut. Therefore, also some minor changes to the build scripts are required. Dracut is mostly used by everything Fedora based (RedHat, CentOS, Qubes) and OpenSuse. Initramfs-tools is used by Debian and derivatives by default. I couldn’t find any important differences between both, which doesn’t mean they don’t exist. Dracut is also in …

Or pass on dracut too and wait until minitrd is available from packages.debian.org?

Minitrd - A minimal initramfs

minitrd is 20 times faster than dracut. It is part of the experimental distri distro that is about implementing fast package management with no goal to make it readily support other OSs.

[madaidan]

@Patrick

However, as long as we’re not using hardened-kernel by default and for non-users of hardened-kernel, does it make sense the load jitterentropy_rng more early?

Yes.

@Patrick

Another kernel module as candidate for early loading inside initrd: virtio-rng

That’s probably auto-loaded before initrd anyway.