kernel recompilation for better hardening

[Patrick]

Using new VM and using https://github.com/madaidan/hardened-vm-kernel/blob/master/usr/share/hardened-vm-kernel/kernel-config - compilation failing.

user@debian-buster-standalone:~/kernel/linux-source-4.19$ make deb-pkg
make clean
/bin/bash ./scripts/package/mkdebian
  TAR     linux-4.19.67-rt24 -rt24 -rt24.tar.gz
tar: You may not specify more than one '-Acdtrux', '--delete' or  '--test-label' option
Try 'tar --help' or 'tar --usage' for more information.
make[1]: *** [scripts/package/Makefile:73: deb-pkg] Error 2
make: *** [Makefile:1374: deb-pkg] Error 2

[Patrick]

Above error does not happen with your old kernel config.

To automate “make menuconfig” just use make oldconfig instead?
(As per bash - How to script make menuconfig to automate Linux kernel build configuration? - Unix & Linux Stack Exchange)
Could that help to provide a shorter kernel .config file in the repository?

(Used old config + make oldconfig. Compilation in progress. Seems to be past that error.)

[HulaHoop]

@Patrick

Don’t wrap APT and tell users to use wapt /usr/bin/wapt (wrapped apt) instead? WAPT apt could be a wrapper to run APT normally but before and after APT any number of hooks could be run. One such hook could be “if hardened-vm-kernel created new packages, install these at the end”. Cleaner but also not great as we’d have to reach users to use WAPT rather than APT and constantly explain this.

I like this. Maybe reserve it exclusively for hardened kernel upgrade process instead of making it the default apt for Whonix? This can be an important message that belongs in the terminal.

[HulaHoop]

@madaidan can you make sure CONFIG_TUN is included or else OpenVPN won’t work. Also while we’re at it, perhaps toggle processor specific optimization for faster runtime (if it doesn’t slow down build time too much)

[HulaHoop]

ccache should also be a nice speedup for repeated building in the future when upgrading (although I never managed to get it to work). I am not sure though if it can help when we are randomizing kernel structure for security - the code may be too different and require a re build instead of using the cached stuff. All this can be detected by benchmarking the build time.

https://nickdesaulniers.github.io/blog/2018/06/02/speeding-up-linux-kernel-builds-with-ccache/

[madaidan]

@Patrick

Are you sure? make menuconfig creates other files too.

Yes, those other files are just needed for configuring the config, not for the actual kernel.

@Patrick

Could you please check the diff between your original kernel config and the one based on the Debian kernel config?

That’s what https://github.com/madaidan/hardened-vm-kernel/commit/407dc6100bfc1708e89c4429daff59d35d0660ba is. There have been minor commits since though.

@Patrick

Could that help to provide a shorter kernel .config file in the repository?

We can do that if you want. I don’t see any advantage though.

@HulaHoop

@madaidan can you make sure CONFIG_TUN is included or else OpenVPN won’t work. Also while we’re at it, perhaps toggle processor specific optimization for faster runtime (if it doesn’t slow down build time too much)

CONFIG_TUN is enabled. I haven’t touched much processor-related settings so if the default Debian config enables them, my one does too (I based it off the Debian config).

Just Ctrl + F https://github.com/madaidan/hardened-vm-kernel/blob/master/usr/share/hardened-vm-kernel/kernel-config for any other option to see if it’s there.

[madaidan]

@Patrick

Using new VM and using https://github.com/madaidan/hardened-vm-kernel/blob/master/usr/share/hardened-vm-kernel/kernel-config - compilation failing.

Have you updated the VM? It might be some outdated packages.

[madaidan]

@Patrick

A few years I invented GitHub - adrelanos/apt-during-apt: https://www.whonix.org/Impressum but it’s a giant hack. I don’t like it at all.

I think it’d be a better solution than messing with wrappers.

@Patrick

How do we do that? Replace /usr/bin/apt and /usr/bin/apt-get with a wrapper? Not easy/clean since APT is already wrapped by uwt . And there are no stackable wrappers yet:

Why not just use the already existing apt-get-update-plus?

[Patrick]

@HulaHoop

Don’t wrap APT and tell users to use wapt /usr/bin/wapt (wrapped apt) instead? WAPT apt could be a wrapper to run APT normally but before and after APT any number of hooks could be run. One such hook could be “if hardened-vm-kernel created new packages, install these at the end”. Cleaner but also not great as we’d have to reach users to use WAPT rather than APT and constantly explain this.

I like this. Maybe reserve it exclusively for hardened kernel upgrade process instead of making it the default apt for Whonix? This can be an important message that belongs in the terminal.

Not Whonix specific indeed. Would be generic. Is that what you meant?

@HulaHoop

Also while we’re at it, perhaps toggle processor specific optimization for faster runtime (if it doesn’t slow down build time too much)

Which processor[s]?

@madaidan

We can do that if you want. I don’t see any advantage though.

Yes, unless something speaks against it. The idea is: smaller file size. Easier to review. And keeping auto generated code out of source code files.

@madaidan

Using new VM and using https://github.com/madaidan/hardened-vm-kernel/blob/master/usr/share/hardened-vm-kernel/kernel-config - compilation failing.

Have you updated the VM? It might be some outdated packages.

No outdated packages.

@madaidan

A few years I invented GitHub - adrelanos/apt-during-apt: https://www.whonix.org/Impressum but it’s a giant hack. I don’t like it at all.

I think it’d be a better solution than messing with wrappers.

apt-during-apt comes with grave disadvantages:

• running in background, without user knowledge
• upgrades might be interrupted. User might think upgrading is done but the systemd unit file might still be working in background.

@madaidan

How do we do that? Replace /usr/bin/apt and /usr/bin/apt-get with a wrapper? Not easy/clean since APT is already wrapped by uwt . And there are no stackable wrappers yet:

Why not just use the already existing apt-get-update-plus ?

It could lead to not upgrading the kernel when users use regular APT instead of apt-get-update-plus .

---

Another idea:
repurpose uwt from “use with Tor” to “ultimate wrapper tool”.
uwt adding stream isolation would just be one feature. Hooking apt for kernel upgrades another feature.

[madaidan]

@Patrick

Yes, unless something speaks against it. The idea is: smaller file size. Easier to review. And keeping auto generated code out of source code files.

It might be easier to review with those # X is not needed lines as it allows someone to grep the config for a certain option to see if it is set or not. If it isn’t there, they might get confused and think that option doesn’t exist when in reality, we just removed those lines.

@Patrick

No outdated packages.

I’m not sure what the problem is then. I tested compilation just last night and it worked fine. Maybe it’s a missing dependency somewhere?

To debug, try strace make deb-pkg (requires kernel.yama.ptrace_scope=0 with sysctl first) and see if you can find any errors.

@Patrick

It could lead to not upgrading the kernel when users use regular APT instead of apt-get-update-plus .

Aren’t users instructed to use apt-get-update-plus? Why should any other way of updating be supported?

@Patrick

Another idea:
repurpose uwt from “use with Tor” to “ultimate wrapper tool”.
uwt adding stream isolation would just be one feature. Hooking apt for kernel upgrades another feature.

This seems like an excellent idea. Especially considering apt is already using uwt.

[HulaHoop]

@Patrick

Not Whonix specific indeed. Would be generic. Is that what you meant?

Good to know. We can apply it to kicksecure too. What I meant is that we make wapt usage an exception applied to select packages instead of the normal go-to command for installing/updating anything?

@Patrick

Another idea:
repurpose uwt from “use with Tor” to “ultimate wrapper tool”.
uwt adding stream isolation would just be one feature. Hooking apt for kernel upgrades another feature.

Makes sense - uwt could act as the backend while still invoking wrapped apt with the “wapt” name

[HulaHoop]

@Patrick

Which processor[s]?

While compiling you can create customized binary for your processor family at expense of portability/compatibility with other x64 machines. Only should be done if everyone’s building their own thing. IMHO if we want this to be as widely used as possible it must be done on the dev machine (ideally usingDebian’s reproducible toolchain for assurance) and maintain updated uploads to our repos.

[madaidan]

More hardening

[madaidan]

Disable ftrace

Whenever making a change to the config, it’s best to run make menuconfig and save (don’t set any options) so it disables/deletes any other options that require the thing you changed. That’s why in my pull requests you see other options being changed (e.g. disabling ftrace wipes out a load tracing options as they depend on ftrace).

[Patrick]

What about make oldconfig rather than make menuconfig? Something non-interactive?

[HulaHoop]

@Patrick

What about make oldconfig rather than make menuconfig? Something non-interactive?

Yeah totally forgot about that but this is how you go with reusing a config for a built kernel with the source of the new.

If you want it to be non interactive run

yes "" | make oldconfig

What does "make oldconfig" do exactly in the Linux kernel makefile?

[madaidan]

@Patrick

What about make oldconfig rather than make menuconfig? Something non-interactive?

Either should work fine.

[madaidan]

I just tested it and the linux-hardened patches do not work with the debian kernel sources.

We can probably modify the patch so it works for us.

[madaidan]

Using GitHub - a13xp0p0v/kconfig-hardened-check: A tool for checking the security hardening options of the Linux kernel, here are all the options we can change (excluding ones that don’t exist):

              option name               | desired val | decision |       reason       ||        check result        
====================================================================================================================
CONFIG_MODULE_SIG_ALL                   |      y      |   kspp   |  self_protection   ||     FAIL: "is not set"     
CONFIG_MODULE_SIG_SHA512                |      y      |   kspp   |  self_protection   ||     FAIL: "is not set"     
CONFIG_MODULE_SIG_FORCE                 |      y      |   kspp   |  self_protection   ||     FAIL: "is not set"     
CONFIG_DEBUG_VIRTUAL                    |      y      |  clipos  |  self_protection   ||     FAIL: "is not set"     
CONFIG_STATIC_USERMODEHELPER            |      y      |  clipos  |  self_protection   ||     FAIL: "is not set"     
CONFIG_RANDOM_TRUST_CPU                 | is not set  |  clipos  |  self_protection   ||         FAIL: "y"          
CONFIG_INTEL_IOMMU_DEFAULT_ON           |      y      |  clipos  |  self_protection   ||     FAIL: "is not set"     
CONFIG_SLUB_DEBUG_ON                    |      y      |    my    |  self_protection   ||     FAIL: "is not set"     
CONFIG_SECURITY_LOADPIN                 |      y      |    my    |  self_protection   ||     FAIL: "is not set"     
CONFIG_PAGE_POISONING_ZERO              | is not set  |    my    |  self_protection   ||         FAIL: "y"          
CONFIG_MODULES                          | is not set  |   kspp   | cut_attack_surface ||         FAIL: "y"          
CONFIG_DEVMEM                           | is not set  |   kspp   | cut_attack_surface ||         FAIL: "y"          
CONFIG_BINFMT_MISC                      | is not set  |   kspp   | cut_attack_surface ||         FAIL: "m"          
CONFIG_INET_DIAG                        | is not set  |   kspp   | cut_attack_surface ||         FAIL: "m"          
CONFIG_PROC_PAGE_MONITOR                | is not set  |grsecurity| cut_attack_surface ||         FAIL: "y"          
CONFIG_USELIB                           | is not set  |grsecurity| cut_attack_surface ||         FAIL: "y"          
CONFIG_CHECKPOINT_RESTORE               | is not set  |grsecurity| cut_attack_surface ||         FAIL: "y"          
CONFIG_MEM_SOFT_DIRTY                   | is not set  |grsecurity| cut_attack_surface ||         FAIL: "y"          
CONFIG_DEBUG_FS                         | is not set  |grsecurity| cut_attack_surface ||         FAIL: "y"          
CONFIG_NOTIFIER_ERROR_INJECTION         | is not set  |grsecurity| cut_attack_surface ||         FAIL: "m"          
CONFIG_ACPI_TABLE_UPGRADE               | is not set  | lockdown | cut_attack_surface ||         FAIL: "y"          
CONFIG_PROFILING                        | is not set  | lockdown | cut_attack_surface ||         FAIL: "y"          
CONFIG_BPF_SYSCALL                      | is not set  | lockdown | cut_attack_surface ||         FAIL: "y"          
CONFIG_KSM                              | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_KALLSYMS                         | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_X86_VSYSCALL_EMULATION           | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_MAGIC_SYSRQ                      | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_USER_NS                          | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_LDISC_AUTOLOAD                   | is not set  |  clipos  | cut_attack_surface ||         FAIL: "y"          
CONFIG_IP_SCTP                          | is not set  |    my    | cut_attack_surface ||         FAIL: "m"          
CONFIG_BPF_JIT                          | is not set  |    my    | cut_attack_surface ||         FAIL: "y"

Some of these aren’t that important e.g. we already set CONFIG_STRICT_DEVMEM so we don’t really need to disable CONFIG_DEVMEM.

We can also disable CONFIG_AIO as per https://github.com/AndroidHardeningArchive/documentation/blob/master/technical_overview.md#attack-surface-reduction

[anontor]

@madaidan
That is a really impressive list of config options! It took me a while but I got through all of them. it looks like you covered all bases.
One suggestion I thought of was a boot parameter dealing with how the initial entropy is handled.
Since systemd (as of Debian Buster) no longer will use data from the previous session for boot time entropy, now the kernel by default relies on the hardware device in the machine (usually RDRAND or similar) for initial seeding. There is a kernel parameter: random.trust_cpu=off that would use the old method (pre Buster kernel default).
Is this something that would matter from a security perspective? I know that there are several sources of entropy besides rdrand that the machine uses together to fill the random pool. When different entropy sources are mixed and hashed, their individual sources do not matter as much as far as hurting secure operations like key generation and such. So if the initial seeding relies on RDRAND only, does that matter really? I’m not really sure