kernel recompilation for better hardening

Should I also get rid of kernel modules entirely (unset CONFIG_MODULES)?

All needed modules can be built into the kernel directly by switching them to y options instead of m.

1 Like

Breaks virtualbox guest additions?

There are some interesting modules which I didn’t get to yet such as LKRG: LKRG - Linux Kernel Runtime Guard

Also another maybe upcoming kernel module related to kernel ISN (Add research idea for Linux TCP Initial Sequence Numbers may aid correlation (#16659) · Issues · Legacy / Trac · GitLab).

1 Like

We can likely compile those directly into the kernel rather than using loadable modules.

1 Like

Does this make things more fragile, likely to break? Speculate not many people built-in virtualbox guest addition modules or other modules.

(And host kernel including virtualbox host module build-in? If we want to maintain just one kernel (enough…) then built in virtualbox guest and host module?)

Do we gain security by using a kernel without module support compared to non-module support?

As of now, not all modules are load all the time. But without module load support, is everything load all the time?

Then we’d need a virtualbox and a non-virtualbox kernel? Or non-virtualbox VMs are just to bite the bullet of having a useless built-in module?

1 Like

Not sure.

This kernel won’t be able to be used on hosts. Otherwise we’d need to enable all hardware config options which would drastically increase the time needed to compile and add attack surface.

This is for VMs only. We can make a kernel for hosts if you want which would be separate from VMs but that’d take too long to compile and won’t be able to be compiled during package upgrades. It’d have to be pre-compiled.

Yes, it prevents loading any module to add attack surface or load a malicious module.

I think so. Not sure.

Yes. The built-in module doesn’t add much and shouldn’t be a problem for non-virtualbox VMs.

1 Like

In doubt lets start simple, see how things go and more hardening later? Otherwise, if someone experiences instability, it may be hard to pinpoint what the cause is.

That might actually be less secure then.

This will probably be contested.

Understood. Makes sense. Package name: hardened-vm-kernel or something like that? (“hardened vm kernel” might have some search results, hopefully we’re not reinventing here. I remember Ubuntu years ago had a VM specialized kernel.)

Yes.

Module loading requires root. “trusted” root. But root could as well as modify the kernel on the disk so next reboot something else will be booted? This would be far more useful if at reboot there was an integrity check which would refuse to boot any tampered kernel images. Related to verified boot:

As an alternative to make all modules built-in rather than dynamically load… What about loading all modules at early boot time and then disabling module loading? Would that be as good as disabling module loading at kernel compile time?

1 Like

Sure. I was only giving ideas.

Only the needed modules will be compiled so it might not be.

hardened-vm-kernel sounds good. I don’t think we’re reinventing. I’ve never heard of a hardened kernel designed for virtual machines before that isn’t grsec.

I think we should still protect the kernel from the root user.

We can fix that with the system wide MAC policy or verified boot.

Not sure. Needs more research.

1 Like

Would RTC and audio be needed in VMs?

I can remove both of them to reduce compilation time.

1 Like

Maybe not hardened but specialized kernels for VM did and do exist.

But Ubuntu thing was a thing of 2013 or earlier. Might have been a whole distribution flavor only for virtual machines.

But System wide MAC policy or verified boot would also prevent modprobe anyway? Hence no extra advantage by disabling module loading during compile time?

audio: workstation yes, gateway no.

RTC: Not sure.

https://github.com/Whonix/Whonix/blob/master/build-steps.d/2600_create-vbox-vm#L103-L107

We need a hardware clock for sure? Otherwise when VM would start it would not have any clock except perhaps some hardcoded time/date information at all? But not necessarily an RTC? Dunno if VirtualBox provides an RTC and a hardware clock or only an hardware clock which is also an RTC. Search term:

site:virtualbox.org real time clock
site:virtualbox.org RTC

Haven’t found something relevant yet.

I haven’t found any “relevant” disadvantages of not having an RTC yet. Only of not having no hardware clock at all.

Could just try if anything breaks without RTC?

Best tested in non-Whonix / non-Kicksecure, plain Debian VM. At boot, time without help of any network synchronization daemons need to be somewhat correct. Shouldn’t rely on internet to fetch. Without any initial time, it’s hard/impossible to securely fetch time from remote. Opens up for many attacks as per:

1 Like

Could you please (based on above perhaps) provide instructions on how to compile the kernel with your custom kernel config? That would allow me and perhaps others to test this sooner rather than later.

1 Like

IMO we should limit the custom kernel to the WS as the risk of leaks would be too great if we applied it to the GW too. We simply don’t know what every module does or how they interact with each other to deliver the same leak protection we have now.

No untrusted software should be running there so there’s no danger of some disabled module being enabled without users’ consent.

2 Likes

I think the kernel config package should be replicated once for each different hypervisor to keep things clean and simple.

There’s no reason to have modules relevant to VBox and Xen enabled for KVM users and vice versa. If we’re about decreasing attack surface let’s go all the way.

@madaidan How active is Linux-Hardened upstream? How much can we rely on it’s author to keep maintaining his fork until KSPP merges the missing features? There’s nothing worse than doing a bunch of effort based on someone’s dead code.

Do the patches work with Debian vanilla source tarball? We should not base on kernel.org’s code.

2 Likes

The MAC policy can prevent module loading but currently it doesn’t. Verified boot doesn’t disable module loading.

  1. Install needed packages:

    sudo apt-get install linux-source build-essential libssl-dev libncurses-dev fakeroot libelf-dev
    
  2. Create directory ~/kernel/ and switch to it

mkdir -p ~/kernel && cd ~/kernel

  1. Extract the kernel source:

    tar -xaf /usr/src/linux-source-4.19.tar.xz
    
  2. Switch to the ~/kernel/linux-source-4.19/ directory.

cd ~/kernel/linux-source-4.19/

  1. Apply the debian kernel patches:

    xzcat /usr/src/linux-patch-4.19-rt.patch.xz | patch -p1
    
  2. Create the file .config and add the config I sent above. kernel recompilation for better hardening - #44 by madaidan

  3. Run make menuconfig, load the .config file and save it. This is needed as my config doesn’t include a whole bunch of # CONFIG_X is not set lines for simplicity but those are needed for compiling.

  4. Compile it as a Debian package:

    make deb-pkg
    
  5. Install the kernel

    sudo dpkg -i ../linux-libc-dev_4.19.67-rt24-1_amd64.deb
    sudo dpkg -i ../linux-headers-4.19.67-rt24_4.19.67-rt24-1_amd64.deb
    sudo dpkg -i ../linux-image-4.19.67-rt24_4.19.67-rt24-1_amd64.deb
    

Are you talking about including every module directly in the kernel rather than using loadable modules? That probably isn’t going to actually be done. It was just an idea. Currently loadable modules are still a thing in the config.

IMO it’d be simpler to just create one config. There are very little vbox/xen/kvm specific options. The different configs would only have a difference of about ~10 lines.

It’s not that active. I’ve spoken with the devs of it and it’s kinda dead due to lack of interest. New features aren’t exactly being developed. It mostly just adds stricter defaults than the vanilla kernel.

There are some important patches in the linux-hardened kernel that aren’t in vanilla though.

It’d be a long time before the KSPP merges all the missing features upstream. One of the main points of linux-hardened was to accept KSPP patches that weren’t accepted upstream.

They should work fine. I haven’t tested them though.

2 Likes

Module or built-in is besides the point. What I mean is: no vulnerable code is loaded by the GW and there is no way it can be so making a custom slimmed down kernel is not necessary to shrink attack surface.

I see.

I see.

2 Likes

Sounds good in theory but I vary the maintenance overhead.

Then let’s use MAC policy to prevent module loading so we don’t have to disable module support in kernel? Seems like a more popular, tested path rather than disabling module loading. Therefore should hopefully be more stable.

Any way to make that non-interactive? Wouldn’t know what options to choose. Just nothing?, go with defaults? Non-interactive also needed for later automation.

2 Likes

error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel

Installing libelf-dev helped. Edited your post. Compiling.

2 Likes

Compilation with your config was successful but took ~ 1 hour in a Qubes VM. Did not test yet if the kernel boots. Due to long compile time, sounds more like a package to be added to Whonix repository?

1 Like

I’m not sure what you mean. How would slimming the kernel config make the GW load vulnerable code?

It would be a better approach although we’d need to create a systemd-modules-load profile so necessary modules can still be loaded at boot.

make menuconfig generates a new .config file with all the # X is not needed lines that we can use for the next time. I just use the config without all those lines for working on it.

You don’t need to chose any options. They’re all already chosen.

It took around an hour for me too. The config can still be slimmed down further and hopefully we can get it to compile faster.

1 Like

Quote BuildADebianKernelPackage - Debian Wiki

Using your current Debian kernel configuration as a starting point

Alternatively, you can use the configuration from a Debian-built kernel that you already have installed by copying the /boot/config-* file to .config and then running make oldconfig to only answer new questions.

When you’re ready… Could you please create a package hardened-vm-config (or so)… Then first add/commit the Debian default .config. Then another commit to make the changes? This is to illustrate in how far our config is different from Debian config. If that makes sense? Or compare with some other starting point?

1 Like

Let’s say an iptables module that is activated when some ruleset is triggered is not loaded by default and only happens when a some network pattern is detected. In a normal kernel it will be there, however with a slimmed down kernel where we haven’t tested every possible behavior/codepath it won’t and there is the hypothetical possibility of failing open.

Also we want to transition to nftables in the medium future and that requires a different set of modules.

You want to make it use all available cores if not already.

2 Likes