kernel recompilation for better hardening

I think apparmor-profile-everything is better suited. Trading off simplicity [1] against higher compile times.


[1] Avoiding complexity. apparmor-profile-everything seems simpler to understand, review, customize, test, etc. than any kernel compilation changes.

1 Like

LTS Linux kernel: good.

But what about linux-hardened? These patches target LTS Linux kernel. Good. But how stable are linux-hardened patches? Are these maintenance only for LTS kernels while newer security features / bigger changes happen for not-yet stable/LTS kernels only? In other words: does linux-hardened introduce new security features for LTS kernels?

These are exactly the thing which I am worried about here. A change by linux-hardened which makes a VM or machine unbootable due to the change.

Tested by whom and in which environments? Host only? Inside VMs? VirtualBox, KVM, Qubes? With hardened-kernel config? On Debian, which suite, buster? Inside Whonix? With apparmor-profile-everything? All of these components (virtualizer, apparmor, Debian, Whonix) add complexity and can result in non-bootable systems.

What I want to prevent here is breaking Whonix Stable Version User Experience.

I.e. a standard apt (kernel) upgrade resulting in breaking boot, connectivity, graphical desktop, apt-get or require manual fixes being applied.

I don’t think this is possible with APT.
APT can show any error messages, yes. But how can APT react? It can only either:

  • leave the system in a non-broken state (i.e. can re-run APT / upgrade / install any package), which is easily overlooked and cannot be easily automated (introducing a special case) or,
  • in a broken state (demanding the previous upgrade to finish, strange/hard to fix [users don’t know] error messages).

Once a package upgrade is done, there is no way to say “oh well, its actually not done, please upgrade again”.

I did not talk about gpg verification of the html with the version number yet. tb-updater has the same issue. Upstream won’t provide gpg signatures for RecommendedTBBVersions. Therefore there is an interactive Download Confirmation Notification. APT interactive is bad. To automate the build process and keep Qubes (specifically Disp)VMs updated these version numbers are hardcoded:

(With Tor Browser it is less severe since its internal updater is considered secure and reliable.)

With gpg verification is a major hassle and security risk. I was referring this this:

curl_opts="--tlsv1.2 --proto =https --location --remote-name-all --remote-header-name"

## TODO: do not use networking as per https://forums.whonix.org/t/kernel-recompilation-for-better-hardening/7598/214

curl $curl_opts "${kernel_source}" -o "${working_folder}/linux-${version}.tar.xz"

curl $curl_opts "${linux_hardened_patch}" -o "${working_folder}/linux-hardened-${version}.a.patch"

The System Security Level development goal for the Whonix software for source code and binary downloads by default is always software signatures verification security level.

If I can maintain an up to date version file on the server, why couldn’t I also maintain a deb package in Whonix APT repository? Then no networking outside of APT needs to be invented.

The problem with the tb-updater package currently is it mixes:

  • simple things such as hardcoded version numbers with
  • code changes that need the usual development process (repository: developers → testers → stable-proposed-updates → stable)

Therefore hardcoded version numbers cannot be easily updated in all repositories as soon as possible. A mistake to not repeat here.

There might also be the misconception about the security that can be expected from the whonix.org server. To clear that up, this chapter was expanded just now:
Placing Trust in Whonix

I should have said superroot.
This is always the case when there is write access. Either users are required to edit the script directly to use custom/testing versions which is harder and doesn’t stop an attacker or things are configurable through a configuration file or command line parameters. The latter is easier for users but doesn’t make an attack easier for an attacker. Since an attacker who has the skills for the attack, wouldn’t have an issue modifying the script directly either vs a config.

2 Likes

New security features aren’t actively being developed much anyway. I’d imagine they’d be ported to LTS kernels though.

The standard kernel has tons of testers due to being one of the most popular open source projects. It’d be tested in a whole load of environments.

The linux-hardened patch are likely tested by anthraxx, the maintainer. Those patches are also used in Arch Linux’s linux-hardened kernel and ClipOS.

With our config and apparmor-profile-everything though, only we will be testing it although the chance a newer kernel will result in these making the system unbootable is low.

Can’t we just do something like:

if ! curl ...; then
  echo "Couldn't connect. Fix your connection and retry."
  exit
fi

?

You’d still need to manually update the version numbers. Whereas with a cron job automatically updating the text file, you won’t need to do anything manually.

2 Likes

It’s possible but then APT would exit 0 and not indicate that the upgrade did not actually finish. (Except for the error message text.) Re-running apt-get dist-upgrade wouldn’t do anything either since the new package is already installed. From perspective of APT, there is nothing more to do. Users would have to read that error message during APT and then act accordingly by running commands manually to finish the upgrade.

Why would there be a new package for APT to upgrade to anyhow? Because a new package was uploaded to Whonix repository. But if a new package is uploaded, why doesn’t it already come with all that is needed, i.e. compilation script, kernel sources and linux-hardend patch?

Need to manually update version: yes
But how does a new package get uploaded to Whonix repository so it will result in a package upgrade? That’s a manual process too.
And if the kernel patch version number doesn’t get manually bumped in package source code, why would the package version number be bumped, the package re-build and uploaded to Whonix repository?

The question is how we inform the system of the user that a kernel upgrade is available? Usually the user notices when running APT upgrades since there is a new package. Ideally we can just use APT here too. Everything else expands into a lot complexity similar to inventing a new package manager. If that is required, better to look if any existing package manager would be suitable than inventing a new one. guix was discussed in past.

The idea is kinda “something should be updated faster than package upgrades are coming from Whonix repository”? Well, that would require a whole mechanism to check for updates, download updates, compile and install. Foreground or background…

For now I think it’s best to just add the kernel archive [1] and linux-hardened patch [1] to hardened-kernel package.

For those who want to use different / newer versions than whatever the hardened-kernel shipped, there can be superroot command line options, config files and/or replacing archive files on disk.


Automated download and gpg verification is an absolute mess:

  • endless data attacks (fill up user disk, crash system, could just be misconfigured web server)
  • rollback attacks (need to cache version numbers, remember, never downgrade)
  • infinite freeze attacks (always reporting outdated version to (targeted) users)
  • security risks of parsing strings
  • gpg signs and verifies files. Not file names. While a signature might be valid for the actual file, so would any signature be valid by the same white listed signing key owner that signed any other file)
  • key management (upstream keys expired, cycled, revoked)
  • signature freshness needs to be checked

Metadata of package managers needs to be signed, verified and fresh (valid-until). These are old, known attacks but not much inside security researchers consciousness. References:

It is all solvable in code but some thing which looks simple at first expands into a lot complexity.


[1] Plus gpg signatures. These would not be automatically checked by the compilation script [2] but are useful for auditors to manually check the hardened-kernel package source code. Signatures would be verified manually whenever developers add new archives to the package source code.
[2] At the time the compilation script which is installed and executed is trusted anyhow.

1 Like

https://github.com/clipos/bugs/issues/38

2 Likes

Good points. The problem now is we’d need to keep a very close eye on new releases to update the kernel quickly. There is already a new release from within 3 days of the last one https://github.com/anthraxx/linux-hardened/releases/download/4.19.94.a/linux-hardened-4.19.94.a.patch

Might be too fast to keep up.

1 Like

Are these fixing CVEs (similar to Debian kernel upgrades) (very time critical) or “only” more security hardening" (less time critical)?

2 Likes

The linux-hardened patches follow kernel releases. There was a 4.19.94 kernel release so linux-hardened also made a new release.

There’s been no new hardening since July https://github.com/anthraxx/linux-hardened/commits/4.19-lts so it is likely security fixes from upstream linux.

There’s been a 4.19.95 kernel release very recently so linux-hardened will likely make a new release very soon https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.19.95.tar.xz

2 Likes

https://github.com/Whonix/hardened-kernel/pull/22

https://github.com/Whonix/hardened-kernel/pull/23

1 Like

https://github.com/Whonix/hardened-kernel/pull/24

1 Like

Tested? I know from kernel module load hardening experiments that inability to auto load binfmt module broke XFCE startup.

Why needed if we already have that in security-misc package?

1 Like

I haven’t tested it yet but will later. I don’t see why XFCE would require custom binary formats though.

security-misc enables it after X starts which isn’t as good as having it enabled all the time.

https://github.com/Whonix/hardened-kernel/pull/25

https://github.com/Whonix/hardened-kernel/pull/26

madaidan via Whonix Forum:

Disable profiling support by madaidan · Pull Request #25 · Kicksecure/hardened-kernel · GitHub

Merged.

1 Like

madaidan via Whonix Forum:

security-misc enables it after X starts which isn’t as good as having it enabled all the time.

I see. Maybe better add this as a sysctl, in a package that gets only
installed inside VMs? (then it would also be enabled in initramfs
already) Reason is that this would be way more easy to disable / debug /
port.

Do we have other (security) settings too which are VM-only?

Do we have other (security) settings too which are host-only?

Also we yet have to create security-paranoid(opt-in package for settings
which are too experimental for security-misc default package?

1 Like
1 Like

hardened-kernel 0.8-1 on Non-Qubes-Whonix Whonix-Gateway and Whonix-Workstation.

DKMS does not run during linux kernel package upgrade. Therefore after reboot LKRG and tirdad no longer functions.

Jan 15 13:36:06 host modprobe[213]: modprobe: ERROR: could not insert ‘p_lkrg’: Exec format error

Workaround until DKMS will be properly run after kernel package upgrade:

sudo apt install --reinstall lkrg-dkms tirdad-dkms

Tirdad is fixed after that procedure.

DKMS might not run since the package is installed directly by dpkg and not through APT?

Next issue after reboot:

sudo modprobe p_lkrg

modprobe: ERROR: could not insert ‘p_lkrg’: No buffer space available

1 Like

Users can set kernel.panic_on_oops=0 if they need to disable it.

Not really.

That isn’t really needed since we can just add those settings to security-misc but make them opt-in.