kernel recompilation for better hardening

Patrick · December 21, 2020, 11:10am

CI build failing.

madaidan · December 21, 2020, 7:42pm

I can’t tell what’s causing this. Can you?

Patrick · December 21, 2020, 8:47pm

Happening when using --debug with environment variable CI=true being set.

+ true 'Sanity test. '\''make oldconfig'\'' should not modify '\''.config'\''.'
+ true https://forums.whonix.org/t/kernel-recompilation-for-better-hardening/7598/317
+ make oldconfig -C /var/lib/hardened-kernel/hardened-vm-kernel/linux-4.19.122/
...
+ diff /home/travis/build/Whonix/hardened-kernel/usr/share/hardened-kernel/hardened-vm-kernel /var/lib/hardened-kernel/hardened-vm-kernel/linux-4.19.122//.config

diff then exits non-zero.

See the diff.

diff /home/travis/build/Whonix/hardened-kernel/usr/share/hardened-kernel/hardened-vm-kernel /var/lib/hardened-kernel/hardened-vm-kernel/linux-4.19.122//.config

.

> # CONFIG_GENERIC_IRQ_DEBUGFS is not set
286a288
> # CONFIG_IOSF_MBI_DEBUG is not set
292a295
> # CONFIG_QUEUED_LOCK_STAT is not set
300a304
> # CONFIG_XEN_DEBUG_FS is not set
302a307
> # CONFIG_KVM_DEBUG_FS is not set
344a350
> # CONFIG_X86_MCE_INJECT is not set
488a495
> # CONFIG_ACPI_CUSTOM_METHOD is not set
496a504
> # CONFIG_ACPI_APEI_EINJ is not set
685c693
< # CONFIG_KPROBES is not set
---
> CONFIG_KPROBES=y
687a696
> CONFIG_OPTPROBES=y
689a699
> CONFIG_KRETPROBES=y
758a769
> # CONFIG_GCOV_KERNEL is not set
802a814,815
> CONFIG_BLK_DEBUG_FS=y
> CONFIG_BLK_DEBUG_FS_ZONED=y
1500a1514
> # CONFIG_ZRAM_MEMORY_TRACKING is not set
2079a2094
> # CONFIG_NETDEVSIM is not set
3718a3734
> # CONFIG_SW_SYNC is not set
3895a3912
> # CONFIG_IOMMU_DEBUGFS is not set
3989a4007
> # CONFIG_RAS_CEC is not set
4019a4038
> # CONFIG_INTEL_TH_DEBUG is not set
4616a4636
> # CONFIG_DYNAMIC_DEBUG is not set
4632c4652
< # CONFIG_DEBUG_FS is not set
---
> CONFIG_DEBUG_FS=y
4733a4754
> CONFIG_FUNCTION_ERROR_INJECTION=y
4749a4771
> # CONFIG_LKDTM is not set
4751a4774
> # CONFIG_KPROBES_SANITY_TEST is not set
4808a4832
> # CONFIG_DEBUG_BOOT_PARAMS is not set

That should not be happening.

Running

make oldconfig -C "$extracted_linux_kernel_sources_folder"

should not be modifying "$extracted_linux_kernel_sources_folder/.config" "${MYDIR}/${kernel_config}" should already be in a state where a later make oldconfig would do nothing. That is to make the process “less magic”. Less auto generated. Easier to review. More predictable. Allow reviewers to review to full kernel configuration. No implicit filling the blanks during compilation. Solution? Run make oldconfig yourself, review, and update config in git. Make make oldconfig out of work.

Build history reveals that this is only happening since Fix issue with compiling host kernel by madaidan · Pull Request #55 · Kicksecure/hardened-kernel · GitHub was merged.

But I don’t see how that pull request could have caused that since it’s opt in and not used on CI.

madaidan · December 21, 2020, 9:02pm

The sanity check runs after the kprobes/ftrace configs are appended: hardened-kernel/usr/share/hardened-kernel/build at 629b62475455ce5ef073e6ba8b970ea76ade88a0 · Kicksecure/hardened-kernel · GitHub

make oldconfig will then shift those configs about to put them in the correct order in the full file. The sanity test must run before those options are added.

The build for that passed though? It seems to fail because of Optionally enable kprobes/ftrace for LKRG support · Kicksecure/hardened-kernel@629b624 · GitHub

Patrick · December 22, 2020, 1:10pm

Indeed.

That sanity test runs if [ "$debug" = "true" ]; then. It is nested in another IF. if [ "$CI" = "true" ]; then. Btw I don’t see why that should be nested. (Perhaps was implemented this way because CI build uses --debug anyhow.)

Could run that sanity test outside of if [ "$debug" = "true" ]; then and earlier. I’ll try that now.

No. On Optionally enable kprobes/ftrace for LKRG support · Kicksecure/hardened-kernel@629b624 · GitHub the red X indicates a failed Travis CI build. I should have noticed this earlier.

Seems like. But I understand how it can be the cause since it added an opt-in. Didn’t changed actual execution on CI.

Patrick · December 22, 2020, 1:17pm

Now I get why this was happening on CI. --debug results in kprobes="true".

That CI issue seems now fixed. Build hasn’t completed yet but it’s past that issue.

Another thing…

if [ "${kprobes}" = "true" ]; then
  cat "${MYDIR}/kprobes-ftrace" >> "$extracted_linux_kernel_sources_folder/.config"
fi

We don’t run make oldconfig -C "$extracted_linux_kernel_sources_folder".

vs

   cat "${MYDIR}/debugging-config" >> "$extracted_linux_kernel_sources_folder/.config"
   if [ "${kernel_config}" = "hardened-host-kernel" ]; then
     cat "${MYDIR}/debugging-config-host" >> "$extracted_linux_kernel_sources_folder/.config"
   fi
   make oldconfig -C "$extracted_linux_kernel_sources_folder"

We do run make oldconfig -C "$extracted_linux_kernel_sources_folder".

Does that look alright?

madaidan · December 22, 2020, 7:08pm

It shouldn’t be too important but running make oldconfig wouldn’t hurt. It’ll rearrange the file properly before build but this isn’t required.

madaidan · January 30, 2021, 1:36am

madaidan · January 30, 2021, 9:38pm

linux-hardened’s FORTIFY_SOURCE_STRICT_STRING has been implemented upstream.

github.com/torvalds/linux

lib: string.h: detect intra-object overflow in fortified string functions

committed 06:46AM - 16 Dec 20 UTC

daxtens

+16 -11

Patch series "Fortify strscpy()", v7. This patch implements a fortified version… of strscpy() enabled by setting CONFIG_FORTIFY_SOURCE=y. The new version ensures the following before calling vanilla strscpy(): 1. There is no read overflow because either size is smaller than src length or we shrink size to src length by calling fortified strnlen(). 2. There is no write overflow because we either failed during compilation or at runtime by checking that size is smaller than dest size. Note that, if src and dst size cannot be got, the patch defaults to call vanilla strscpy(). The patches adds the following: 1. Implement the fortified version of strscpy(). 2. Add a new LKDTM test to ensures the fortified version still returns the same value as the vanilla one while panic'ing when there is a write overflow. 3. Correct some typos in LKDTM related file. I based my modifications on top of two patches from Daniel Axtens which modify calls to __builtin_object_size, in fortified string functions, to ensure the true size of char * are returned and not the surrounding structure size. About performance, I measured the slow down of fortified strscpy(), using the vanilla one as baseline. The hardware I used is an Intel i3 2130 CPU clocked at 3.4 GHz. I ran "Linux 5.10.0-rc4+ SMP PREEMPT" inside qemu 3.10 with 4 CPU cores. The following code, called through LKDTM, was used as a benchmark: #define TIMES 10000 char *src; char dst[7]; int i; ktime_t begin; src = kstrdup("foobar", GFP_KERNEL); if (src == NULL) return; begin = ktime_get(); for (i = 0; i < TIMES; i++) strscpy(dst, src, strlen(src)); pr_info("%d fortified strscpy() tooks %lld", TIMES, ktime_get() - begin); begin = ktime_get(); for (i = 0; i < TIMES; i++) __real_strscpy(dst, src, strlen(src)); pr_info("%d vanilla strscpy() tooks %lld", TIMES, ktime_get() - begin); kfree(src); I called the above code 30 times to compute stats for each version (in ns, round to int): | version | mean | std | median | 95th | | --------- | ------- | ------ | ------- | ------- | | fortified | 245_069 | 54_657 | 216_230 | 331_122 | | vanilla | 172_501 | 70_281 | 143_539 | 219_553 | On average, fortified strscpy() is approximately 1.42 times slower than vanilla strscpy(). For the 95th percentile, the fortified version is about 1.50 times slower. So, clearly the stats are not in favor of fortified strscpy(). But, the fortified version loops the string twice (one in strnlen() and another in vanilla strscpy()) while the vanilla one only loops once. This can explain why fortified strscpy() is slower than the vanilla one. This patch (of 5): When the fortify feature was first introduced in commit 6974f0c4555e ("include/linux/string.h: add the option of fortified string.h functions"), Daniel Micay observed: * It should be possible to optionally use __builtin_object_size(x, 1) for some functions (C strings) to detect intra-object overflows (like glibc's _FORTIFY_SOURCE=2), but for now this takes the conservative approach to avoid likely compatibility issues. This is a case that often cannot be caught by KASAN. Consider: struct foo { char a[10]; char b[10]; } void test() { char *msg; struct foo foo; msg = kmalloc(16, GFP_KERNEL); strcpy(msg, "Hello world!!"); // this copy overwrites foo.b strcpy(foo.a, msg); } The questionable copy overflows foo.a and writes to foo.b as well. It cannot be detected by KASAN. Currently it is also not detected by fortify, because strcpy considers __builtin_object_size(x, 0), which considers the size of the surrounding object (here, struct foo). However, if we switch the string functions over to use __builtin_object_size(x, 1), the compiler will measure the size of the closest surrounding subobject (here, foo.a), rather than the size of the surrounding object as a whole. See https://gcc.gnu.org/onlinedocs/gcc/Object-Size-Checking.html for more info. Only do this for string functions: we cannot use it on things like memcpy, memmove, memcmp and memchr_inv due to code like this which purposefully operates on multiple structure members: (arch/x86/kernel/traps.c) /* * regs->sp points to the failing IRET frame on the * ESPFIX64 stack. Copy it to the entry stack. This fills * in gpregs->ss through gpregs->ip. * */ memmove(&gpregs->ip, (void *)regs->sp, 5*8); This change passes an allyesconfig on powerpc and x86, and an x86 kernel built with it survives running with syz-stress from syzkaller, so it seems safe so far. Link: https://lkml.kernel.org/r/20201122162451.27551-1-laniel_francis@privacyrequired.com Link: https://lkml.kernel.org/r/20201122162451.27551-2-laniel_francis@privacyrequired.com Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Francis Laniel <laniel_francis@privacyrequired.com> Reviewed-by: Kees Cook <keescook@chromium.org> Cc: Daniel Micay <danielmicay@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

HulaHoop · March 4, 2021, 10:43pm

CONFIG_KFENCE is a Low-Overhead Memory Safety Feature introduced in Linux 5.12 that is more competent than the KASAN. Used for uncovering similar out-of-bounds / use-after-free / invalid-free errors but with no overhead.

madaidan · March 12, 2021, 6:23pm

jonathanvlan · March 21, 2021, 7:54pm

hi a different function and greater security
some guys shared a topic : firejail / seccomp / More Options for Program Containment
there is a security patch Landlock : Landlock: unprivileged access control

CONFIG_SECURITY_LANDLOCK=y

tried a sandbox inside another sandbox ./landlock1 /bin/sh firejail *application

With the exploits and bypasses that there are lately,
I believe that greater security is necessary

Theory : Apparmor > Landlock > Firejail > Tor browser

Kernel Patch Hardening : https://patchwork.kernel.org/project/kernel-hardening/list/

madaidan · March 21, 2021, 8:32pm

Landlock currently isn’t in mainline; it’s only in linux-next: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Documentation/userspace-api/landlock.rst?h=next-20210319&id=f00397ee41c79b6155b9b44abd0055b2c0621349

When it does land in mainline, it will be a very long time before it reaches LTS and besides, I don’t see a significant advantage of this over AppArmor + bubblewrap. Landlock is intended for use cases in which the user does not have the adminstrator privileges necessary to configure traditional security policies like AppArmor which does not apply to us.

Combining a whole bunch of different access control mechanisms would be redundant — it’s the same kernel enforcing all of these policies and a single kernel exploit can bypass all of them. Unless Landlock exposes the ability to restrict something that isn’t already covered by our current approach, I don’t see an advantage.

madaidan · March 26, 2021, 11:49pm

HulaHoop · April 22, 2021, 1:12am

New CONFIG options in 5.9 we can enable:
https://outflux.net/blog/archives/2021/04/05/security-things-in-linux-v5-9/

Guide on configuring Yama’s ptrace restriction levels:

HulaHoop · May 13, 2021, 11:19pm

The Landlock LSM is now mainlined in 5.13. I wonder if it can be stacked wit the app sandbox we are working on since it too can restrict unprivileged apps.

HulaHoop · May 22, 2021, 5:34pm

Ryzen 3 has a PSF feature that causes spectre vulns. Trning it off is supported in 5.13, though it’s optional and won’t be a default.

madaidan · May 29, 2021, 3:46pm

See:

madaidan · June 4, 2021, 7:03pm

As per Consider Pulling in LKRG · Issue #408 · GrapheneOS/os-issue-tracker · GitHub, I believe we should get rid of the --lkrg option and enable CONFIG_KPROBES by default, while also disabling the config options mentioned in the issue to get rid of the attack surface exposed to user space.

Tirdad also requires kprobes:

madaidan · June 4, 2021, 8:02pm

Will also update documentation to reflect this.