https://github.com/Whonix/lkrg/commit/99439fe2d95f5f588f4e3d3f15842e336da07acc
1 Like
A blocker for installation onf LKRG by default in Non-Qubes-Whonix Whonix-Gateway:
I experienced the VM freezing during LKRG installation. Freezes GUI (seconds counter at the clock freezes and will never move on again.) Happening in a VirtualBox VM during LKRG DKMS compilation with 768 MB RAM and 512 MB swapfile. Not always reproducible. I provided various SysRq output to LKRG upstream in private e-mail. Not sharing here as it might contain sensitive information about my system.
While swap gets used, swap does not run out of memory. Check in another console tab using:
while true ; do free -m && sleep .2 ; done
Gathered the log using a serial console. Journal does not have anything interesting.
This doesn’t happen when increasing RAM. Whonix-Gateway VirtualBox by default comes with 768 MB RAM and a 512 MB encrypted swapfile. Whonix-Workstation VirtualBox by default comes 2048 MB RAM and no swapfile. (By adding >= 2000 MB RAM to Whonix-Gateway, no swap file will be created by swap-file-creator (Whonix github project). I’d very much like to increase default RAM of Whonix-Gateway to 2048 MB RAM but that won’t work out of the box for many of Whonix users. Let’s say someone has host system with 4 GB RAM which is quite common. Hard to run two VMs there. Kernel Samepage Merging (KSM) might be available for VirtualBox too, but there were some security issues with it. Some references.
LKRG compilation should be possible even though swap is required? Maybe LKRG isn’t the cause but only the trigger? 768 MB RAM for a Debian VM with XFCE might be too little RAM nowadays?
It could be this bug - a long standing kernel bug related to system stress and swap:
https://bugzilla.kernel.org/show_bug.cgi?id=196729
A bug which might not be fixed anytime soon. That would be a blocker for Non-Qubes-Whonix Whonix-Gateway LKRG installation by default. Then LKRG could could be installed by default in Non-Qubes-Whonix Whonix-Workstation. (LKRG in Qubes-Whonix has its own blockers for installation by default.)
Maybe another hack could be to install the LKRG package by default but only attempt to actually compile it if enough RAM is available so that it would at least be usable by those who have the system resources to assign sufficient RAM.
2 Likes
2 Likes
Maybe a memory leak? So this amount of RAM is only needed during compilation? After that do you run into problems when it is lowered back again?
Excuse my ignorance, but couldn’t we pre-compile during image builds and provide it as a binary? Also as binary updates in Whonix repo?
Possibly. So even if https://bugzilla.kernel.org/show_bug.cgi?id=196729 or similar is the issue, it was still worth contacting upstream.
Indeed.
That would break on kernel upgrade (comes from packages.debian.org) or when using a custom kernel. That’s why DKMS was invented. There are many kernel modules such as wireguard or guest additions. Not sure that’s a leap of faith but if DKMS (kernel module compilation during installation) was easily avoidable, I guess that would be standard practice in Debian.
2 Likes
LKRG is not yet compatible with hardened kernel.
Not worth contacting upstream about it yet. I am still waiting for upstream to provide signed git commits so I can test a newer version before more experimentation is worthwhile.
2 Likes
Upstream providing signed git commits now.
LKRG pacakge in Whonix git master and developers repository updated as per:
LKRG version commit 403f2fa92cdb8071a39afa031f08719972aa563e
Date: Thu Jan 16 06:05:15 2020 +0000
Commit message: Fix compilation on non-x86 platforms for kernel 5.3+
Fixed. Reported. And found maybe another (smaller) issue:
2 Likes
VirtualBox host support is now on the horizon:
https://www.openwall.com/lists/lkrg-users/2020/01/24/1
1 Like
As for LKRG loading it as early as possible is non-trivial. Currently loading LKRG as early as possible would cause some issues. For example it would detect changes by tirdad as malicious and show a lot false positive messages. Asked upstream about it:
1 Like
The wiki page got noticed by spender.
1 Like
2 Likes
Above fixed.
2 Likes
LKRG is compatible with Debian Linux buster-backports 5.4.8-1~bpo10+1 kernel
A Debian issue but unrelated to LKRG:
linux-image-amd64 vs linux-headers-amd64 Debian buster-backports version mismatch bpo.2 vs bpo.3
2 Likes
- testing LKRG
- LKRG unfit for installation by default in Whonix / Kicksecure due to kernel boot console output - usability issue
Related: T950: prevent kernel info leaks
1 Like
Rebased on upstream master branch.
This was addressed by upstream:
lkrg-users - Re: LKRG unfit for installation by default in Whonix / Kicksecure due to kernel boot console output - usability issue
A bug was privately reported and upstream is on it already:
https://bitbucket.org/Adam_pi3/lkrg-main/pull-requests/4
Added a workaround (useful anyhow) in meanwhile:
2 Likes
Quote lkrg-users - rootkit detection
Date: Sun, 14 Jun 2020 17:37:59 +0200
From: Solar Designer <solar@…nwall.com>
To: lkrg-users@…ts.openwall.com
Subject: rootkit detectionHi,
Adam found this interesting Master’s Thesis of Juho Junnila, entitled
“Effectiveness of Linux Rootkit Detection Tools”:http://jultika.oulu.fi/files/nbnfioulu-202004201485.pdf
It shows LKRG as the most effective kernel rootkit detector (of those
tested), as long as LKRG is loaded before the rootkit. It also shows
LKRG sometimes detect rootkits even when LKRG is loaded after the
rootkit (we never intended that to work, but it sometimes does anyway,
which is OK). Finally, it shows that LKRG never detects pure userspace
rootkits (as expected; it would probably be a bug if it did).In particular, when LKRG is loaded before the rootkit, it detected 8 out
of 9 kernel rootkits tested: Diamorphine, Honey Pot Bears,
LilyOfTheValley, Nuk3 Gh0st, Puszek, Reptile, Rootfoo Linux Rootkit,
Sutekh. There were no false positives. No other tested rootkit
detector was anywhere close to LKRG’s effectiveness at detecting kernel
rootkits, with AIDE, OSSEC, and Rootkit Hunter detecting 2 out of 9
each, and Chkrootkit detecting none. These other detectors did,
however, detect some userspace rootkits, which LKRG doesn’t try to. The
combination of AIDE and LKRG is shown to be most effective, detecting 14
out of 15 rootkits total (both user and kernel space ones).None of the tools detected Keysniffer, which the thesis describes as:
“Keysniffer is an LKM that can be classified as a kernel mode rootkit
due to its kernel hooking capabilities. It is a keylogger that records
keyboard events to debugfs, an in-memory file system used for Linux
kernel debugging. It does not feature any particular anti-detection
mechanisms except that it prevents accessing the log file by non-root
users and the module is named “kisni.ko” by default to avoid immediate
suspicion when listing kernel modules. (Jana, 2019.) While Keysniffer
may not meet the criteria of all rootkit definitions due to its lack of
specific hiding functionalities, it does meet the criteria by Harley and
Lee (2007) and is thus included in this study.”While the above doesn’t say it, I think the non-detection of Keysniffer
by LKRG is due to it being too similar to non-malicious kernel modules.
The functionality described would conceivably be provided by a module
not intended for malicious use, and the module doesn’t need to mess with
the kernel’s integrity to provide this functionality. The non-detection
by other tools is probably because Keysniffer wasn’t popular enough, as
its signature-based detection is definitely practical.Of course, none of these rootkits tried to bypass LKRG. If they wanted
to, they could e.g. simply “rmmod p_lkrg” before loading themselves into
the kernel. (Or the attacker could do that manually.) In LKRG
development, we’re currently more focused on exploits run before the
attacker got legitimate-looking privileged access to the system. We’re
not as focused on rootkits, which are loaded with legitimate-looking
privileged access.That said, there are a few things we could do in the future to make it
more difficult and/or less reliable to bypass detection by LKRG-aware
rootkit loading. We could password-protect the module unloading and
sysctl interfaces of LKRG, and/or provide a way to disable those after
initial configuration. We could also provide setup instructions and/or
our own userspace components around LKRG where alerts would be sent to a
remote system, including alerts of LKRG being unloaded or reconfigured.Alexander
2 Likes
LKRG 0.8.1 is in all repositories since some time now.
https://www.openwall.com/presentations/OSTconf2020-LKRG-In-A-Nutshell/