man issues actually make this look really ugly during upgrading.
Processing triggers for man-db (2.8.5-2) ...
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
/usr/bin/mandb: zcat: Bad system call (core dumped)
Perhaps we config-package-dev displace man and use the hardened-malloc-disable wrapper?
Already using this:
Two more issues…
Shutdown of Non-Qubes-Whonix in VirtualBox without hardened-malloc-kicksecure was almost instant. Less than 5 seconds.
It now takes 30 seconds longer. I found clue on this by Enabling Persistent Systemd Journal Log and Checking Systemd Journal Log of Previous Boot.
The following command before (no hardened-malloc-kicksecure) were almost instant.
sudo systemctl stop swap-file-creator
(This happens during shutdown.)
It odes this:
sudo bash -x /usr/share/swap-file-creator/stop
Further tracked down to the following command which is now very slow:
sudo cryptsetup --batch-mode luksClose swapfile
Any idea how to fix this?
VirtualBox crash issue is back.
And LKRG is unrelated. Happens with and without. Will keep looking into and come back with log later.
Sounds good for now.
You could try using hardened-malloc-disable for cryptsetup?
I’m also looking into the Chromium issue now.
The Chromium issue appears to be a real bug in Debian’s old package. I’ve reported it here https://bugs.debian.org/971876
man is functional when using hardened-malloc-disable but console output still showing errors.
hardened-malloc-disable man nano
ERROR: ld.so: object ‘/usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so’ from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object ‘/usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so’ from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
hardened-malloc-disable chromium
Still not starting. Similar error message. hardened-malloc-disable seems to not fully disable hardened malloc?
I.e. newer versions of Chromium are functional? Then perhaps upstream Chromium already fixed it either knowingly or unknowingly (by refactoring / rewrite of the related code)?
sudo mandb -cq
(This is what happens during apt install / upgrade / dpkg mandb trigger.
mandb: zcat: Bad system call (core dumped)
tons of repetitions
Maybe it would be fixed by the man wrapper. Perhaps both, man and mandb need to be wrapped with hardened-malloc-disable.
We could add 2>/dev/null but that would hide other, valid errors.
It’s not to do with hardened_malloc. It’s to do with the sandbox.
You need to enable unprivileged user namespaces or run Chromium without a sandbox (the --no-sandbox flag) as bubblewrap disables the setuid sandbox.
Yes, most likely. We could try Debian backports.
Yes. Much too intrusive.
Still not solution for swap-file-creator on the horizon.
That fixes it but would add the ERROR: ld.so: object '/usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored. message to the boot process. Well, we could hide that, but then legitimate error which might happen in future (for some users) would be hidden from our view and fixing too.
Also cryptsetup luksFormat got slower by approximately factor 7.
(5 seconds without; 35 seconds with.)
Though, no errors reported with, except the performance regression.
Worth reporting upstream or to be expected?
Wondering about other solutions…
To enable hardened malloc globally is there any other option than editing /etc/ld.so.preload?
Can systemd somehow ignore contents of /etc/ld.so.preload? Then that could fix swap-file-creator.
The man / mandb issue is currently breaking the package build. It somehow messes the man pages up a bit. Lintian will complain.
genmkfile: manpage-has-bad-whatis-entry usr/share/man/man8/genmkfile.8.gz
Didn’t try wrapping man / mandb with hardened-malloc-disable yet.
Not sure. We’ll need to investigate what exactly in cryptsetup is causing this. For now, can you try rebuilding hardened_malloc but with a few more expensive security features disabled? GitHub - GrapheneOS/hardened_malloc: Hardened allocator designed for modern systems. It has integration into Android's Bionic libc and can be used externally with musl and glibc as a dynamic library for use on other Linux-based platforms. It will gain more portability / integration over time.
Try disabling CONFIG_WRITE_AFTER_FREE_CHECK, CONFIG_SLOT_RANDOMIZE, etc. and check if performance changes at all. See if you can narrow it down a bit.
Keep our custom config options though. Just append these ones to it.
We could build it into glibc but that would require maintaining a Whonix fork.
Side note: musl is developing another hardened malloc implementation inspired by hardened_malloc that will be built into it by default (although it’s not as hardened as hardened_malloc).
https://www.openwall.com/lists/musl/2020/05/13/1
I don’t think so.
glibc feature request: LD_ETC_IGNORE - environment variable to ignore /etc/ld.so.preload configuration file on a per-application basis
I’ve made a summary of all open issues:
Hardened Malloc
That did not go too well yet.
That still wouldn’t give us a way to disable hardened malloc on a per application basis without extraneous stderr error message?
( ERROR: ld.so: object ‘/usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so’ from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.)
adding hardened-malloc-disable not appropriate as this would inject stderr output ERROR: ld.so: object ‘/usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so’ from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored. during early boot process (swap-file-creator) and not appropriate to silence stderr either as this would hide legitimate error messages too.
Here is a new approach for disabling hardened malloc on a per-application basis without extraneous console stderr output.
Why hide folder /usr/lib/libhardened_malloc.so from application if we can hide /etc/ld.so.preload configuration file?
sudo touch /etc/empty
bwrap --dev-bind / / --ro-bind /etc/empty /etc/ld.so.preload man nano
That would be an ld-system-preload-disable tool.
(But not a global ld-disable-tool because it would not modify LD_PRELOAD environment variable. We could invent such a tool easily too if needed.)
That would be good enough?
It would not be an hardened-malloc-disable tool because other entries (unrelated user specific preloads) in /etc/ld.so.preload configuration file would be excluded too. A hardened-malloc-disable could copy /etc/ld.so.preload to a (temporary?) file, use
str_replace /usr/lib/libhardened_malloc.so/libhardened_malloc_kicksecure.so "" /path/to/other-ld.so.preload-file
and then use brwap to point the application to that file instead.
Should could added to helper-scripts package. (Previously my mistake directing hardened-malloc-disable at usablity-misc package.) Is an hardened-malloc-disable tool worth the effort or is ld-system-preload-disable tool sufficient?
Is it a security issue that users (non-root) can disable /etc/ld.so.preload? Is that something only root should be able to configure?
Yes, that would be great. Verified with /proc/self/maps that hardened_malloc is not loaded.
ld-system-preload-disable should be sufficient for now.
No. An attacker can only disable it for new processes that they create which there isn’t any point in. They can’t disable it for already existing processes. E.g. a malicious site can’t just disable hardened_malloc for Firefox and exploit it.
Is there any reason to create a new file instead of:
bwrap --dev-bind / / --ro-bind /dev/null /etc/ld.so.preload man nano
?
I tried Chromium from Debian backports and still got the same error so the error could be from Debian’s patches or the backport isn’t a recent enough version.
I’ll look over the patches and see if I can find anything. Looking at these has also made me realize just how terrible the Debian Chromium package is (they disable important mitigations like CFI, automatic variable initialization, etc.)
I just tried the same version of Chromium as in the Debian repos but on a different distro (Arch) and the error wasn’t there. It’s most likely Debian’s patches.
madaidan via Whonix Forum:
Is there any reason to create a new file instead of:
bwrap --dev-bind / / --ro-bind /dev/null /etc/ld.so.preload man nano?
While not tested by me yet, using /dev/null seems much better.
Will add the new wrapper soon and remove the old one (wrong package, my
mistake, and probably no need to reintroduce).
Implemented that.
There is one issue.
This fails if file /etc/ld.so.preload does not exists.
bwrap: Can’t create file at /etc/ld.so.preload: Permission denied
Can this be fixed at bwrap level? Or must I make sure at packaging level that file /etc/ld.so.preload always exists?
(That is for systems with hardened-malloc-kicksecure-enable not installed.)
Could you please report your chromium related findings to Debian? I guess replying again to
https://bugs.debian.org/971876 for the hardened malloc issue? (i.e. Debian patches are probably responsible for introducing the issue.)
And a separate ticket about CFI etc.?
Not really unless it’s run as root.
Uploaded to developers and testers repository just now.