AppArmor for Complete System - Including init, PID1, Systemd, Everything! - Full System MAC policy

[Patrick]

@madaidan

@Patrick have you tested this on Qubes yet?

With Qubes dom0 kernel:

Does not work with dom0 kernel. This is expected since by using dom0 kernel it also used dom0 initrd. Therefore apparmor-profile-everything cannot function. The good news about this is, that it does not seem to break anything either.

sudo aa-status

apparmor module is loaded.
16 profiles are loaded.
16 profiles are in enforce mode.
/**/*-browser/Browser/firefox
/usr/bin/apt-get
/usr/bin/man
/usr/bin/whonixcheck
/usr/lib/sdwdate/url_to_unixtime
/usr/lib/security-misc/pam_tally2-info
/usr/lib/security-misc/permission-lockdown
/usr/sbin/haveged
bootclockrandomization
firejail-default
init-systemd
man_filter
man_groff
nvidia_modprobe
nvidia_modprobe//kmod
system_tor
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
/usr/sbin/haveged (519)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

---

With Qubes VM Kernel:

Results coming soon.

[Patrick]

Nov 23 10:49:09 host audit[961]: AVC apparmor=“DENIED” operation=“exec” profile=“/usr/lib/security-misc/pam_tally2-info” name=“/usr/sbin/pam_tally2” pid=961 comm=“pam_tally2-info” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0
Nov 23 10:49:09 host audit[961]: AVC apparmor=“DENIED” operation=“open” profile=“/usr/lib/security-misc/pam_tally2-info” name=“/usr/sbin/pam_tally2” pid=961 comm=“pam_tally2-info” requested_mask=“r” denied_mask=“r” fsuid=0 ouid=0
Nov 23 10:49:09 host kernel: audit: type=1400 audit(1574506149.778:19): apparmor=“DENIED” operation=“exec” profile=“/usr/lib/security-misc/pam_tally2-info” name=“/usr/sbin/pam_tally2” pid=961 comm=“pam_tally2-info” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0
Nov 23 10:49:09 host kernel: audit: type=1400 audit(1574506149.778:20): apparmor=“DENIED” operation=“open” profile=“/usr/lib/security-misc/pam_tally2-info” name=“/usr/sbin/pam_tally2” pid=961 comm=“pam_tally2-info” requested_mask=“r” denied_mask=“r” fsuid=0 ouid=0

[Patrick]

/usr/sbin/pam_tally2 mrix,

[Patrick]

@madaidan

@Patrick have you tested this on Qubes yet?

No, doesn’t work yet.

@Patrick

With Qubes VM Kernel:

Results coming soon.

cat /proc/version

Linux version 4.19.0-6-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)

sudo aa-status

apparmor module is loaded.
16 profiles are loaded.
16 profiles are in enforce mode.
/**/*-browser/Browser/firefox
/usr/bin/apt-get
/usr/bin/man
/usr/bin/whonixcheck
/usr/lib/sdwdate/url_to_unixtime
/usr/lib/security-misc/pam_tally2-info
/usr/lib/security-misc/permission-lockdown
/usr/sbin/haveged
bootclockrandomization
firejail-default
init-systemd
man_filter
man_groff
nvidia_modprobe
nvidia_modprobe//kmod
system_tor
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
/usr/sbin/haveged (431)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

[user@dom0 ~]$ sudo xl console whonix-ws-15

.[30m.[47mWelcome to GRUB!

.[37m.[40m.[37m.[40m.[37m.[40m.[3;35H [ grub-xen.cfg 424B 100% 1.53KiB/s ].[3;1Herror: no such device: /boot/xen/pvboot-x86_64.elf.
Reading (xen/xvda,gpt3/boot/grub/grub.cfg
.[H.[J.[1;1H.[1;36H [ grub.cfg 5.67KiB 100% 8.15KiB/s ].[1;1Herror: file /boot/grub/fonts/unicode.pf2' not found. error: no suitable video mode found. .[H.[J.[1;1H Booting Whonix GNU/Linux’

Loading Linux 4.19.0-6-amd64 …
.[4;24H [ vmlinuz-4.19.0-6-amd 5.03MiB 100% 4.95MiB/s ].[4;1HLoading initial ramdisk …
.[5;22H [ initrd.img-4.19.0-6- 25.46MiB 100% 23.39MiB/s ].[5;1H[ 0.000000] Linux version 4.19.0-6-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.19.0-6-amd64 root=/dev/xvda3 ro xen_scrub_pages=0 root=/dev/mapper/dmroot console=hvc0 console=tty0 swiotlb=8192 noresume intel_iommu=on amd_iommu=on slab_nomerge slub_debug=FZP mce=0 pti=on mds=full,nosmt

user@host:~$ sudo update-initramfs -u

update-initramfs: Generating /boot/initrd.img-4.19.0-6-amd64
I: The initramfs will attempt to resume from /dev/xvdc1
I: (UUID=aa5657bd-b175-42e0-93df-833b6cd3f8c7)
I: Set the RESUME variable to override this.
user@host:~$ ls /boot/initrd.img-4.19.0-6-amd64
/boot/initrd.img-4.19.0-6-amd64
user@host:~$ ls -la /boot/
total 31436
drwxr-xr-x 3 root root 4096 Nov 23 11:09 .
drwxr-xr-x 20 root root 4096 Nov 23 10:40 …
-rw-r–r-- 1 root root 206361 Nov 11 00:30 config-4.19.0-6-amd64
drwxrwxr-x 2 root root 4096 Nov 23 10:43 grub
-rw-r–r-- 1 root root 26696863 Nov 23 11:09 initrd.img-4.19.0-6-amd64
-rw-r–r-- 1 root root 5270768 Nov 11 00:30 vmlinuz-4.19.0-6-amd6

[Patrick]

The initrd looks alright in so far that it contains strings of the changes to initrd.

mkdir /tmp/initrd
cd /tmp/initrd
zcat /boot/initrd.img-4.19.0-6-amd64 | cpio -idmv
grep -r -i apparmor

scripts/init-bottom/apparmor-profile-everything:echo “profile init-systemd /lib/systemd/systemd flags=(complain) {}” | /sbin/apparmor_parser -a
scripts/init-bottom/ORDER:/scripts/init-bottom/apparmor-profile-everything “$@”
Binary file usr/sbin/apparmor_parser matches
Binary file usr/bin/udevadm matches
Binary file usr/lib/systemd/systemd-udevd matches

[madaidan]

@Patrick_mobile

Maybe a wrapper script (whitelisted in apparmor) that sets hardened malloc ld preload?

The binary the wrapper script executes would have to be whitelisted. We can’t just whitelist the wrapper.

We could probably make a list of programs to use hardened_malloc with and whitelist them but then that would also allow any malicious LD_PRELOAD tricks on those programs.

Or, maybe we can make an issue on the apparmor gitlab repo about adding specific variables that can be whitelisted when using environment scrubbing if that’s even possible.

e.g.

/bin/bash Pix allow_var="LD_PRELOAD=/usr/lib/libhardened_malloc.so",

So this would only allow preloading hardened_malloc and nothing else.

[Patrick]

@Patrick

@Patrick have you tested this on Qubes yet?

No, doesn’t work yet.

apparmor-profile-everything breaks Qubes upgrading
https://phabricator.whonix.org/T936

[Patrick]

@madaidan

Or, maybe we can make an issue on the apparmor gitlab repo about adding specific variables that can be whitelisted when using environment scrubbing if that’s even possible.

e.g.

/bin/bash Pix allow_var="LD_PRELOAD=/usr/lib/libhardened_malloc.so",

So this would only allow preloading hardened_malloc and nothing else.

Thank you for submitting this feature request!

Add a way to whitelist variables from environment scrubbing (#66) · Issues ·...

[madaidan]

Got a response:

This one will not come quickly: the environment scrubbing isn’t actually performed by AppArmor but by the system c library. AppArmor sets the same flag that’s used when setuid/setgid programs are executed, and when the program starts, libc notes the flag and scrubs environment before going very far.

Providing granularity when scrubbing the environment is something we want to do but I don’t think it’ll happen for a while.

The good news is that you could steal the environment scrubbing routine from glibc, add it as a constructor to your libhardened_malloc.so, and modify AppArmor policy to stop setting the secure exec flag when you want to use this library. That way you could still get a mostly-scrubbed environment and use the different allocator. It might not be as easy as this fine-grained scrubbing feature but you could be using it a lot sooner.

I think the bottom part is suggesting we modify hardened_malloc to scrub the environment for us instead of using apparmor for it.

[Patrick]

Worth contacting hardened malloc upstream?

@madaidan

modify AppArmor policy to stop setting the secure exec flag when you want to use this library

Can you make head or tail of this? If not, let’s ask?

[madaidan]

@Patrick

Worth contacting hardened malloc upstream?

Probably.

@Patrick

Can you make head or tail of this? If not, let’s ask?

It means to make the apparmor policy execute those binaries without scrubbing the environment as hardened_malloc would handle it (with the proposed solution).

[madaidan]

Add a way to scrub the environment

[madaidan]

The issue was closed and he said:

I’d recommend just linking those programs against it. I don’t want to add this.

Linking programs against it is something that is done at compile time AFAIK which isn’t a viable solution for most apps so we can’t have both environment scrubbing and hardened_malloc. Unless we add it to /etc/ld.so.preload and use hardened_malloc globally (with exceptions added for broken programs).

[Patrick]

Add a way to whitelist variables from environment scrubbing (#66) · Issues · AppArmor / apparmor · GitLab getting worse news. (Bold added by me.)

John Johansen commented:

So not that @setharnold is wrong current environment scrubbing is handled by the loader as Seth described, and it is entirely insufficient.

However there are indeed plans for apparmor to pick up environment variable scrubbing. It can be done in the kernel, and we actually have a basic prototype for it but it isn’t a prioritized work item and there is still a lot of work to do on before it can land.

[Patrick]

Could you fix these please?

user@host:~$ sudo journalctl -b | grep -i denied
Nov 25 14:04:40 host audit[1533]: AVC apparmor=“DENIED” operation=“link” info=“link not subset of target” error=-13 profile=“/usr/bin/apt-get” name=“/usr/lib/security-misc/pam_tally2-info.dpkg-tmp” pid=1533 comm=“dpkg” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0 target=“/usr/lib/security-misc/pam_tally2-info”
Nov 25 14:04:40 host kernel: audit: type=1400 audit(1574690680.014:19): apparmor=“DENIED” operation=“link” info=“link not subset of target” error=-13 profile=“/usr/bin/apt-get” name=“/usr/lib/security-misc/pam_tally2-info.dpkg-tmp” pid=1533 comm=“dpkg” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0 target=“/usr/lib/security-misc/pam_tally2-info”
Nov 25 14:04:44 host audit[1830]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host kernel: audit: type=1400 audit(1574690684.266:20): apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host kernel: audit: type=1400 audit(1574690684.266:21): apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host kernel: audit: type=1400 audit(1574690684.266:22): apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host kernel: audit: type=1400 audit(1574690684.266:23): apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host audit[1830]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host audit[1830]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:44 host audit[1830]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1830 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:04:47 host audit[2295]: AVC apparmor=“DENIED” operation=“capable” profile=“/usr/bin/apt-get” pid=2295 comm=“(sd-askpwagent)” capability=24 capname=“sys_resource”
Nov 25 14:04:47 host kernel: audit: type=1400 audit(1574690687.514:24): apparmor=“DENIED” operation=“capable” profile=“/usr/bin/apt-get” pid=2295 comm=“(sd-askpwagent)” capability=24 capname=“sys_resource”

---

user@host:~$ sudo journalctl -b | grep -i denied
Nov 25 14:06:43 host audit[1549]: AVC apparmor=“DENIED” operation=“link” info=“link not subset of target” error=-13 profile=“/usr/bin/apt-get” name=“/usr/lib/security-misc/pam_tally2-info.dpkg-tmp” pid=1549 comm=“dpkg” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0 target=“/usr/lib/security-misc/pam_tally2-info”
Nov 25 14:06:45 host audit[1837]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1837 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:06:45 host audit[1837]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1837 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:06:45 host audit[1837]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1837 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:06:45 host audit[1837]: AVC apparmor=“DENIED” operation=“file_inherit” profile=“/usr/lib/security-misc/permission-lockdown” name=“/dev/pts/1” pid=1837 comm=“permission-lock” requested_mask=“wr” denied_mask=“wr” fsuid=0 ouid=0
Nov 25 14:06:48 host audit[2300]: AVC apparmor=“DENIED” operation=“capable” profile=“/usr/bin/apt-get” pid=2300 comm=“(sd-askpwagent)” capability=24 capname=“sys_resource”

[Patrick]

@madaidan

unless debian provides a glibc-hardened-malloc package or similar.

Seems very unlikely.

• RFP: hardened-malloc – hardened memory allocator
• glibc feature request - consider using Hardened Malloc
• debian glibc feature request - consider using hardened malloc (hardened memory allocator)
• glibc-alpha mailing list - consider using Hardened Malloc
• Hardened Malloc - Hardened Memory Allocator - #50 by Patrick

@madaidan

I’m thinking of using apparmor’s environment scrubbing on all binaries although that would break using LD_PRELOAD for hardened_malloc.

Also, using a different environment for apt is not enough. Attackers can still write to /etc/ld.so.preload and other /etc/ld.so.* files so everything preloads their malicious library, including apt even with a new environment.

But, using environment scrubbing and disallowing writing to /etc/ld.so.preload will prevent us from using hardened_malloc at all unless …

Users could boot into admin mode (without apparmor-profile-everything) and configure /etc/ld.so.preload?

We could allow root to run a yet to be invented script which copies a file /etc/ld.so.preload_template to /etc/ld.so.preload.

/etc/ld.so.preload_template would have the following contents:

/usr/lib/libhardened_malloc.so/libhardened_malloc.so

That way while apparmor-profile-everything is enabled, root could enable hardened malloc but not configure an arbitrary (malicious) /etc/ld.so.preload?

[madaidan]

@Patrick

Users could boot into admin mode (without apparmor-profile-everything) and configure /etc/ld.so.preload ?

Yes, but we should rely on booting without apparmor as little as possible.

@Patrick

We could allow root to run a yet to be invented script which copies a file /etc/ld.so.preload_template to /etc/ld.so.preload .

/etc/ld.so.preload_template would have the following contents:

/usr/lib/libhardened_malloc.so/libhardened_malloc.so

That way while apparmor-profile-everything is enabled, root could enable hardened malloc but not configure an arbitrary (malicious) /etc/ld.so.preload ?

That sounds like a good idea.

[madaidan]

@Patrick

Could you fix these please?

Fix apt

Fix permission-lockdown

[Patrick]

Thanks, merged.

[Patrick]

sudo journalctl -b | grep -i denied
Dec 06 15:35:06 host audit[1538]: AVC apparmor=“DENIED” operation=“link” info=“link not subset of target” error=-13 profile=“/usr/bin/apt-get” name=“/usr/lib/security-misc/pam_tally2-info.dpkg-tmp” pid=1538 comm=“dpkg” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0 target=“/usr/lib/security-misc/pam_tally2-info”
Dec 06 15:35:06 host kernel: audit: type=1400 audit(1575646506.818:19): apparmor=“DENIED” operation=“link” info=“link not subset of target” error=-13 profile=“/usr/bin/apt-get” name=“/usr/lib/security-misc/pam_tally2-info.dpkg-tmp” pid=1538 comm=“dpkg” requested_mask=“x” denied_mask=“x” fsuid=0 ouid=0 target=“/usr/lib/security-misc/pam_tally2-info”

Could you fix this one please?