Benefiting from security-misc inside Qubes-Whonix.

Qubes-Whonix
1

Users can benefit from security-misc inside of Qubes-Whonix by replacing passwordless root with a dom0 prompt[1], causing only few issues.

Further hardening is possible by completely denying any sudo attempts inside of anon-whonix and sys-whonix.

Reduce Kernel Information Leaks

No issues encountered inside of anon-whonix.

Breaks networking of any vm connected to sys-whonix.

SUID Disabler and Permission Hardener

Works perfectly fine inside of anon-whonix and sys-whonix.

hidepid

No issues inside of anon-whonix.

Breaks anon-connection-wizard inside of sys-whonix, the following log trace is produced when attempting to connect to tor via anon-connection-wizard.

tor_status was called.
tor_status status: tor_disabled
ERROR: pkexec /usr/libexec/anon-gw-anonymizer-config/tor-config-sane Exit Code: 127
torrc_file_path: /usr/local/etc/torrc.d/40_tor_control_panel.conf
ACW: executing: pkexec /usr/libexec/anon-connection-wizard/acw-write-torrc /tmp/tmpby2795nq
Error checking for authorization com.kicksecure.anon-connection-wizard.acw-write-torrc: GDBus.Error:org.gtk.GDBus.UnmappedGError.Quark._g_2dfile_2derror_2dquark.Code4: Failed to open file ?/proc/2538/status?: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/anon_connection_wizard/anon_connection_wizard.py", line 1273, in next_button_clicked
    subprocess.check_call(command)
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['pkexec', '/usr/libexec/anon-connection-wizard/acw-write-torrc', '/tmp/tmpby2795nq']' returned non-zero exit status 127.
zsh: IOT instruction (core dumped)  anon-connection-wizard
  • Temporary workaround: Connect to tor first, then enable proc-hidepid.service inside of sys-whonix template.
Remount Secure

Enabling via grub has no effect, have to create simple remount-secure.service as temporary workaround.

[Unit]

[Service]
Type=oneshot
ExecStart=remount-secure 3

[Install]
WantedBy=multi-user.target

Any level with noexec flag breaks updates in template via salt - still able to update manually inside template via terminal.

Traceback (most recent call last):
  File "/usr/bin/qubes-vmexec", line 5, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3/dist-packages/qubesagent/vmexec.py", line 55, in main
    os.execvp(command[0], command)
  File "<frozen os>", line 574, in execvp
  File "<frozen os>", line 597, in _execvpe
PermissionError: [Errno 13] Permission denied
  • Didnā€™t investigate further, but the issue is obvious - noexec blocking something relevant from executing.

Virtualization check error when running commands like qvm-copy-to-vm to vm with noexec level enabled.
qvm-copy-to-vm anon-whonix example-file

Failed to check for virtualization: Permission denied

  • Same issue as before, just this time without any log trace.

Breaks bind dirs such as /var/lib/tor by remounting /var, causing tor guard nodes to rotate on each sys-whonix boot.

  • Temporary workaround: Edit remount-secure and disable mounting of /var by running sudoedit /usr/bin/remount-secure and commenting out _var inside of main.

This is me documenting my experiments, I am using security-misc inside of Qubes-Whonix with band-aid fixes stated above with no issues.

I understand that Qubes OS does not believe in restricting root for security, but security-misc is useful against non sophisticated malware and remount-secure could be expanded by optionally mounting more dirs as tmpfs to go easier on disk.

I am willing to make and contribute full patches for listed issues, just need heads up about security-misc future for Qubes-Whonix

[1] https://forum.qubes-os.org/t/replacing-passwordless-root-with-a-dom0-prompt/19074/ - archive

1 Like
2

I am certainly interested in sudo hardening.
(Multiple Boot Modes for Better Security: an Implementation of Untrusted Root)

So any patches are appreciated. If itā€™s only mostly done and a few remaining issues, I can certainly try looking into any remaining breakage.

hidepid breaks pkexec. Thatā€™s a pkexec upstream issue. I certainly wonā€™t find a solution for that. Reference might be here somewhere:

That is expected because itā€™s enabled by default nowadays.

I am not sure what future that approach has.

Tons of approaches, discussions.

In essence: the new idea is to ship a hardened /etc/fstab.

file needs more work: security-misc/usr/share/doc/security-misc/fstab-vm at master Ā· Kicksecure/security-misc Ā· GitHub

For Qubes you could contribute fstab hardening directly to Qubes, starting here:

Does pkexec work?

This might be outdated. This is evidenced by the discussion and the ticket still being open:

File /etc/sudoers.d/qubes already had this comment removed. Website is probably also outdated. Reported just now.

1 Like
3
1 Like
4

No, causes log trace similiar to what is observable inside of anon-connection-wizard log trace.
pkexec yes

Error checking for authorization org.freedesktop.policykit.exec: GDBus.Error:org.gtk.GDBus.UnmappedGError.Quark._g_2dfile_2derror_2dquark.Code4: Failed to open file ?/proc/3696/status?: No such file or directory
zsh: exit 127 pkexec yes

You got me interested in investigating this, feels like providing fix to pkexec would be most beneficial to prioritize as other distros that rely on polkit could utilize hidepid aswell after upstream fix.

Will investigate pkexec source code, log trace shows where the error happens - hopefully pokit logic will be compatible with hidepid logic, will report any findings here / directly contribute to polkit.

This is perfect idea and is what Iā€™ve been doing to my non Whonix templates.

For Whonix I decided to go with Remount Secure because it was part of security-misc and thought in future it would get updates, so I would never have to touch /etc/fstab ever.

I will experiment with custom hardened /etc/fstab entries further and soon report my findings here / kicksecure git / Qubes OS git.

Thank you for the correction and the documentation commit, your assumption as you state in the pull request is correct.

I believe that this might have created the impression that Qubes isnā€™t interested in sudo hardening, which I think is no longer true.

Thanks for the heads up!

1 Like
5

Looked up the most on topic references for the hidepid issue:

Ignoring hidepidā€¦ Only by upgrading from Qubes passwordless root to a more hardened sudo configurationā€¦ Does that work? If thereā€™s issues, also patches welcome. Also does that alone (without hidepid) break pkexec?

If there are non-hidepid related sudoers (and maybe even pkexec) issues I might be able to fix these. Did some of these in the past.

1 Like
6

Documentation Remount Secure was outdated. The dracut module is at time of writing disabled and needs to be moved to a different folder to enable it. Alternatively, there is an already provided systemd unit remount-secure. Documentation (previous link) slightly updated just now.

related:

1 Like
7

Forgot to mention this previously, this did not seem to be the case for me? Iā€™ve been using the same Qubes-Whonix version 17 templates since earliest Qubes OS 4.2 release candidate, so maybe that is why.

pkexec whoami

==== AUTHENTICATING FOR org.freedesktop.policykit.exec ====
Authentication is needed to run `/usr/bin/whoamiā€™ as the super user
Authenticating as: root

dom0 prompt happens

polkit-agent-helper-1: error response to PolicyKit daemon: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: No session for cookie
==== AUTHENTICATION FAILED ====
Error executing command as another user: Not authorized
This incident has been reported.
zsh: exit 127 pkexec whoami

sudo whoami
dom0 prompt happens

root

anon-connection-wizard still doesnā€™t work as expected due to pkexec still being broken even with proc-hidepid.service disabled.

Thanks, looks like I missed the original remount-secure.service.

8

Still all unrelated to hidepid:
pkexec in Qubes with dom0 prompt could be difficult.
pkexec works different than sudo. That is to say, pkexec whoami might be expected to fail because thereā€™s no pkexec configuration file for whoami.

Itā€™s config files are in /usr/share/polkit-1. Most are in:

ls -la /usr/share/polkit-1/actions

But I donā€™t know much about Qubes pkexec hardening in VMs yet.

I would also suggest to try some other applications that use pkexec that arenā€™t developed by Kicksecure, Whonix to make sure itā€™s not because of bugs originating from there. Because that was added only relatively recently. Suggested steps:

  1. Try pkexec ā€œnormallyā€ without extra hardening to see how it works.
  2. Make it compatible with Qubes sudo hardening.
  3. Kicksecure, Whonix specific fixes if any are requried.

If upgraded itā€™s harder not using permission-hardener than using it because its run from security-misc.postinst.

1 Like
9

Running programs that have pkexec configuration such as mousepad or thunar output the following log:

Error executing command as another user: Not authorized

This incident has been reported.
zsh: exit 127 pkexec mousepad

The reason why pkexec breaks is due to removal of these files:

  • Disable PolKitā€™s default-allow behavior:
[root@debian-8]# rm /etc/polkit-1/rules.d/00-qubes-allow-all.rules
[root@debian-8]# rm /etc/polkit-1/localauthority/50-local.d/qubes-allow-all.pkla

from qubes sudo dom0 prompt guide

qubes-allow-all.pkla 
[Qubes allow all]
Identity=unix-group:qubes
Action=*
ResultAny=yes
ResultInactive=yes
ResultActive=yes
00-qubes-allow-all.rules 
//allow any action, detailed reasoning in sudoers.d/qubes
polkit.addRule(function(action,subject) { if (subject.isInGroup("qubes")) return polkit.Result.YES; });

I am guessing these files are removed because otherwise pkexec becomes instant way to obtain root without any obstacles.

I got intereted in 00-qubes-allow-all.rules becuase of its syntax and found its docs: https://www.freedesktop.org/software/polkit/docs/latest/polkit.8.html

In there I found:

string spawn( string argv);

With the following example:

Run an external helper to determine if the current user may reboot the system:

polkit.addRule(function(action, subject) {
    if (action.id.indexOf("org.freedesktop.login1.reboot") == 0) {
        try {
            // user-may-reboot exits with success (exit code 0)
            // only if the passed username is authorized
            polkit.spawn(["/opt/company/bin/user-may-reboot",
                          subject.user]);
            return polkit.Result.YES;
        } catch (error) {
            // Nope, but do allow admin authentication
            return polkit.Result.AUTH_ADMIN;
        }
    }
});

The example made me realize that this may be simpler than thought to fix, so I undid any sudo hardening changes and attempted to implement dom0 prompt just for pkexec by editing 00-qubes-allow-all.rules.

polkit.addRule(function(action, subject) {
	try {
		polkit.spawn([
			"/usr/lib/qubes/qrexec-client-vm",
			"dom0", "qubes.VMAuth"
		]);

		return polkit.Result.YES;
	} catch(error) {
		return polkit.Result.NO;
	}
});

pkexec
dom0 prompt happens

Error executing command as another user: Not authorized

This incident has been reported.
zsh: exit 127 pkexec

Replacing the polkit.spawn(["/usr/lib/qubes/qrexec-client-vm"... with polkit.spawn(["echo"]); ā€œworksā€.

pkexec
logs in as root

I checked exit code in terminal manually:
qrexec-client-vm dom0 qubes.VMAuth
dom0 prompt happens, I press OK

1

echo $?

0

qrexec-client-vm dom0 qubes.VMAuth
dom0 prompt happens, I press Cancel

0

echo $?

126

So I thought the polkit.spawn may expect that spawned process doesnā€™t take long time (as in, pressing OK in the prompt takes some time) as some kind of protection, so I edited qubes.VMAuth to always allow - with no luck, same output about authorization failure as before.

Didnā€™t investigate further, ran out of ideas.

1 Like
10

I doubt that. Try a zenity or something prompt inside the VM not dom0 to test that theory?

2 Likes
11

There are various alternative polkit agents for kde, gnome, mate, etc. Maybe youā€™d need to create a custom polkit agent?

Aka how to combine polkit or with yubikey?

2 Likes
12

https://packages.debian.org/sid/polkit-1-auth-agent

You probably need to implement something simpler but hopefully a lot simpler with less dependencies.

https://www.freedesktop.org/software/polkit/docs/0.105/polkit-agents.html

2 Likes
13

Great news, I got dom0 prompt for pkexec working with some trivial debugging!

Iā€™ve made polkit.spawn run "/usr/bin/pkexec-prompt" which is a bash script that Iā€™ve created.

I then observed some variables by outputting them to /tmp

whoami > /tmp/whoami
qrexec-client-vm dom0 qubes.VMAuth > /tmp/qrexec
echo $? > /tmp/exit

cat /tmp/whoami

polkitd

cat /tmp/qrexec

cat: /tmp/qrexec No such file or directory
zsh: exit 1 cat /tmp/qrexec

cat /tmp/exit
1 if ā€œOkā€ was pressed, 126 if ā€œCancelā€ was pressed.

Immediately can observe different behavior when running qrexec-client-vm in terminal as user vs running it via polkit.spawn.

  1. No output from qubes.VMAuth qrexec - meant to output 1 when ā€œOkā€ was pressed and 0 when ā€œCancelā€ was pressed, in this case there is no output.
  2. Program exit codes are different, qrexec-client-vm is meant to exit with 0 if ā€œOkā€ gets pressed and non zero otherwise, in this case it exits with 1.

With this behavior in mind I made working pkexec-prompt.

qrexec-client-vm dom0 qubes.VMAuth
exit $(($? - 1)) # DANGEROUS!

And with that in place pkexec prompt works, exit is 0 on ā€œOkā€, 125 on "Cancel!
pkexec
dom0 prompt happens, I press ā€œOkā€
gets logged in as root
pkexec
dom0 prompt happens, I press ā€œCancelā€

Error executing command as another user: Not authorized

This incident has been reported.
zsh: exit 127 pkexec

So now this is not permanent solution, different behavior needs to be investigated.
I have not tested whet ever this works with sudo dom0 prompt, will be doing that shortly just reporting my findings.

Iā€™ve tried running "zenity", "--info" via polkit.spawn, nothing happend and got unathorized message meaning that the polkitd user doesnt have access to display and most likely other important resources to spawn said gui, on that topic the differing behaviour is clearly somewhere in fault of polkitd user configuration.

Avoided this for now, most likely not needed yet. Will look into it if behavior cannot be made same via more casual solution.

2 Likes
14

You probably need

&> /tmp/qrexec

(bashism) to catch the stderr or:

qrexec-client-vm dom0 qubes.VMAuth > /tmp/qrexec 2>&1

You could post the full script somewhere. If short enough, I can suggest some refactoring, debugging.

1 Like
15

Ok

qrexec-client-vm: qrexec-agent-data.c:378:handle_data_client: Data vchan connection failed

Cancel

Request refused

I also tried tracing qrexec-client-vm via strace but couldnā€™t get it to work when ran by polkit user.

Is wrong, 1 seems to happen when connection (Ok) happend but no data could be recv, otherwise just exit code from remote process. Someone with better understanding of qrexec-client-vm could clear this up, but currently the exit code handling seems to have no issues.

qrexec-client-vm dom0 qubes.VMAuth

case $? in
0 | 1)
		exit 0
		;;
*)
		exit 1
		;;
esac

But since the original community made sudo prompt uses data received instead of relying on exit code hack - as mentioned before not optimal solution.

Seems that this is more Qubes related? This information could help bring vm auth qrexec by default into debian, fedora templates if sudo and polkit is all they use and just confirmation that pkexec is compatible with sudo hardening, meaning Qubes-Whonix can fully benefit.

On Whonix related note: I noticed I got dom0 sudo prompts when booting pkexec testing vm (based on gateway, doesnā€™t happen in workstation), I wasnā€™t able to trace what is invoking them (maybe systemcheck?). I have automatic deny for sudo prompt inside sys-whonix and it seems to be causing no issues.

1 Like
16

Yes. So it might also help creating a ticket at Qubes or mailing list discussion to get further input.
If itā€™s shells scripting, I might notice some stderr redirection issues, exit code handling, error handling and similar issues. Even if Qubes related I would contribute.

What attempted to use sudo (but was refused) should be in the systemd journal.

Could the dom0 prompt learn a feature to show what command is asked or denied execution?

1 Like
17

offtopic: github seems to hate freedom and doesnā€™t allow me to sign up via tor or throwaway email, I like torproject for having their own git server but unfortunately I think that is out of interest scope for Qubes, kicksecure, Whonix due to infrastructure reasons.
Anyone willing to relay current information to https://github.com/QubesOS/qubes-issues/issues/2695 until I get github account sorted would be nice.

Qubes related steps:

  1. Include qrexec service for sudo prompt in dom0 by default.
  2. Make qubes-builder apply relevant pam and polkit configuration for sudo prompt.
  • Thatā€™s all that is currently required (atleast what is known for full sudo prompt support by default), next generation of templates could have this already implemented. Also consider investigating possible privilege escalation surface to make sure no obvious flaw is in place.

For Whonix then remains:

  1. Investigate hide-hardware-info.service breaking sys-whonix (will do personally soon).
  2. Help with hidepid compability in pkexec (will do personally soon).
  3. Trace pkexec requests in gateway on boot (tried but failed, any help with this welcome).
  4. Look into any other issues mentioned in feature specific threads and investigate them as well.

Remount secure needs to be made Qubes (in Qubes-Whonix case atleast) compatible so some information needs to be relayed as well.

After all that, could security-misc be enabled by default, is security-misc intended to be enabled by default if compatible (in future)? - or atleast testers only banner removed?

I already tried checking both dom0 and vm journals, could not find any relevant information.

Maybe can try looking through gateway exclusive startup services, or maybe clone some relevant gateway repo in which such files may reside and grep -rn through them?

Good idea, I think what the prompt displays is currently out of user control. Would need some kind of sudo wrapper or pam config to inform current qrexec of action - qrexec-client-vm dom0 qubes.VMAuth+"sudo whoami", may be suspect to spoofing and extra dom0 attack surface.

1 Like
18

As for patches, git branches, pull requestsā€¦ Btw no github needed. Just in case that impression was created.

  • Kicksecure, Whonix: Any git branch anywhere can be reviewed. There is no need for a pull request as long as there is a git branch. Could even be git-format-patch (forums, e-mail compatible) (best avoided) but I hope a git branch somewhere would be possible. Or some other alternative.
  • Qubes: Has also mailing list, forums. Not sure git branches on git repositories other than github (or git-format-patch or similar) would be reviewed if sent there. Chances are they would be.

security-misc is already enabled by default for the most part with just ā€œsome partsā€ being opt-in.

The goal is of course to enable everything by default that doesnā€™t break too much stuff. Whatā€™s ā€œtoo muchā€ is a hard decision.

related:

As per Search the Source Codeā€¦
In derivative-maker folder:

find -type f -not -iwholename '*.git*' | grep -i sudoers

Orā€¦

find -type f -not -iwholename '*.git*' | grep -i polkit

or myfind: Dev/git - Kicksecure chapter Find Files in Kicksecure wiki

That lists all sudoers files.

Or even open them all in your favorite editorā€¦

find -type f -not -iwholename '*.git*' | grep -i sudoers | xargs editor

Yeah. That stuff is always welcome for review.

1 Like
19

Yes, the Qubes devs have always accepted patches via email (git format-patch) on the qubes-devel mailing list. :slight_smile:

https://www.qubes-os.org/doc/source-code/#how-to-send-patches

(And no Google account is necessary, in case anyone is worried about that.)

2 Likes
20

Having trouble tracing whatever prompts for pkexec at boot, briefly searched through derivative-maker for pkexec, any scripts that invoke pkexec, listed all services all with no obvious lead.

Tried dynamically listing all services in gateway and one by one restarting them, also tried parsing all their ExecStart and grepping for pkexec, found nothing. Really bad luck with this and truly a mystery to me where this thing could be coming from.

Makes sense, will make post on their forums SOON and list stuff that needs investigating to bring some attention to this, also because I found some more Qubes related issues.

1 Like