Testers wanted! Upgrade Qubes-Whonix 13 -> Qubes-Whonix 14

News
#1

Originally published at: News - Whonix Forum

Qubes-Whonix 14 testers-only. Upgrades are possible.

https://www.whonix.org/wiki/Upgrading_Whonix_13_to_Whonix_14

Qubes-Whonix 14 downloadable TemplateVMs will be released later.

1 Like
Youtube working for you at normal speed with sound in Whonix 13 / 14?
Youtube working for you at normal speed with sound in Whonix 13 / 14?
#2

I did an upgrade from Qubes-Whonix 13 -> Qubes-Whonix 14:

After starting up the AppVM based on Whonix Workstation 14 template, along with the ProxyVM based on Whonix Gateway 14. It poped up this:

ERROR: whonix_firewall failed to load! 

The whonix_firewall failed to load for some reason. This could be due to the firewall being mis-configured or race-condition. Try restarting the VM to see if this error persists. 

Failure file /var/run/anon-firewall/failed.status does not exist, ok. 

output of systemctl status whonix-firewall: 

######################################## 
● whonix-firewall.service
Loaded: masked (/dev/null; bad)
Active: inactive (dead) 
######################################## 

To see this for yourself... 
1. Open a terminal. (dom0 -> Start Menu -> ServiceVM: sys-whonix-14-test -> Terminal) 
2. Run. 
systemctl status whonix-firewall

2. Also see. 
sudo journalctl -u whonix-firewall | cat

3. Try to manually start Whonix firewall. 
sudo whonix_firewall
If you know what you are doing, feel free to disable this check. Create a file /etc/whonix.d/50_whonixcheck_user.conf and add: 
whonixcheck_skip_functions+=" check_whonix_firewall_systemd_status "
2 Likes
#3

And then, when using arm in the gateway, there were established circuits, but there were neither Internet connection nor Tor control port connection in the workstation, which was as expected.

Firewall status in gateway:

user@host:~$ systemctl status whonix-firewall
● whonix-firewall.service
   Loaded: masked (/dev/null; bad)
   Active: inactive (dead)
user@host:~$ sudo journalctl -u whonix-firewall | cat
-- No entries --

Everything works fine after doing:

user@host:~$ sudo whonix_firewall 
OK: Loading Whonix firewall...
OK: Skipping firewall mode detection since already set to 'full'.
OK: (Full torified network access allowed.)
OK: TOR_USER: 107
OK: CLEARNET_USER: 1001
OK: USER_USER: 1000
OK: ROOT_USER: 0
OK: TUNNEL_USER: 1002
OK: SDWDATE_USER: 108
OK: WHONIXCHECK_USER: 113
OK: NO_NAT_USERS:  1001 1002 107
OK: The firewall should not show any messages,
OK: besides output beginning with prefix OK:...
OK: Whonix firewall loaded.
2 Likes
#4

Some trivial thing about the Wiki page :
http://dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Upgrading_Whonix_13_to_Whonix_14

I noticed that when using Tor Browser with media or high security level, the select code function does not work. There is no such a button anymore. What I did was double left-click my mouse to select all and the tricky thing is it actually select something like:

\n
code
\n

Therefore, when I copy and paste the selected thing to the Konsole, it will actually not work the first time I do a choice on [y/n].

This is definitely not a Whonix issue, but other testers may be aware of it now. :slight_smile:

These two commands seems inflexibly related to the name of two VMs:

sudo qubesctl state.sls qvm.anon-whonix

sudo qubesctl state.sls qvm.updates-via-whonix

Should people who use Whonix in Qubes always have anon-whonix and sys-whonix?

2 Likes
#5

All over all, the upgrading process is very smooth. The previous problem that forever waiting on apt-get remove also disappeared. :slight_smile:

I will keep using the newly upgraded templates and report issues if I meet one.

Great job for everybody who has been contributing to the Whonix!

2 Likes
#6

It turns out every time I (re)boot up the gateway, it will complain about ERROR: whonix_firewall failed to load!. And I have to do a sudo whonix_firewall to temporarily fix it.

I even tried to reload the firewall in template whonix-gw (not sure if I should do that), but the problem still exists.

1 Like
#7

I also had this experience on Qubes upgrade - both in 3.2 and 4.
As stated, systemctl status whonix-firewall shows it is masked.
Removing the mask and enabling the service in the template allows sys-whonix to start as expected.
(Question - is this the right thing to do? Or is there service that will run in sys-whonix that will unmask and enable?)

N.B Because the service is initially masked, whonixcheck takes a loooong time to produce any output.

Once this is fixed, sys-whonix started with firewall enabled and running.

I found that in 4.0rc5 whonixcheck in sys-whonix reports that Tor is disabled, although systemctl status shows it is running.
Trying to enable using whonixsetup results in error 0 unable to add "DisableNetwork 0" to /usr/local/etc/torrc.d/40_anon_connection_wizard.conf.
Indeed that file does not exist, although 95_whonix.conf does in same location.

2 Likes
#8

Hi @unman ! Thank you so much for reporting the bug. I tried to fix it in this PR. Could you please help to test it, @Patrick ?

#9

ust because the torrc parsing result is DisableNework 0 does not necessarily mean Tor is connected to the Tor network. For example:

  1. Tor process may even not start (because of a corrupted torrc option)
  2. Tor process starts, but DisableNetwork is set to 1 through control port/socket without writing to any torrc files.

This commit is an experimental demonstration on if we should talk to Tor control port directly. There are three questions need answering before adopting it:

Q: Can we say Tor is not enabled just because cookie does not exit?
Q: Can we say Tor is not enabled just because its control port does not exit?
Q: Shall we make it another check to say if Tor control socket is available?

#10

iry:

Hi @unman ! Thank you so much for reporting the bug. I tried to fix it in this PR. Could you please help to test it, @Patrick ?

parse correct torrc files in tor_enabled_check() by irykoon ¡ Pull Request #3 ¡ Kicksecure/helper-scripts ¡ GitHub

I don’t think this is good. Why should we parse fewer files there? If
the code is broken, it should be fixed so it would still generically
parse all files.

1 Like
#11

unman:

I also had this experience on Qubes upgrade - both in 3.2 and 4.
As stated, systemctl status whonix-firewall shows it is masked.
Removing the mask and enabling the service in the template allows sys-whonix to start as expected.

It whonix-firewall.service shouldn’t be masked to begin with. That might
be the root cause. I wonder what I masking it.

There was a migration from package whonix-gw-firewall /
whonix-ws-firewall to whonix-firewall package:

whonix-firewall/whonix-firewall.service at master ¡ Whonix/whonix-firewall ¡ GitHub

(Question - is this the right thing to do?

Maybe an ok workaround for testing but not the right fix.

Or is there service that will run in sys-whonix that will unmask and enable?)

No such services in Debian. Not a usual way to do things in Debian.

N.B Because the service is initially masked,

whonixcheck takes a loooong time to produce any output.

There is code to wait up to 30 seconds to avoid race conditions.

https://github.com/Whonix/whonixcheck/blob/master/usr/lib/whonixcheck/check_services.bsh#L93

Once this is fixed, sys-whonix started with firewall enabled and running.

I found that in 4.0rc5 whonixcheck in sys-whonix reports that Tor is disabled, although systemctl status shows it is running.

Not related to systemctl status.

This check:
https://github.com/Whonix/whonixcheck/blob/master/usr/lib/whonixcheck/check_tor_enabled.bsh

Backend:
https://github.com/Whonix/anon-shared-helper-scripts/blob/master/usr/lib/anon-shared-helper-scripts/tor_enabled_check

Trying to enable using whonixsetup results in error 0 unable to add "DisableNetwork 0" to /usr/local/etc/torrc.d/40_anon_connection_wizard.conf.

really whonixsetup?

whonixsetup refers to https://github.com/Whonix/whonixsetup CLI tool.

There is a bug in
https://github.com/Whonix/whonixsetup/blob/master/usr/lib/whonixsetup_/ft_m_1
indeed.

If /usr/local/etc/torrc.d/40_anon_connection_wizard.conf exists but does
not include #DisableNetwork 0, then ed -s /usr/local/etc/torrc.d/40_anon_connection_wizard.conf [...] will fail.
Please fix.

Indeed that file does not exist, although 95_whonix.conf does in same location.

So it must be yet another bug. Running with bash -x might help to find
out why it’s not created by whonixsetup?

#12

Because I was thinking since Whonix 14 is not really using %include directory feature by default, parsing all the files will also parse those are not actually used by Tor. This may cause whonixcheck has different result from what Tor is really using (like what happened to unman, whonixcheck complains about Tor is disabled while Tor works fine. )

For example, if there is a file called 60_xx.conf or 50_user.conf~ which contains an option DisableNetwork 1. The files are never used by Tor but they will be parsed by whonixcheck and then whonixcheck think Tor is disabled.

I agree that we should generically parse all the files but maybe only after Whonix really supports %include directory by default?

At that time, I can do a PR to parse all the files ends with .conf .

How do you like this idea?

1 Like
#13

Fixed by switching to another implementation. Not sure if this is a good idea. :slight_smile:

https://github.com/Whonix/whonixsetup/pull/1

When parsing the torrc file, we switch from command line flavor to using the python functions availabed in anon_connection_wizard package.

The advantages are as follows:

  1. Better consistency with anon_connection_wizard style parsing

While the command line flavor parsing comments (out) #DisableNetwork 0, anon_connection_wizard writes DisableNetwork 0 or DisableNetwork 1 to 40_connection_wizard.conf . This can be a issue when user uses both anon-connection-wizard and whonixsetup for at least once. Using the same implementation will aviod the inconsistency.

  1. More robust in handling different cases

Considering the inconsistency descriped above, the command line flavor parsing was not robust enough. For example, it will assume the final value in anon_connection_wizard.conf is 0 right after it sees a DisableNetwork 0 line. This will ignore the case where there is also a DisableNetwork 1 line after it.

  1. Lower maintanance cost

It allows us to only maintance one piece of code.

  1. Simplified the logic in the bash script

The disadvantages are as follows:

  1. anon-connection-wizard becomes a dependency of whonixsetup now

However, since whonixsetup should be installed in both Whonox-Gateway and Whonox-Workstation, but anon-connection-wizard should NOT be installed in Whonix-Worksatation, we cannot claim the dependency in Debian control.

  1. Less error info in CLI

The error happens in the python function may not be well-reflected to the CLI which is seen by user. For example, user may only know the attempt to enable Tor failed, but does not know if the problem is because of the missing of anon_connection_wizard package or anything else.

1 Like
#14

It didn’t happen to unman as far as I understand. It’s not what he reported.

The report is a mismatch between Tor enabled and Tor systemd running status. They are related. Sound the same rationally but are different on technical level. That is causing confusion.

Bug:

I would prefer to keep the parsing code as is:

  • hopefully soon enough whole torrc.d gets parsed
  • we could even call it a bug calling the folder .d if not all of it gets parsed but we also don’t want to change implementation too much back and forth to finally get the release out
  • simply ignoring config files not parsed here that would disable Tor if above bug didn’t exist also seems bad
1 Like
#15

This is very clever, very good!

Could be resolved by moving the shared cli-only code to whonixsetup or better anon-shared-helper scripts for Whonix 15.

(“shared cli-only code” roughly defined:

  • just doing “cli stuff” (systemd, config files)
  • no gui dependencies)
1 Like
#16

I have an idea now why this might be happening.

First whonix-firewall (new merged package) gets installed. Then either whonix-gw-firewall or whonix-ws-firewall package (old separate package) gets autoremoved. The latter may be running the prerm / postrm Debian maintainer scripts which disables systemd. Could be a bug in the Debian package migration process.

This is now covered in upgrade documentation.

Error - Whonix

Please test.

1 Like
#17

Yes, it’s hardcoded.

qubes-mgmt-salt-dom0-virtual-machines/qvm at master ¡ QubesOS/qubes-mgmt-salt-dom0-virtual-machines ¡ GitHub

Easy answer for 99,9% of users: Yes.

1 Like
#18

Generally it’s good to have code to check these things.

Could you please grep all of Whonix code to see how check_tor_enabled_do is used? It’s used multiple times. Changing it could break various things. Would require a rebuild of Whonix and test.

check_tor_enabled_do is a bad function name chosen by me back then. It’s more like check_tor_networking_enabling_in_config_do and specific as that.

  • Q: Can we say Tor is not enabled just because its control port does not exit?
  •  file_name_list+=" "
  • if [ ! -e “/var/run/tor/control” ]; then
  •  TOR_ENABLED="1"

  •  return 0
    fi
    fi

Not in that function check_tor_enabled_do. Config might be DisableNetwork 0 but but some other error prevents Tor from starting. Then whonixcheck would advice to run Whonix Setup to enable Tor but this is not really the issue.

socat - UNIX-CONNECT:/var/run/tor/control < <(echo $command_auth;echo $command_get_conf; echo $command_quit) | grep “250 DisableNetwork=0”

This needs to be protected by timeout. Please refer to other uses of timeout in Whonix source code. Also needs exit code checking since likely to exit non-zero in error case.

Maybe a new separate whonixcheck test in the appropriate order? There are quiet a few check_tor_… tests already.

1 Like
#19

I was thinking about this. But it seems this will make both whonixsetup and anon-connection-wizard depends on anon-shared-helper scripts.

#20

Tested in the Whonix-Gateway Template VM.

The problem has been fixed after adopting the steps in the instructions. Thank you, @Patrick !