After upgrading Whonix-Gateway to latest version it’s even worse.
The upgrade process:
sudo apt update
sudo apt full-upgrade
sudo apt autoremove
sudo reboot
sudo release-upgrade
sudo reboot
sudo apt autoremove
Changed guard (not recommended for security) few times by:
sudo systemctl stop tor@default
sudo rm /var/lib/tor/state
sudo systemctl start tor@default
What anon-log
shows (selected lines from logs and these are very frequent):
vanguards.service:
Tor bug #29699: Got 1 dropped cell on circ ... (in state HS_SERVICE_INTRO HSSI_ESTABLISHED; old state HS_SERVICE_INTRO HSSI_CONNECTING)
We force closed circuit ...
Possible Tor bug, or possible attack if very frequent: Got 1 dropped cell on circ ... (in state HS_SERVICE_REND HSSR_JOINED; old state HS_SERVICE_REND HSSR_CONNECTING)
Tor has been failing all circuits for 30 seconds!
Tor has been failing all circuits for 60 seconds!
Tor has been failing all circuits for 90 seconds!
Circ ... exceeded CIRC_MAX_HSDESC_KILOBYTES: ... > ...
Possible Tor bug, or possible attack if very frequent: Got 1 dropped cell on circ ... (in state GENERAL None; old state None None)
Possible Tor bug, or possible attack if very frequent: Got 2 dropped cell on circ ... (in state GENERAL None; old state None None)
Possible Tor bug, or possible attack if very frequent: Got 3 dropped cell on circ ... (in state GENERAL None; old state None None)
.......... up to 60 .........
What /var/run/tor/log
shows (the first line is very frequent):
[notice] Tried for 120 seconds to get a connection to [scrubbed]:80. Giving up. (waiting for circuit)
[notice] Your network connection speed appears to have changed. Resetting timeout to 60000ms after 18 times and 520 buildtimes.
[notice] We tried for 16 seconds to connect to '[scrubbed]' using exit ..... at ..... Retrying on new circuit.
[warn] Invalid hostname [scrubbed]; rejecting
[notice] Failed to find node for hop #1 of our path. Discarding this circuit.
[warn] Guard ... is failing a very large amount of circuits. Most likely this mean the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 126/247. Use counts are 98/98. 123 circuits completed, 0 were unusable, 0 collapsed, and 10 timed out. For reference, your timeout cutoff is 60 seconds.
My old approach to fix connectivity issues is a cron job that:
1. Connect to one of hidden services (60 ms timeout)
2. Try again (120 ms timeout)
3. If no success, then: systemctl restart tor@default
After Gateway upgrade it’s very frequent that the cron job restarts Tor service.
Is it correct approach to do restart? I did this because Tor itself could not recover from broken all connections for hours and I noticed that restarting Tor usually helps however it takes about 10 minutes to make all connections work again.
Troubleshooting
When host system is under load, then it’s most likely that Tor connections will fail in Whonix-Gateway. Dunno why. High system load may cause response latencies.
If there is some latency, do Tor nodes close/fail circuits with the Gateway?
Also checking RAM (it constantly changes a bit):
MiB Mem: 331.5 total 7.4 free 250.5 used 82.2 buff/cache
MiB Swap: 496.0 total 473.6 free 22.4 used 83.3 buff/cache
When you lower RAM to 256 MB, then no hidden service works.
Now RAM is 384 MB and should be fine. If this is a cause, I will try add more RAM.
What happens after update
- Even more frequent connectivity issues:
- hidden services do not work or randomly stop to work (0xF2 not connected to introduction point or just connection timeout)
- also sometimes no Internet connection in Whonix-Workstation
- very long time to wait for first opening of every hidden service
- Changing guard (not recommended) helps for a while or doesn’t help.
Tor version is 0.4.7.13 and it also shows:
[warn] Tor was compiled with zstd 1.5.2 but is running with zstd 1.5.4. For safety, we'll avoid advanced zstd functionality.
Questions
- Tor 0.4.8 introduced Proof of Work. Should update to resolve the issues?
- Should I pick some specific guard node instead of random? How safe is it?
- What to change in Tor configuration or any related services (vanguards, etc.) to make hidden services respond faster and to fix connectivity issues?
- Is it good approach to automatically restart Tor when connection is dead? Is there a way to force Tor rebuild circuits?
I’m still reading Whonix docs but any ideas to fix issues are appreciated.