Unable to get VPN Before Tor (User -> VPN -> Tor -> Internet) working, network interfaces are reset

Support
22

Thanks for the help.

Comment out, granted, I meant comment in, of course, sorry about that.

I don’t understand the steps under “Firewall settings”:

First it says to edit: /etc/whonix_firewall.d/50_user.conf
Then it says to edit (when I expand it, I can’t believe I overlooked that): /etc/whonix_firewall.d/30_default.conf

Which file should I edit?

I tried removing /etc/whonix_firewall.d/50_user.conf and instead adding VPN_FIREWALL=1 to /etc/whonix_firewall.d/30_default.conf

Still not working, same problem.

Still working on the logs, sorry this takes a while.

23

This isn’t easy.

If I try just running openvpn manually, it doesn’t work. I get write UDP not allowed. I run it as sudo -u tunnel, then I get the ioctl TUNSETIFF error.

Note that it runs fine with systemctl start.

Then I tried editing the file /lib/systemd/system/openvpn@openvpn.service and adding a --log argument to the openvpn ExecStart line. I reload systemd and try executing openvpn through systemd start openvpn@openvpn.

However, this does nothing (no logging, and systemctl status shows it ran without even giving the --log argument …). I’m not experienced enough with systemd to figure this out right off the bat.

Edits:

I can verify now that systemctl DOES run with the correct argument, and I’m logging under /tmp/logs/openvpn.log, however, there’s nothing there.

I check the process’ file descriptors: /proc/4524/fd, and there’s nothing. No open descriptors to the /tmp/logs/openvpn.log

I can hardly believe the amount of work I have to put in to make this work.

24

I think I’ll just manually add in the capabilities required again with setcap.

Edits:

I need some more advice to make progress I think.

I cannot reproduce the problem when I run openvpn, tor, and sdwdate manually. I have logging working now, but I had to use setcap to manually change permissions, so I’m not sure of the validity of the test either.

As it is now, sdwdate gives:

sdwdate - WARNING - Tor Bootstrap Result: Tor’s Control Port could not be reached.

25

Instead of importing the Whonix images every time you want a clean VM, you can use VM snapshots. Take a snapshot before configuring your VPN. If you want to start over, just revert to the last snapshot.

https://www.howtogeek.com/150258/how-to-save-time-by-using-snapshots-in-virtualbox/

sudo nano /etc/whonix_firewall.d/50_user.conf

Add.

VPN_FIREWALL=1

Next, reload Firewall

sudo whonix_firewall

For you can run the following commads for VPN (debug) The last command will give you the log output. (Does this work for you?)

sudo /usr/sbin/openvpn --rmtun --dev tun0

sudo /usr/sbin/openvpn --mktun --dev tun0 --dev-type tun --user tunnel --group tunnel

cd /etc/openvpn/

sudo -u tunnel openvpn /etc/openvpn/openvpn.conf

Thought it would be a good idea to make sure you are not missing anything and we are on the same page) When you get to the end, or you come across a problem, Stop don’t make any changes. A fix early on can bread something later. Lets debug from that point. Logs will be needed. The commands for VPN debug (above) will be Ok if the VPN does not function. If the VPN is functional move on to Enable Tor (step 9).

For Tor logs.

cat /var/run/tor/log

For sdwdate logs

cat /var/log/sdwdate.log

Note: Reinstalling Whonix is not necessary.But remove any settings not provided in the instructions

1. Firewall settings

sudo nano /etc/whonix_firewall.d/50_user.conf

add

VPN_FIREWALL=1

2. Reload firewall

sudo whonix_firewall

3. Sudoers configuration

sudo nano /etc/sudoers.d/tunnel_unpriv

Add

tunnel ALL=(ALL) NOPASSWD: /bin/ip
tunnel ALL=(ALL) NOPASSWD: /usr/sbin/openvpn *
Defaults:tunnel !requiretty

4. Create VPN secrets file.

sudo nano /etc/openvpn/auth.txt

Add

user_name
password

5. Set up VPN configuration file.

sudo nano /etc/openvpn/openvpn.conf

Add your config file.

6. Set folder permissions

sudo chown -R tunnel:tunnel /etc/openvpn

sudo chown -R tunnel:tunnel /var/run/openvpn

7. Create the OpenVPN systemd service file.

sudo cp /lib/systemd/system/openvpn@.service /lib/systemd/system/openvpn@openvpn.service

Enable the OpenVPN systemd service file.

sudo systemctl enable openvpn@openvpn

Start the OpenVPN systemd service.

sudo systemctl start openvpn@openvpn

Check the OpenVPN systemd service status.

sudo systemctl status openvpn@openvpn

8. Enable Tor

sudo whonixsetup

9. (optional)Force Tor to wait for OpenVPN.

Create a folder /etc/systemd/system/tor.service.d.

sudo mkdir /etc/systemd/system/tor.service.d

Create a file /etc/systemd/system/tor.service.d/50_user.conf.

sudo nano /etc/systemd/system/tor.service.d/50_user.conf

Add

[Unit] After=openvpn.service

1 Like
26

I’m now using snapshots, thanks.

I will stop at the first problem I encounter now.

I might actually be encountering a problem very early on.

Should this now be blocking traffic, since the VPN is not up yet? I can still browse through the Whonix gateway even after reloading the firewall. This doesn’t seem right.

27

Thats OK. Your VPN is not configured yet (no tun0 interface).

Might be of interest : https://github.com/Whonix/whonix-firewall

1 Like
28

I’ve gathered the logs now.

But now I’m faced with a new problem, how to get the logs off of the Whonix gateway.

I thought I would scp them over to a box on my LAN, I apt-get ssh, stop openvpn, stop tor, stop sdwdate, cleaned out all the iptables logs, I verify I can ping the machine, great, but then, when I ssh:

connection refused.

Works fine from other machines, there are no rules in iptables. What is blocking the access?

29

Hi rob75

You can use VirtualBox shared folders. Whonix-Gateway → Host → Whonix-Workstation

https://whonix.org/wiki/VirtualBox/Guest_Additions#For_Whonix_14_and_above

1 Like
30

I have now obtained the logs, thank you for your assistance so far.

Please note that after OpenVPN and Tor has started, I am online, everything works. I know I said this before but I just want to stress it. Again, if sdwdate is masked and never start, my problem never occurs.

I have tried setting the clock manually with date and hwclock --systohc, this doesn’t seem to change anything.

After sdwdate finishes, I get these:

pcnet32 0000:00:03.0 eth0: link up, 100Mbps, full-duplex
pcnet32 0000:00:08.0 eth1: link up, 100Mbps, full-duplex

But they are not present in the logs, they are in dmesg and written to the console.

OpenVPN log: Sat Nov 24 15:26:11 2018 OpenVPN 2.4.0 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] - Pastebin.com
Tor log: Nov 24 15:25:17.000 [notice] Tor 0.3.2.10 (git-0edaa32732ec8930) opening new log - Pastebin.com
sdwdate log: 2018-11-24 15:25:58 - /usr/bin/whonix-gateway-firewall - OK: Skipping firewall m - Pastebin.com

I’ve allowed it to run for a while after it fails.

After 15:27:50 is when it fails and I get the pcnet32 dmesg problem.

sdwdate claims that:

2018-11-24 16:37:50 - sdwdate - WARNING - Clock got changed by something other than sdwdate. seconds_to_sleep: 4200 time_delta: 4201

Is it possible the network interfaces are forcibly reset (like ifconfig down/up or similar) when this warning is detected? Why the clock would get changed like, I have no idea.

1 Like
31

rob75:

I’ve gathered the logs now.

But now I’m faced with a new problem, how to get the logs off of the Whonix gateway.

I thought I would scp them over to a box on my LAN, I apt-get ssh, stop openvpn, stop tor, stop sdwdate, cleaned out all the iptables logs, I verify I can ping the machine, great, but then, when I ssh:

connection refused.

Works fine from other machines, there are no rules in iptables. What is blocking the access?

Stream isolation. (See Stream Isolation .)

In short, to deactivate stream isolation, i.e. to prevent forcing
through Tor (needed for local LAN connections) use:

ssh.anondist-orig

Also you most likely need to run under user “clearnet”, otherwise
connection would be blocked by firewall.

sudo -u clearnet ssh.anondist-orig

I wonder if we have this documented in the wiki somewhere? Some wiki
entry to just quickly answer this very question?

1 Like
32

No problem. Thanks for all the efforts with the logs and debugging. When I have something I will let you know. (see if I can reproduce behavior)

1 Like
33

Hi rob75

Do you have a firewall on you host?

Could you please provide VPN logs from when sdwdate is disabled. Meaning disable sdwdate then restart Whonix-Gateway then get the logs. Sorry, I know this is a PITA.

1 Like
34

If by host you mean the Whonix gateway, then not that I’m aware of. It is just a standard Whonix gateway install.

My Whonix gateway is connected to a typical wireless router, my Whonix gateway is on the LAN side of this, so the whonix gateway performs standard NAT as needed.

There are no issues with multiple other systems using the same Internet connection.

I will get the logs you’re asking for.

I will do it by doing systemctl mask sdwdate, mask tor, mask openvpn, then reboot the system.

Then I will start openvpn manually as you described before, and then systemctl unmask and start tor.

35

Here are the new logs.

OpenVPN: Tue Nov 27 19:21:26 2018 OpenVPN 2.4.0 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] - Pastebin.com
Tor: Nov 27 19:21:50.000 [notice] Tor 0.3.2.10 (git-0edaa32732ec8930) opening new log - Pastebin.com

sdwdate is disabled/masked.

Nothing happens. I’ve verified with the Whonix gateway that everything is working as it should.

I let it run for a while just to show that the VPN connection rekeys itself but this is normal behavior.

1 Like
36

Host OS. Such as Ubuntu,Debian, Windows.

Guest OS. Such as Whonix-Gateway,Whonix-Workstation.

1 Like
37

Ah of course, no, there is no firewall on the host.

38

Hi rob75

Just to recap:

1. OpenVPN and Tor connect when Whonix-Gateway first starts.
2. ~ 120 to 240 seconds later you see this in dmesg and at the same time the OpenVPN connection goes down.

pcnet32 0000:00:03.0 eth0: link up, 100Mbps, full-duplex
pcnet32 0000:00:08.0 eth1: link up, 100Mbps, full-duplex

3. You are able to reconnect the VPN but disconnection continues to happen at the same time interval.
4. This problem is remedied by disabling sdwdate

I wasn’t able to produce any of the same behavior until I saw this.

When I run sudo ifconfig I get interesting results.

https://pastebin.com/RXDJ0ynn

While I didn’t produce the exact behavior i.e. No VPN connected issue. This issue continued to be a problem on concurrent VM boots.This might be causing an problem with time sync and/or whonix_firewall.

To be clear, ifconfig shouldn’t cause an issue. But possible running before whonix_firewall is loaded is interfering somehow.

I’ll have to test more.

1 Like
39

Thanks for your patience and taking the time to help me.

Your summary is largely accurate.

1: Yes.

2: Yes.

3: Not really, after the pcnet32 issue, then it doesn’t reconnect, at all to my knowledge. However, if I do systemctl restart for both openvpn@openvpn and tor, a few times (back and forth), then things start working again.

4: Yes.

Ah, this happens regardless of me running ifconfig or not (between reboots and even between reinstalls of Whonix gateway).

40

I want to say thanks for all the help so far.

However, I still have this problem. Should I file a bug report?

Could someone else, perhaps, please verify that what I’m doing works on their system?

I cannot believe that I’m still missing some simple step, which is the cause of all my issues with sdwdate. If I was, and I keep making the same mistake over and over when reinstalling the Whonix gateway, I would say it makes sense to assume others also have made the same mistake as I did.

It is probably in the interest of the community to point out this crucial step (if this is indeed the problem) that I’m making in the documentation so that others can avoid the same problem.

I can imagine that many simply give up when they encounter the problem.

41

Hi rob75

Could you try unloading the Apparmor sdwdate profile and see if that helps. Apparmor profile unload.

List the apparmor profiles

ls /etc/apparmor.d/

You should see output similar to this:

abstractions    local                usr.bin.whonixcheck.dpkg-old
cache           system_tor           usr.lib.onion-grater
disable         tunables             usr.lib.sdwdate.url_to_unixtime
force-complain  usr.bin.whonixcheck  usr.sbin.haveged

Using the file above, this would unload the sdwdate profile.

sudo aa-disable /etc/apparmor.d/user.lib.sdwdate.url_to_unixtime

Now try running VPN with sdwate enabled.