[HOME] [DOWNLOAD] [DOCS] [BLOG] [SUPPORT] [TIPS] [ISSUES] [Priority Support]

Multiple Whonix workstations that can communicate with each other.


#1

I wish to have a group of VMs attached to a sys-whonix type proxyVM where each of the group is able to communicate with every other member of the group.

I have managed this in Virtualbox (all the VMs on internal network ‘Whonix’ could communicate) but I have had no joy so far in Qubes. I have edited iptables in both sys-whonix and sys-firewall in vain but I hope the solution lies in this direction.

I imagine there isn’t much demand for this configuration but no harm in asking.


#2

It’s not clear where sys-firewall fits into your setup. Are your VMs connected to sys-firewall which is then connected to sys-whonix? or are the VMs connected directly to sys-whonix?

Have you seen this already? (“Enabling networking between two VMs” Section): https://www.qubes-os.org/doc/qubes-firewall/


#3

Thank you entr0py.

The VMs all connect to sys-whonix which then connects to sys-firewall.

I had tried using the iptable rules suggested in [quote=“entr0py, post:2, topic:2524”]
Have you seen this already? (“Enabling networking between two VMs” Section): https://www.qubes-os.org/doc/qubes-firewall/
[/quote] on sys-whonix. As that didn’t help, I used the same rules on sys-firewall. Alas still no joy.

The Qubes installation is not critical so I can and will experiment within reason.


#4

I advice to get inter-VM communication working without involving Whonix first. This helps to prevent any extra obstacles that Whonix might introduce. While doing so, you get access to the specialized Qubes help https://www.qubes-os.org/help/. Once that works for you, replace that same with Whonix and see if that works.


#5

An update.

Using a group of Debian 8 VMs I can achieve communications between the VMs if they are connected directly to sys-firewall and sys-firewall has iptables FORWARD chain rules that explicitly allow this.
This can also be achieved if the VMs are connected to VPNs as long as the client VPN config file is suitably amended and sys-firewall has iptables FORWARD chain rules that explicitly allow this.

Connecting the VMs to sys-whonix however does not allow communication between the VMs even if it has an iptables FORWARD chain rules that explicitly allow this.

I have looked through a verbose listing of the various sys-whonix iptables but have found no dropped packets.

Next step is to set iptables in sys-whonix to allow everything everywhere and see what happens.

Any other suggestions are welcome.


#6

An update.

I attempted to disable or bypass sys-whonix iptables by entering the following commands in the sys-whonix terminal:

sudo iptables -F,
sudo iptables -X,
sudo iptables -t nat -F,
sudo iptables -t nat -X,
sudo iptables -t mangle -F,
sudo iptables -t mangle -X,
sudo iptables -P INPUT ACCEPT,
sudo iptables -P FORWARD ACCEPT,
sudo iptables -P OUTPUT ACCEPT.

After entering the first command, the terminal stopped accepting input and the attached VMs lost connectivity.
I entered the remainder of the above commands using ‘Run command in VM’ from Qubes VM Manager but nothing changed except the terminal became responsive.

I shall reboot sys-whonix and the VMs then repeat these actions.

As ever I welcome all suggestions.


#7

I’ll play with this too - little short on time atm.


#8

#9

Using this script did disable the firewall properly. The VMs have connectivity but not yet interconnectivity.

Next I shall retrace the steps so far.


#10

ps Thank you Patrick for the script.


#11

I am also trying to do this: https://tor.stackexchange.com/questions/13522/how-to-configure-whonix-gateway-for-communication-between-two-local-workstations
I’m disheartened that it apparently does not work.

It sounds like sys-whonix does not route packets the way sys-firewall does.

EDIT: indeed, /proc/sys/net/ipv4/ip_forward is 0 on sys-whonix and 1 on sys-firewall
EDIT: echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward on sys-whonix should resolve your problem. I’m still trying to figure out what firewall rule is filtering my pings


#12

Xloem:

EDIT: indeed, /proc/sys/net/ipv4/ip_forward is 0 on sys-whonix and 1 on sys-firewall

Leave that as is. No need to change.

You need to open ports in the iptables firewall. Figure out how to do
inter-vm communication for non-Whonix AppVMs first. Then you should be
able to replicate that the same way using Qubes-Whonix.


#13

Thanks, Patrick.

Like beaky, I have successfully set up inter-vm communication for non-Whonix AppVMs, following https://www.qubes-os.org/doc/firewall/#enabling-networking-between-two-qubes . However, the same set of steps does not work for Whonix VMs. Pings give no replies. A packet log on the whonix gateway shows no packets.

If I enable ip forwarding on the gateway, the pings do get replies, but the replies are “Packet filtered”. I need to understand it more to find the solution.


#14

qubes-firewall-user-script indeed won’t work. You might have to edit /usr/bin/whonix_firewall (in a persistent way) forward rules.


#15

sys-whonix handles packets like any other Linux distribution - using routing tables and iptables. Perhaps you are referring to Tor’s handling of packets instead. Tor receives traffic through the INPUT filter, wraps it up, then sends it out through the OUTPUT filter. So strictly speaking, Tor is not a FORWARDing process.

Forgot about this thread but indeed, I have played around a bit. Since you are a Qubes-Whonix user, I originally wanted to suggest using a proxyVM as a mini-LAN router. While trivial to set up, it has a major flaw. It doesn’t allow IsolateClientAddr to work because all client IPs are MASQUERADEd - so only partial (weak) stream isolation.

To do what you want, you have to modify Whonix-Gateway templateVM (or use a bind dir?) which sometimes runs the risk of leading to future breakage or leaks. If you’re comfortable with that, here are some examples you can add to /usr/bin/whonix_firewall's IPv4 FORWARD section. Use one of the examples below:

### Enable 2-way inter-VM traffic. All protocols allowed, including ICMP (ping).
$iptables_cmd -I FORWARD 1 -i vif+ -o vif+ -j ACCEPT
### Enable 1-way inter-VM traffic. Limit to specific IPs. TCP only.
$iptables_cmd -I FORWARD 1 -i vif+ -o vif+ -m state --state RELATED,ESTABLISHED -j ACCEPT
$iptables_cmd -I FORWARD 2 -i vif+ -s <source.ip.address> -o vif+ -d <dest.ip.address> -p tcp -j ACCEPT

Also, enable FORWARD filter: [not leak tested with Whonix! Do so at your own risk!]

sudo mv /etc/sysctl.d/ipv4-forward-disable.conf /etc/sysctl.d/ipv4-forward-enable.conf
sudo nano /etc/sysctl.d/ipv4-forward-enable.conf
# Replace '0' with '1'

Reboot.


#16

Not advisable to enable IP forwarding. Not leak tested. Always avoided this because without it, networking is so much simpler. Praise Tor’s DnsPort and TransPort.


#17

:open_mouth: Well, that complicates things… [Are you saying it’s possible to do this without forwarding?] On principle, I would really rather use a separate proxyVM… But I think the only way to do that is to have multiple interfaces from one proxyVM connecting to Whonix-Gateway… (not ready to tackle that one.)


#18

No.


#19

I guess the right way to do this would be to set up a proxy on the gateway for the services / ports I want to use. For my use-case of SSH, I would connect to the gateway on port say 2122, and some process running on the gateway would copy the packets to the other workstation, changing the port to 22. This could be done with socat or netcat running as a service.

EDIT: hrm now I need to figure out how to make exceptions for the normal torification of the connections so that I can contact this local proxy


#20

Better use a socket over qrexec instead.

Here is an example doing ssh through qrexec but different from what you want (inter-vm). Perhaps that helps.

https://www.whonix.org/wiki/Dev/Qubes#ssh_into_Qubes_dom0

There are also qrexec docs on Qubes website.