ps Thank you Patrick for the script.
I am also trying to do this: firewall - How to configure Whonix Gateway for communication between two local Workstations in Qubes? - Tor Stack Exchange
I’m disheartened that it apparently does not work.
It sounds like sys-whonix does not route packets the way sys-firewall does.
EDIT: indeed, /proc/sys/net/ipv4/ip_forward is 0 on sys-whonix and 1 on sys-firewall
EDIT: echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward on sys-whonix should resolve your problem. I’m still trying to figure out what firewall rule is filtering my pings
Xloem:
EDIT: indeed, /proc/sys/net/ipv4/ip_forward is 0 on sys-whonix and 1 on sys-firewall
Leave that as is. No need to change.
You need to open ports in the iptables firewall. Figure out how to do
inter-vm communication for non-Whonix AppVMs first. Then you should be
able to replicate that the same way using Qubes-Whonix.
Thanks, Patrick.
Like beaky, I have successfully set up inter-vm communication for non-Whonix AppVMs, following Firewall | Qubes OS . However, the same set of steps does not work for Whonix VMs. Pings give no replies. A packet log on the whonix gateway shows no packets.
If I enable ip forwarding on the gateway, the pings do get replies, but the replies are “Packet filtered”. I need to understand it more to find the solution.
qubes-firewall-user-script indeed won’t work. You might have to edit /usr/bin/whonix_firewall (in a persistent way) forward rules.
sys-whonix handles packets like any other Linux distribution - using routing tables and iptables. Perhaps you are referring to Tor’s handling of packets instead. Tor receives traffic through the INPUT filter, wraps it up, then sends it out through the OUTPUT filter. So strictly speaking, Tor is not a FORWARDing process.
Forgot about this thread but indeed, I have played around a bit. Since you are a Qubes-Whonix user, I originally wanted to suggest using a proxyVM as a mini-LAN router. While trivial to set up, it has a major flaw. It doesn’t allow IsolateClientAddr to work because all client IPs are MASQUERADEd - so only partial (weak) stream isolation.
To do what you want, you have to modify Whonix-Gateway templateVM (or use a bind dir?) which sometimes runs the risk of leading to future breakage or leaks. If you’re comfortable with that, here are some examples you can add to /usr/bin/whonix_firewall’s IPv4 FORWARD section. Use one of the examples below:
### Enable 2-way inter-VM traffic. All protocols allowed, including ICMP (ping).
$iptables_cmd -I FORWARD 1 -i vif+ -o vif+ -j ACCEPT
### Enable 1-way inter-VM traffic. Limit to specific IPs. TCP only.
$iptables_cmd -I FORWARD 1 -i vif+ -o vif+ -m state --state RELATED,ESTABLISHED -j ACCEPT
$iptables_cmd -I FORWARD 2 -i vif+ -s <source.ip.address> -o vif+ -d <dest.ip.address> -p tcp -j ACCEPT
Also, enable FORWARD filter: [not leak tested with Whonix! Do so at your own risk!]
sudo mv /etc/sysctl.d/ipv4-forward-disable.conf /etc/sysctl.d/ipv4-forward-enable.conf
sudo nano /etc/sysctl.d/ipv4-forward-enable.conf
# Replace '0' with '1'
Reboot.
1 Like
Not advisable to enable IP forwarding. Not leak tested. Always avoided this because without it, networking is so much simpler. Praise Tor’s DnsPort and TransPort.
Well, that complicates things… [Are you saying it’s possible to do this without forwarding?] On principle, I would really rather use a separate proxyVM… But I think the only way to do that is to have multiple interfaces from one proxyVM connecting to Whonix-Gateway… (not ready to tackle that one.)
1 Like
No.
I guess the right way to do this would be to set up a proxy on the gateway for the services / ports I want to use. For my use-case of SSH, I would connect to the gateway on port say 2122, and some process running on the gateway would copy the packets to the other workstation, changing the port to 22. This could be done with socat or netcat running as a service.
EDIT: hrm now I need to figure out how to make exceptions for the normal torification of the connections so that I can contact this local proxy
Better use a socket over qrexec instead.
Here is an example doing ssh through qrexec but different from what you want (inter-vm). Perhaps that helps.
There are also qrexec docs on Qubes website.
Thank you so much for this! I’ve written up the solution as a 3-step process at firewall - How to configure Whonix Gateway for communication between two local Workstations in Qubes? - Tor Stack Exchange .
2 Likes
entr0py, what’s this other problem you speak of?
@xloem Thanks for your interest. Problem is a very low priority pet project: Physical Isolation is back! Qubes-Whonix style - #2 by entr0py. I haven’t begun investigating yet. A good start would be learning how physical switches preserve source IP’s without masquerading an intermediary IP. Don’t know enough about networking in general - perhaps it’s done with ARP tables? The trivial, messy solution would be to assign a separate proxyVM to each WS.
In the past, I’ve used a configuration that is relevant to your original question and also affected by my issue:
Whonix-GW ^^ || || LAN proxyVM ^^ ^ || +---+--------+---------+ || | | | WS-A WS-B WS-C WS-D
Here, A has internet access via Whonix-GW and also LAN access. B,C,D are restricted to LAN-only. Luckily, I only had one client connect to Whonix-GW. [Disclaimer: B,C,D should not be considered non-networked or air-gapped because leaks can happen through a compromised proxyVM or by exploiting WS-A via the LAN to leak through the proxyVM.]
1 Like
I think that’s called Network Address Translation? The linux kernel can do it. ARP tables are for mapping MAC addresses to and from IP addresses.
An important question here is whether Whonix-GW separates streams based on virtual interface or based on IP address. If it’s the former, you’ll need separate internal netvms for each client.
The qrexec solution could be briefly messy if these are different physical boxes. You’d need to invent your own channel to send the process streams over.
1 Like
Talked to Marek at 33c3. In Qubes-Whonix (as opposed to Non-Qubes-Whonix) using direct IP is impossible to have inter VM communication without enabling IP forwarding. However, enabling IP forwarding is strongly discouraged. Therefore use the qrexec based solution.
Ok, this qrexec method is great, but how can I use it for something other than ssh? ssh seems luck to have a ProxyCommand option, but most situations are not so lucky.
For instance say bitcoind is listening on 127.0.0.1:8332 in a dedicated workstation. How can I have curl from another workstation use this qrexec method? The only thing I could come up with so far is using this qrexec + ssh method from and then setting up an ssh listener in /rw/config/rc.local. However this seems hacky.
Qubes specific question. Should be asked at Qubes support as per:
https://forums.whonix.org/t/what-to-post-in-this-qubes-whonix-forum-and-what-not
Kicksecure Forums Usage Instructions, Best Practices and FAQ
Well, whonix-gw TemplateVM is the default TemplateVM for sys-whonix which is a TemplateBased ProxyVM.
I did not use whonix-ws TemplateVM as ProxyVM. whonixcheck might complain but you could ignore that. Feel free to experiment with it.
We have that documented.
Connecting to Tor before a VPN
Dangerous? → Combining Tunnels with Tor applies.
Qubes specific. → https://forums.whonix.org/t/what-to-post-in-this-qubes-whonix-forum-and-what-not