Hello, I would like to allow certain traffic between my “normal” Qubes AppVMs and the Whonix Workstation (as installed by the Qubes installer).
I’ve already configured everything in all the other AppVMs as well as the sys-firewall. My iptables-rules in sys-firewall are very strict, so unless someone finds a vulnerability inside iptables, I should be save here.
However, the Whonix Gateway seems to not forward any traffic. So far, I tried the following:
sysctl -w net.ipv4.ip_forward=1
iptables -I FORWARD -s 10.137.0.0/16 -d 10.137.0.0/16 -j ACCEPT
It is intentional that the iptables-rule is lax here, because I want to leave all the complicated stuff up to the sys-firewall. So what I would like to do on Whonix Gateway is essentially enabling forwarding of all traffic to and from all other Qubes.
Please help me.
whonix39184:
sysctl -w net.ipv4.ip_forward=1
You really shouldn’t do that. Using that opens a can of worms. In
Whonix, we’re really lucky we don’t need net.ipv4.ip_forward=1 and you
should keep it that way.
Tunneling through qrexec is the way to go. Please see if you find
documentation for that at the Qubes website or mailing lists. Otherwise
please ask on the Qubes mailing list.
I have read all the information I could find regarding this topic on the whonix-page as well as on the tor-project-page. I accept the decision of the development team that they do not want to have unsecure communication as a feature and in 99% of all cases this makes really sense to me.
A few years ago I decided to go from Windows to Linux. The main reason for this was that I didn’t want to accept anymore that my operating system is not under my control, but essentially Microsoft is deciding what I can or cannot do with the system (to a certain extent, often there are workarounds). So, in one word, the reason was that I wanted freedom. The freedom to do with my system what I want. Along with this decision came of course the responsibility.
To me it makes sense that every workflow that I have on a regular basis manifests itself as a mounting/network-structure. I use read-only mounts, file permissions, some chroot-jails and so on. I’m trying to make everything as secure as possible without making my workflows much more inefficient. Copy & Paste makes everything inefficient, because I cannot automate it. I have tons of scripts that I was working for months now to port them all to Qubes OS. (I also deleted and re-wrote some of course.)
When I enabled networking on my VMs I decided to trust in iptables to secure it. No matter how you configure your system, you always have to trust in some software to secure it.
Maybe you think this is a bad idea. Maybe you have arguments against it which are nowhere on any wiki-page I’ve read. If this is the case, then do it. Make good counter-arguments! But please, don’t speak of a “can of worms” without giving any explanation.
IP forwarding can result in obscure IP leak bugs. Without IP forwarding, the setup is much simpler. Without IP forwarding, the workstation either talks directly to Tor on the gateway or the gateway’s iptables rules redirect traffic to either Tor’s TransPort or Tor’s DnsPort. By using IP forwarding you are going into unchartered waters.
With qrexec you were given a fine approach to establish networking tunnel between VMs. Freedom, anonymity and security is all covered by that. Why not use it?
Okay, good point. If that is really the case then it would be wise in my opinion to place this information to a place where it is visible. This way I cannot verify if this is just your opinion or a fact. You are a mod at this forum which implies some trustworthiness and your argument sounds logical to me, but still, I cannot find it anywhere in the wiki. Is it really such an obscure idea that someone wants to allow internal networking between vms?
Hm, I’m googeling for qrexec without finding anything. I also found no man-pages regarding that. Maybe I’ve overlooked something…
Perhaps you are talking about qvm-run? Yeah, that allows me to pass stdin, stdout and stderr, which technically allows networking. But at least to me it seems, it is a bit of a workaround. The problem are again the “tons of scripts” I was talking about. Some of these rely on a local IP network.
As I am not able to re-write all the software I use inside my scripts, I would have to talk a high-level-protocol like http over IP over stdin/stdout. I’m sure that this is possible somehow, but I don’t know how and I don’t see the real reason for doing that. In particular, I don’t see any security benefit in using stdin/stdout as an additional layer below IP.
It gets even more complicated in some scenarios where more than two clients have to talk to each other in a defined way. I’ve had a really hard time the last few months setting all this up. Please don’t tell me that I did everything wrong. (I felt really cool setting up all the complex iptables-stuff and automating even the setup itself. ;))
Back to Whonix: If we assume that all you were saying is true, would it be okay to setup an additional ethernet-card to the Whonix Workstation? This way I could connect the Whonix Workstation directly to the sys-firewall. (I’m no expert in the TOR infrastructure unfortunately.)
As a last idea, I could give up and use a regular TOR Browser Bundle (inside its own vm of course). I think it would be cool if I don’t have to do that…
It might be possible to use qrexec to emulate a network connection. I heard this in discussions, but I never tried it myself, because well, I didn’t need it yet and due to time restraints. Now I remember, it may have been a combination of qrexec and socat. If I remember right, socat can create a TCP listener and forward stdin/stdout and then on the other endpoint forward it to another TCP listener.
Btw I am also the funder of Whonix. It’s wise to assume anyone could be lying indeed. Of course, this forum or my account could be hacked or whatnot. What you’re trying to do is quite advanced, more for developer type of users. You may take my suggestion of researching qrexec and/or asking on the Qubes mailing list for more input or not. And then verify.
Right, it’s not on the wiki. I’d love to see that question asked on documented on either the Qubes website or Whonix wiki.
What you’re trying in context of Whonix is rare indeed. That’s why you probably need to ask on the Qubes mailling list.
No, I am not talking about qvm-run. Btw qvm-run is based on qrexec.
You find qrexec:
- search term…
site:qubes-os.org qrexec - or search in Qubes mailing lists for qrexec
An additional Ethernet card might work, (I don’t know off hand how to do that with Qubes) but then you’d need to hack Whonix firewall, so I still think qrexec is king here.
Whoopsie, I was not expecting that. Now that I know it, I see you on the about-page. So, 100% (okay, 99,9%) trust from now on. I am indeed a software-developer and I’ve studied computer science with main topic network security, yes.
Yeah, I was expecting a workaround like that, but I don’t want to use such ugly things if there is not a really good reason for doing so. Thanks anyway for the suggestion.
I must have overlooked the qrexec-page on Qubes OS wiki (twice actually). But it is not easy to find it if you google it quick.
That’s why you probably need to ask on the Qubes mailling list.
Nope, I know how to setup a network between vms. I just don’t know how Whonix is configured. I believe, I’m in the right forum. Networking works fine so far. And of course, even if the solution from my first post would have worked, I would have asked nevertheless if this is okay from the perspective of Whonix. Nothing is worse than circumventing security by accident and a lack of information.
As far as I see it, qrexec 1) only allows passing stdin/stdout which does not satisfy me (as already explained) 2) always runs through dom0 which is even worse than every IP-based networking, because IP-traffic “only” runs through sys-firewall. So theoretically, if a vm is able to use qrexec at will, it could compromise dom0.
And even worse, qrexec is - as far as I see it - much harder to control if you have fully automatic processes going on (no user-interaction, no okay-clicking). In the IP-world you can limit your traffic to certain IP-addresses and ports and you can limit the permissions of the process listening on a port to a minimum. This way it should not be easy to escalate your priviledges across vms. Furthermore, IP-implementations are heavily used and therefore we can assume that they are more robust and well tested than a (in comparison) rare solution like qrexec. (I don’t want to be mean to your baby, just telling the truth.)
Don’t misunderstand me: For the use-case for which qrexec is built, it’s nice. I assume that most users of Qubes OS have no problem at all, clicking an “okay”-button each time they want to do a certain operation across vms. For me - as an example - it was a nice way to copy all the prepared config-files into their destination. And it’s still a nice way to boot the whole system using a script in dom0, which relies on scripts in each vm.
But if communication between your vms is essential and happens 24/7, it’s maybe not the right solution.
Yeah, I also don’t know how to setup this in Qubes OS, but I’m expecting that it is possible, will work and I’m able to research how to do it myself. It was just an idea I came up with and I must state that I’m surprised that you see no problems in doing that.
Maybe you’ve misunderstood me: I was not suggesting to put this ethernet-card to the Whonix Gateway, but to the workstation! So this way I’m completely circumventing your firewall - at least for the internal networking stuff. Maybe you understand now that I was surprised about your calm reaction.
P.S.: Maybe I’ll ask the Qubes OS team if I’m allowed to re-write their networking-across-vms-page. I’ve read this suggestion between the lines in your comment. We’ll see.
I am sure the Qubes mailing list is a better place. You can quote me that I’ve sent you there since ip forwarding should stay disabled in Whonix-Gateway. We actually discussed this at 33c3 meeting.
The thing is, there are people much more knowledgeable on qrexec than me. I actually even know that HW42 would be one who could give a good answer.
qrexec does not always go though dom0. All Qubes tools such as qvm-run or qvm-copy-to-vm or copy and paste are based on qrexec. So if you don’t like qrexec, then you don’t like Qubes which is based on it. qrexec is an invention by Qubes.
qrexec can be made unattended. (No more manual user input required.) By using Qubes RPC policy. What and how often qrexec throws confirmation dialogs is configured in dom0 /etc/qubes-rpc/policy.
Here is an example (that Marek created for me) how to ssh into dom0.
Dev/Qubes - Whonix
With modifications I am sure it’s possible to also connect between VMs.
Okay, understood. I’ll do that. Thank you for your time.