Testers wanted! New FIN ACK / RST ACK Leak Test

[html]

Mike Perry recently discovered a leak bug in custom transparent proxies (not related to Whonix!) and published his findings on the tor-talk mailing list:

https://lists.torproject.org/pipermail/tor-talk/2014-March/032503.html

This leak test has been adapted for Whonix and documented here:

https://www.whonix.org/wiki/Dev/Leak_Tests#FIN_ACK_.2F_RST_ACK_-_Leak_Test

Fortunately, I wasn’t able to reproduce this leak using Whonix. Probably because the Linux version Whonix is using isn’t affected by this bug and/or because Whonix’s Firewall uses iptables default policy drop for input-, output-, fowardchain and only allows the Tor user to establish external connections. However, other users using different host operating systems and setups than I should repeat the test.

Please feel encouraged,

– to comprehend the original thread on the tor-talk mailing list

– verify yourself that this leak test doesn’t find a leak and share your results

– check if upstream (Linux kernel / iptables) consider this a bug and if it has already been reported (this is not clear yet)


[/html]

Bad news. Tried this with KVM and there is a leak. Here is what I did. I install and run tcpdump on gateway. Run the socket script in workstation and it shows the connections in tcpdump logs which is fine. Then I turn off Tor on gateway. After that I close the connection on workstation, but it still shows communication going on in the tcpdump log, although Tor is disabled beforehand.

Patrick, any comments or ideas on fixing this?

That depends on the exact commands you were running and the logs.

Does it show connections to that IP when you run tcpdump on the host?

This time I followed the instructions exactly and ran tcpdump on the host instead of the gateway vm and I can still see connections to the google ip after Tor is shutdown.

How dangerous is this bug, can it be fixed by us or is it exclusively upstream’s to take care of?

I really don’t know how I can safely send the logs or if they need to be scrubbed to preserve anonymity.

How did the host tcpdump know that I was making a connection to Google, if its supposed to be routed through Tor first?

Are you sure you didn’t have a clearnet connection in a clearnet host browser to google for a a while?

Can you reliably reproduce this leak doing this leak test?

Having the host reliably see the connection to google ip even though it originated from Whonix-Workstation would be awful.

If you still can confirm, I would be wondering why I couldn’t reproduce this bug using VirtualBox, but you can using KVM.

At least as dangerous as dns leak, I think.

Apparently we could use a workaround. Starting points:

Would require adjustment to work with Whonix, unfortunately.

Perhaps because tcpdump saw it and because there maybe really is a leak.

Are you sure you didn't have a clearnet connection in a clearnet host browser to google for a a while?
Yes. I don't use the host for browsing.
Can you reliably reproduce this leak doing this leak test?
I have managed to reproduce it everytime. Did it twice on the host to confirm that I was getting valid results.
Having the host reliably see the connection to google ip even though it originated from Whonix-Workstation would be awful.
Its quite unsettling to see that, its our equivalent of Heartbleed to say the least. What I think is the most dangerous is the false sense of security this provided. But its good to catch it now than later or never.
If you still can confirm, I would be wondering why I couldn't reproduce this bug using VirtualBox, but you can using KVM.
KVM relies on iptables for everything concerning its networking setup and isolation. VirtualBox uses their own networking driers AFAIK.

Really bad, looks like a new todo item for KVM.

I see. That could be a bug in KVM as well as in Linux. And perhaps a workaround in Whonix’s firewall could solve it.

To my knowledge, no one reported this bug yet against Linux, which is really bad.

To my knowledge, no one reported this bug yet against KVM, looks like even reporting a bug that upstream can reproduce with minimal steps will be difficult.

After discussing this with the Tor devs on irc, they believe that the workaround is sufficient and should be tested thoroughly before judging this as an upstream bug.

Edit:
I’m thinking, maybe we should switch to using nftables and run it through a battery of leaktests. Its new architecture is surely an improvement over what we have now.

* Now talking on #tor * Topic for #tor is: Welcome to #tor, the discussion channel for Tor users and operators | https://www.torproject.org/ | https://www.torproject.org/docs/faq | https://tor.stackexchange.com/ | Developer discussion on #tor-dev | It's 'Tor', not 'TOR' | non-tor talk => #nottor * Topic for #tor set by ChanServ!services@services.oftc.net at Sun Feb 2 13:56:09 2014 * Riastradh (~riastradh@82JAAD1W2.tor-irc.dnsbl.oftc.net) has joined #tor I am working on the Whonix KVM port and ran the leaktest for this bug reported here: https://lists.torproject.org/pipermail/tor-talk/2014-March/032503.html It turns out that this issue applies to us too and its an upstream bug i believe you upstream meaning what -- the kernel? iptables? I am thinking iptables You guys can probably get to them easier than we can is there a patch? * drtor (~drtor@9YYAAK2T4.tor-irc.dnsbl.oftc.net) has joined #tor We are looking at some workaround in iptables but are not sure how well this will work but one thing for sure is it needs to be fixed upstream because it affects any Linux platform that uses Tor as a transproxy AFAIK Patrick -Whoix leader- doesn't write kernel level code https://www.whonix.org/forum/index.php/topic,243.0.html * realitygaps has quit (Remote host closed the connection) According to my findings, the leak can be seen in tcpdump running on the host, as direct communication to Google without going through any Tor node. Its a catastrophe by all means a kernel patch is not necessarily appropriate, and upstream might well not consider it a bug the iptables workaround i gave is in a followup post from mike in that thread it would appear that "it needs to be fixed upstream" is just false i only skimmed that forum page quickly, and i really don't want to wade through every bit of it it contains my workaround based on --state INVALID That's fine we will apply your advice, but are you sure that there is nothing fundamentally wrong with the kernel or iptables though? but it doesn't clearly seem to have been used apply it and test, a lot there is not yet any solid case made that there is something "fundamentally wrong" using iptables to redirect for transproxying relies on connection tracking i think the conclusion is "it is hard to use these things safely" which isn't really a bug in upstream so much as a challenge with the approach you have to test people don't test thoroughly they just read a man page or get some cookbook approach from some post, and run with it that is why something like this goes unnoticed for so long (gee, sounds like some other notable problems ...) Sorry to throw another link your way, but so far this is what we do when testing: https://www.whonix.org/wiki/Dev/Leak_Tests#FIN_ACK_.2F_RST_ACK_-_Leak_Test Any advice if this is enough or anything we should add to ensure its safe after a workaround? * quiliro has quit (Quit: Leaving.) do violent tests with your networking connection You are very sharp btw. That bug was a great find interrupt it and restore it at various moments, for various periods * blumenkraft (~eristisk@a1.networktest.34sp.com) has joined #tor well actually i never announced or reported the problem, so even though i fixed the problem for myself a couple years ago, mike appropriately gets credit for finding it. i just get credit for a better workaround. * amiller has quit (Remote host closed the connection) y'all need to understand that connection tracking was never intended to be more than a loose fit around categories of traffic, not some leakproof box do you know if this applies to nftables too? Its gradually becoming the successor to iptables and was added in 3.13 when the kernel's info on a connection is lost or dropped (timeout), it's gone forever, so subsequent traffic won't be redirected i haven't used it and don't know. unless there's been some massively radical re-architecting of connection tracking, it's unlikely to be totally different, though there could be minor differences in behavior. the tails approach is starting to look a bit smarter * eristisk has quit (Ping timeout: 480 seconds) tails is robus but it has to secure an entire rich os stack to protect the user approach to torifying the traffic i mean ok i don't argue that wrapping stuff in vm's can make it harder for an attacker to break out in that tails doesn't rely on transproxy redirection, yes it's better what specifics are you talking about for torifying maybe we can replicate that? but torifying outgoing streams is harder to make safe than explicit proxy settings * darkclown (~darkclown@p2011-ipbf7306marunouchi.tokyo.ocn.ne.jp) has joined #tor * anong0 has quit (Ping timeout: 480 seconds) right, we've always warned that transproxying is not as great as it seems "set your proxy settings to use tor, only allow tor to use the network" * Gentlecat has quit (Ping timeout: 480 seconds) so, implicitly, "don't allow anything that isn't configured to use tor to use the network" and most people just ignore the warnings about transproxying, because "just stuffing everything through tor" seems just what they want, and so simple. but not really. * mdik is now known as Guest6988 * mdik (~mdik@brln-4d0c41d4.pool.mediaWays.net) has joined #tor the problem is when a box is rooted the software that is configured to use Tor can do things it shouldn't do but I get your point * Guest6988 has quit (Ping timeout: 480 seconds) its those people who think stuffing flash through a transproxy pipe think their safe that problem only exists for you because you insist on running tor in isolation * huseby has quit (Ping timeout: 480 seconds) "be sure to send unexpected traffic over tor so it can't hurt us" vs "be sure to drop unexpected traffic so it can't hurt us" option b sure seems wiser then a bad guy who breaks in can, at worst, generate traffic that looks expected so it goes through tor Useability wise, just having unique circuit per PID might be reasonable compromise. * TheCthulhu1 has quit (Remote host closed the connection) i think that's a totally separate topic arma: an important goal we are trying to achieve is for operators of hidden services to be able to have a fighting chance in case they are targeted. Trying to educate users "don't throw * at tor, but selectively firewall everything" seems like a losing battle though. it does? screw those users then. better to write software that works well and the ones who want to use the safe software will use it right, it can only be won or lost if you consider it to be a battle. users screw themselves, every day and we can't do much for the ones who choose to do it wrong a better analogy might be leading a horse to water * Rym has quit (Ping timeout: 480 seconds) That sounds like somewhat elitist stance. "You dont deserve privacy because you're dumb" vs "Safe defaults built in." obviously better defaults are better, if they actually exist and are actually safer (I can see how it can go downhill from there if users are babysitted too much, though) but defaults never cover everything, because software is flexible, and people want it to be The less decisions that need to be made in security, the better everyone will be and hence why transproxy is an idea worth working towards so give them only the applications that you know behave well, with those applications configured to use tor correctly that principle is correct but the conclusion about transproxying is wrong letting them pick their own apps and hope they are safe is a recipe for surprises again and again we do, but we can't guarantee that nothing dangerous happens when these apps are compromised unless it *all* goes through Tor Ultimately the problem with transproxy is just that it's difficult to identify the source app. * sd uses assigned source port ranges, but it is far from ideal Kgpg, TorBrowser and xchat come by default even if it all goes through tor, you can't guarantee that nothing dangerous happens, you only have considered the first layer of potential problems updated system and AppArmor are other layers we have that's all very nice and good but against a well funded adversary you must throuw everything in the toolbox and the toolbox itself to protect users actually it's just more and more desperate band-aiding the effort would be much better directed towards thoroughly auditing apps Auditing firefox is out of our league but we have verifiable builds for Whonix Velope, that goal is far from realistic when we're talking huge blobs of code such as web browser or libpurple. it's the true longterm challenge that people get scared away from all the time * mttp_ (~mttp@9YYAAK2WG.tor-irc.dnsbl.oftc.net) has joined #tor and the "community" pays for it dearly with things like heartbleed

Interesting!

[quote=“HulaHoop, post:9, topic:229”]Edit:
I’m thinking, maybe we should switch to using nftables and run it through a battery of leaktests. Its new architecture is surely an improvement over what we have now.[/quote]
nftables is not in Debian yet. Therefore difficult to implement (package, update).

Patrick can you please make this bug a priority? IMO it counts as a security hole and while we don’t recommend the KVM version yet, its for other reasons than such a serious one.

I am still working on splitting Whonix into multiple packages, which should make any form of contribution/cooperation easier.

I was hoping you are working on this.

“Solutions”:

  1. quick one, not really a solution:
    As a stopgap, you could test if deactivating Transparent Proxying (instructions here: Stream Isolation) stops the leak.

  2. quick one, that needs testing:
    I quickly thrown together updated firewall rules. Try replacing /usr/bin/whonix_firewall on Whonix-Gateway with this file:
    https://github.com/Whonix/Whonix/blob/7feecf40a4a41c91e10e18012fa0540ad267d9e0/whonix_gateway/usr/bin/whonix_firewall

Then reload Whonix firewall:
sudo whonix_firewall

Then try again if you can still produce this leak.

I am still working on splitting Whonix into multiple packages, which should make any form of contribution/cooperation easier.

I was hoping you are working on this.

Sounds good take your time. Just wanted to know where we are with respect to this vulnerability and that’s why I mentioned it.

I don’t know how to code anything and so couldn’t do anything to help in that regard. But I did test the first solution you proposed, and found that disabling the transparent proxy does not fix this. The second solution I couldn’t try out as moving data into a vm is an absolute pain in the arse. So I will wait when its pushed as part of an experimental update in Whonix repo.

Have you tested, that transparent proxy really was disabled? When you run whonixcheck, it should tell you, that Tor’s TransPort Test failed.

I would be wondering how you can do this test when transparent proxy is disabled, because then ‘s = socket.create_connection((“74.125.28.104”, 80))’ should not be able to establish an connection at all.

Done. Experimental 8.2.2 uploaded.

sudo whonix_repository

→ developers repository

sudo apt-get update && sudo apt-get --yes dist-upgrade

Have you tested, that transparent proxy really was disabled? When you run whonixcheck, it should tell you, that Tor's TransPort Test failed.

I would be wondering how you can do this test when transparent proxy is disabled, because then ‘s = socket.create_connection((“74.125.28.104”, 80))’ should not be able to establish an connection at all.

I can’t recall if it was this specific error, however there was clearly a message about whonix time sync not being able to connect tot the internet.

The browser wasn’t able to connect I am sure.

Now after . selecting the repo and performing an update, I rebooted then ran the test with the new firewall config file loaded and I still get the same leak result on tcpdump running on the host.

I can't recall if it was this specific error, however there was clearly a message about whonix time sync not being able to connect tot the internet.
The browser wasn't able to connect I am sure.
Then something went wrong. That shouldn't happen. Disabling transparent proxying won't result in timesync / browser no longer being able to connect.
Now after . selecting the repo and performing an update, I rebooted then ran the test with the new firewall config file loaded and I still get the same leak result on tcpdump running on the host.
Do you see the added code from https://github.com/Whonix/Whonix/commit/7feecf40a4a41c91e10e18012fa0540ad267d9e0 in /usr/bin/whonix_firewall?

By the way, reboot should not be required. “sudo /usr/bin/whonix_firewall” should suffice.

If you got the added code from https://github.com/Whonix/Whonix/commit/7feecf40a4a41c91e10e18012fa0540ad267d9e0 and still have this leak…

Since you got the added code in your VM now, feel free to make some experiments.

kdesudo kwrite /usr/bin/whonix_firewall

Cut the new code block. And move it up. Look for.

[code]###########################

IPv4 OUTPUT

###########################[/code]

Which is a few lines above. Then add it right below it. Then reload Whonix’s firewall.

sudo /usr/bin/whonix_firewall

And try again.

Alright I will try experimenting but I need to solve this error first that prevents kwrite from appearing:

It says:

kdesudo: cannot connect to X server :0

[quote=“HulaHoop, post:18, topic:229”]Alright I will try experimenting but I need to solve this error first that prevents kwrite from appearing:

It says:

[quote]
kdesudo: cannot connect to X server :0[/quote][/quote]

You’re not using the graphical gateway? Well, if you’re in terminal, just replace “kdesudo” with “sudo” and replace “kwrite” with the editor of your preferences such as “nano”.

I just wanted to say “edit that file with root rights”. How to get root and which editor to use is irrelevant.