Improve apt-get Stream Isolation for Qubes Templates

[entr0py]

Credit to @tasket @rustybird for highlighting the issue. (can’t find the post) Just remembered: https://github.com/rustybird/qubes-updates-cache

1. For Tor users, the remote network has some insight about what’s installed on the same physical computer, because there is no Tor circuit isolation between update requests from different clients. This can be prevented by waiting at least 10 minutes (or sending NEWNYM) between updating different clients. (The same is true of qubes-updates-proxy.)

Desired behavior: apt-get should use new circuits for each client VM and also use new circuits for each repository. In non-Qubes-Whonix, multiple VMs that perform apt-get update use different circuits because of IsolateClientAddr. In Qubes-Whonix, all TemplateVMs share the same circuits for apt-get (due to Update Proxy design?).

Current observed behavior: Circuit A is used to route traffic for all apt-get requests from all TemplateVMs going to vwakviie2ienjx6t.onion. Circuit B is used for all traffic going to sgvtcaew4bxjd7ln.onion. Circuit C is used for all traffic to clearnet repos, including qubes-os.org, whonix.org, and 3rd party repos. These potentially allows for some linkability between TemplateVMs - both by the Exit Node as well as by the repository server.

I don’t know enough about tinyproxy (or http proxies in general) to propose a solution. Some possibilities:

*1. (For non-Qubes-Whonix) Enable IsolateDestAddr for apt-get port. From /usr/share/tor/tor-service-defaults-torrc:

## Operating system updates: apt-get
## Not using IsolateDestAddr IsolateDestPort, because too much
## performance loss, too much load on Tor network and no gain
## in security.
SocksPort 10.137.1.1:9104

This was probably written pre-Qubes. Enabling IsolateDestAddr will cause an extra tor circuit per clearnet repo - not too much load. And no additional load if using .onion repos which are isolated by default. This would fix per-repo isolation but not per-client isolation.

! This doesn’t work for Qubes-Whonix because uwtwrapper is disabled for apt-get. Instead apt-get requests are sent to tinyproxy port 8082 and then sent to Tor via (TransPort???). Can we spin the packet around and have it route through a SocksPort?

*2. (For Qubes-Whonix) Somehow masquerade Source IP so that IsolateClientAddr is activated.

*3. (For Qubes-Whonix) Somehow enable SOCKSAuth so that some category of apt-get request is assigned a random SOCKS password.

[Patrick]

Desired behavior: apt-get should use new circuits for each client VM and also use new circuits for each repository.

Yes.

In Qubes-Whonix, all TemplateVMs share the same circuits for apt-get (due to Update Proxy design?).

If all TemplateVMs use the same sys-whonix as their NetVM, unfortunately yes.

## Operating system updates: apt-get
## Not using IsolateDestAddr IsolateDestPort, because too much
## performance loss, too much load on Tor network and no gain
## in security.
SocksPort 10.137.1.1:9104

This was probably written pre-Qubes.

Was written pre-Qubes indeed.

Enabling IsolateDestAddr will cause an extra tor circuit per clearnet repo - not too much load.

More than that.

When you connect run nslookup ftp.us.debian.org for several times, you get several results. DNS round robin. It’s actually multiple servers. Each apt-get package download is a separate fetch. So it would be one circuit per resolved ftp.us.debian.org IP when multiple packages are downloaded.

I agree we should revisit this. Can you please ask on tor-dev if it’s fine for Whonix to use IsolateDestAddr IsolateDestPort for its apt-get Tor SocksPort?

! This doesn’t work for Qubes-Whonix because uwtwrapper is disabled for apt-get. Instead apt-get requests are sent to tinyproxy port 8082 and then sent to Tor via (TransPort???).

Yes.

Can we spin the packet around and have it route through a SocksPort?

I wouldn’t know how. That would be preferred. However, tinyproxy does not support socks proxy settings, which would be a lot better. It just uses normal networking ("speaks only trans"). It runs under its own linux user account at least which might make things easier.

The question you are asking is generally a hard one. How can one force an arbitrary application that does not support socks proxy settings through a socks proxy. Well, there there are proxifiers / socksifiers such as torsocks (and uwt scripting it). But that probably won’t work with tinyproxy.

sudo su
torsocks /usr/sbin/tinyproxy -d -c /etc/tinyproxy/tinyproxy-updates.conf

/usr/sbin/tinyproxy: Could not create listening socket.

One cannot socksify an application as well as at the same time have it open a listener port using a socksifier.

redsocks might be worth exploring.

(There is some information on redsocks here - although it’s on a different topic you can see the idea - https://www.whonix.org/wiki/Tunnels/Connecting_to_Tor_before_a_proxy#Transparent_Proxying_Method.(

Although all DNS would still go through Tor’s DnsPort. [Might be fingerprintable, because TCP traffic without previous DNS request.]

---

Do you think qrexec based updates proxy will help with this? @marmarek

[marmarek]

Do you think qrexec based updates proxy will help with this? @marmarek

Maybe. Connection to the “proxy” will arrive as qrexec call, stream
there will be “HTTP proxy” protocol. So, if you pass it to some “HTTP
proxy -> socks” filter (instead of tinyproxy), that would work. I think
socat can not do that, but maybe something else can?

You’ll have there name of source VM, so can use that to direct it to
isolated circuits.

[Patrick]

Can you please ask on tor-dev if it’s fine for Whonix to use IsolateDestAddr IsolateDestPort for its apt-get Tor SocksPort?

Talked to Roger at 33c3: No, he prefers us not enabling that.

/usr/sbin/tinyproxy: Could not create listening socket.
One cannot socksify an application as well as at the same time have it open a listener port using a socksifier.

This might be possible in Debian stretch based Whonix 14. torsocks learned AllowOutboundLocalhost 1, which will be enabled by default in Whonix 14 in the uwt package.

[nyxnor]

Any update on stream isolation when updating qubes?

Add SOCKS support

[nyxnor]

I think tinyproxy config needs to change. It needs to bind certain client addresses to certain virtual hosts.

As it is right now, it does not show the client addr,

CONNECT   Jul [487354]: Connect (file descriptor 10): localhost [127.0.0.1]
CONNECT   Jul [487354]: Request (file descriptor 10): GET http://deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/r4.1/vm/dists/bullseye/InRelease HTTP/1.1
INFO      Jul [487354]: No upstream proxy for deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion
INFO      Jul [487354]: opensock: opening connection to deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion:80

I think it is the way Qubes implemented Tinyproxy, as we are connecting directly to a localhost socket, we are localhost in the traffic, it can’t differ clients (different qubes templates).

QubesOS/qubes-core-agent-linux/blob/a90c3568d19026e85959fad1a16fd5cad88845cd/vm-systemd/qubes-updates-proxy-forwarder.socket#L9

Running 2 non-whonix templates and using sys-whonix as update vm:
Circuit reuse because tinyproxy does not pass the client addr.
The DNS request uses localhost ip, while the connection uses the gateway qube local ip.

$ tor-ctrl-stream -z -m

StreamId StreamPurpose StreamTarget StreamClient CircuitId CircuitPurpose
------------------------------------------------------------------------------------------------------
376 USER 10.228.239.209-(2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion:80) 10.137.0.8:40006 252 HS_CLIENT_REND
386 USER 10.240.194.32-(deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion:80) 10.137.0.8:47834 285 HS_CLIENT_REND
387 DNS_REQUEST deb.debian.org-(199.232.138.132:0) 127.0.0.1:57744 153 GENERAL
388 DNS_REQUEST deb.debian.org-([2a04:4e42:62::644]:0) 127.0.0.1:57744 153 GENERAL
389 USER deb.debian.org-(199.232.138.132:443) 10.137.0.8:45936 154 GENERAL
392 USER 10.240.194.32-(deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion:80) 10.137.0.8:47842 285 HS_CLIENT_REND
395 USER 10.240.194.32-(deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion:80) 10.137.0.8:47846 285 HS_CLIENT_REND

[nyxnor]

Stream isolation is fully function for different WS using the same GW.

Template stream isolation is a QubesOS issue that only pass 127.0.0.1 as the client address to sys-whonix tinyproxy.

How to identify which template requested to use the proxy?

Hello,
Our automated spam filter, Akismet, has temporarily hidden your post in How to identify which template requested to use the proxy? for review.
A staff member will review your post soon, and it should appear shortly.
We apologize for the inconvenience.

[Patrick]

To understand my post, prerequisite knowledge from Tor manual:

• SocksPort
• TransPort
• IsolateClientAddr
• IsolateSOCKSAuth

@nyxnor

Any update on stream isolation when updating qubes?

https://github.com/tinyproxy/tinyproxy/issues/40#issuecomment-368326575

Actually, I have to correct my former self. Now, I don’t think tinyproxy now having socks support will help a lot with improving apt-get stream isolation for Qubes Templates. All Qubes Template APT traffic would then end up in the same Tor SocksPort. A tiny improvement over all of that traffic ending up in Tor’s TransPort.

tinyproxy socks proxy support is useful for other things. Other than stream isolation. Namely, Whonix-Gateway firewall could perhaps be be simplified if tinyproxy was configured by the qubes-whonix package to use a Tor SocksPort.

@nyxnor

I think it is the way Qubes implemented Tinyproxy, as we are connecting directly to a localhost socket, we are localhost in the traffic, it can’t differ clients (different qubes templates).

Indeed. From the perspective of tinyproxy, traffic is “suddenly coming from nowhere”, in other words, technically from other VM, through qrexec, arriving at tinyproxy localhost.

tinyproxy receives no client addr. Neither tinyproxy has a concept similar to Tor such as IsolateClientAddr and IsolateSOCKSAuth.

Probably required:

• qrexec feature request: tinyproxy would have to receive client addr which would require some qrexec modification.
• tinyproxy feature request: tinyproxy would have to set a different Tor socks user name for each different client addr. This would result in Tor using a different Tor circuit for each Tor socks user name (IsolateSOCKSAuth) set by tinyproxy.

Not sure that is realistic?

• Qubes: An older unrelated qrexec feature request https://github.com/QubesOS/qubes-issues/issues/5253 didn’t have any activity.
• tinyproxy: How interested would tinyproxy contributors be to implement more granular, finer Tor stream isolation support?
  • A feature request similar to the following:
    • Use different socks user name per account (Tor) (#9213) · Issues · gajim / gajim · GitLab
    • make use of stream isolation (#6359) · Issues · Legacy / Trac · GitLab

Help welcome with submitting feature request welcome and anything else. Otherwise I don’t think any progress will be made here.

The only other alternative that I can see is making Qubes-Whonix Templates networked by default to patch/fix the direct communication between APT and a Tor SocksPort. Otherwise if a proxy (tinyproxy) is wretched in between, granular stream isolation will be difficult to implement.

[nyxnor]

qrexec feature request: tinyproxy would have to receive client addr which would require some qrexec modification.

I think this is the most realistic option because tinyproxy needs to get different clients first, as of know only getting 127.0.0.1.

My proposal is either use tinyproxy on a virtual network so it can receive the virtual network ip from templates and only that (but this may be insecure, idk)

Or Tinyproxy listening to connection on different ports for different templates, but this is cumbersome to configure on template clones and then having to modify its config port.