Securing/Removing DHCPClient from GW (was: Netstat Gateway log)

[phabricator-migrator]

Information

ID: 559
PHID: PHID-TASK-ggfz4xqvowllgusur5w2
Author: HulaHoop
Status at Migration Time: resolved
Priority at Migration Time: Normal

Description

On the GW the netstat -tulpen output shows dhclient as working on all interfaces including the internal network. This is very bad especially since its not just listening. The only place where dhclient makes sense is the outer NIC of the GW where its a trusted NAT network that assigns dynamic addresses - however it should never be looking at the internal network for anything.

Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode       PID/Program name

udp        0      0 0.0.0.0:16151           0.0.0.0:*                           0          11238       858/dhclient    
udp        0      0 0.0.0.0:68              0.0.0.0:*                           0          11262       858/dhclient    
udp        0      0 10.152.152.10:5300      0.0.0.0:*                           0          18054       3435/tor        
udp6       0      0 ::: (sanitized*)                    :::*                                0          11239                  
            858/dhclient 

*Sanitized since I am not familiar with IPv6 addresses

The Tor UDP connection is unusual too. Any idea what that is about?

Comments

---

Patrick

2016-09-23 15:38:28 UTC

Tor port 5300 is easily answered. It’s Tor’s DnsPort.

anon-gw-anonymizer-config/usr/share/tor/tor-service-defaults-torrc.anondist at 49ce21f97965609e4fe06af20005637878324fcc · Whonix/anon-gw-anonymizer-config · GitHub

---

dhclient is a dependency of #anon-gw-dhcp-conf.

https://github.com/Whonix/anon-gw-dhcp-conf/blob/b4999b1a6f79c280afbdecc521a1a05feb84526a/debian/control#L17

It is used to obtain a lan ip address for eth0 from the virtualizer. It should not be reachable from Whonix-Workstations since it is blocked by Whonix-Gateway firewall.

Anyhow. It would be useful to make dhclient listen on eh0 only. Can you figure out how to configure isc-dhcp-client that way? We could also switch the dhcp client while we are at it.

---

HulaHoop

2016-09-23 17:52:39 UTC

Perhaps its not a problem:

Even though netstat shows dhcpd has a socket open that is listening, but
it only processes dhcp requests on the named interface.

You can also use the local-address statement, but beware the special requirements.

How to configure dhcp to listen to a specific interface instead of 0.0.0.0?

However a couple of replies down:

I’ve found that version 3.0.4-13+etch2 (debian) listens and steals traffic not intended for it on interfaces which it is configured to not use.

How to configure dhcp to listen to a specific interface instead of 0.0.0.0?

I don’t know much to disprove it either way but as long as it only listens (firewall takes care of that) and doesn’t attempt talking to a dhcp server on the WS, it should be OK.

---

An alternative could be the pydhcplib in Debian:

http://pydhcplib.tuxfamily.org/pmwiki/index.php?n=Site.OptionsList

However I’m leaning towards keeping everything as it is since you seem to have paid great attention to blocking DNS resolution setting using script hooks, and changing to something else might need more effort to reach the same safety level.

---

Patrick

2016-09-23 18:20:52 UTC

Even a functional /etc/resolv.conf on Whonix-Gateway would not be scary. As long as there is no local Tor DnsPort, it wont work.

The listener on all interfaces is more scary.

How is dhclient called? → dhcp - which program invokes dhclient on debian squeeze? - Stack Overflow

The way it is called influences our options to replace it.

And DHCP involves raw packages that circumvent iptables. Which is even more scary. A difficult topic.

https://mailman.boum.org/pipermail/tails-dev/2014-December/007553.html

---

Patrick

2016-09-23 18:20:58 UTC

---

Patrick

2016-09-24 13:33:55 UTC

A question I thought that @HulaHoop asked. (Because I confused myself in T239#10445.)

Why not remove the DHCP client on Whonix-Gateway to reduce attack surface?

Good question. Has long not been reconsidered. In past perhaps because of physical isolation, but that would not count anymore nowadays.

Virtualizer specific. Some comments here:

whonix-gw-network-conf/etc/network/interfaces.d/30_non-qubes-whonix at master · Whonix/whonix-gw-network-conf · GitHub

• VirtualBox: as per comments, seams doable.
• KVM: no idea
• Qubes: does not need this

So if I make it work for VirtualBox, I wouldn’t know if the same config would work for KVM. In case of VirtualBox, I don’t remember if it caused conflicts with real LAN, other VMs or cloned Whonix-Gateways.

@Patrick:

I don’t remember if it caused conflicts with real LAN, other VMs or cloned Whonix-Gateways.

@HulaHoop:

What kind of conflicts?

Let’s suppose we no longer had a DHCP client installed and used static IPs.

VirtualBox:

That this is non-trivial, is mentioned here:
Tutorial - Guest Static IP with NAT networking mode - virtualbox.org

• Whonix-Gateway 1 using static IP: 10.0.2.15
• Whonix-Gateway 2 using static IP: 10.0.2.15
• Some non-Whonix VM using NAT, getting IP 10.0.2.15 from VirtualBox DHCP server.

Would they conflict within the VirtualBox NAT network? That has to be tested.

Des VirtualBox static networking work reliable? That has to be tested.

Could you fool the Gateway filtering rules with spoofed IPs?

No. Imagine several physically isolated Whonix-Gateways behind the same physical router. Whonix-Workstations still cannot connect to other Whonix-Gateway which they are not directly connected to or otherwise circumvent them.

---

HulaHoop

2016-09-25 02:11:21 UTC

The way it is called influences our options to replace it.

Here is some similar info on what invokes it and how to change it:

[all variants] Change the default dhcp client

Do you know what happens if no clients listed in ifupdown are installed?

Good question. Has long not been reconsidered. In past perhaps because of physical isolation, but that would not count anymore nowadays.

The effect it has on different hypervisors and how to change it is very hard to know and might cause even more usability friction. I’m for replacing dhclient with something else that we can configure to more sensible defaults.

---

Choices:

• The pydhcplib package contains the client pydhcp that can be called from commandline to parse DHCP packets the way we would like. Seems to have the ability to alter interfaces on demand. Let me know if this is useful for our purposes:

http://manpages.ubuntu.com/manpages/precise/man8/pydhcp.8.html

“Option up tells pydhcp to set the network interface up if
not. noup tells not to set up the interface. up and noup are only
useful in combination with the device type. Default is noup.”

• Debian -- Error

Very barebones busybox implementation. Listed in ifup list as one of the options it cycles thru if dhclient is not available.

---

Patrick

2016-09-25 03:53:32 UTC

! In T559#10454, @HulaHoop wrote:
Do you know what happens if no clients listed in ifupdown are installed?

Didn’t test. Would speculate, no DHCP, no connectivity then.

The effect it has on different hypervisors and how to change it is very hard to know and might cause even more usability friction. I’m for replacing dhclient with something else that we can configure to more sensible defaults.

Yes, that would be great. Ideally written in a memory safe language. And ideally neither listening for nor sending raw packages.

Far best solution and to be sorted out first however is a static network configuration without any dhcp. Seems less effort to me than reinventing a new dhcp solution.

---

HulaHoop

2016-09-25 05:05:17 UTC

Confirmed. You are right that it uses raw sockets which do bypass iptables. For the record Shellshock directly affected dhcpclient. I think something this dangerous should be ripped out then questions asked later if it doesn’t completely break things.

[QP] raw sockets & iptables – DiabloHorn

Potential solution is a “derooted DHCPclient that does not need CAP_NET_RAW”

Likely possible with systemd because it can whitelist CAPS. Not sure if it can be run non-root however.

Apparmor can block raw socket access. AFAIK the version in Debian cannot control network access yet?

Its worst than I thought. This behavior is part of an RFC so likely this is how other clients might behave… There is pushback when someone suggested to change this:

https://kb.isc.org/article/AA-00378/0/Why-does-DHCP-use-raw-sockets.html

---

You can indirectly disable raw sockets by removing the raw cap with an Apparmor profile for dhcpclient.

---

HulaHoop

2016-09-25 12:19:57 UTC

To check if this measure works show all raw sockets on system:

c - check all socket opened in linux OS - Stack Overflow

cat /proc/net/raw

---

Patrick

2016-09-26 18:52:06 UTC

Switched to static network configuration.

• https://github.com/Whonix/anon-gw-dhcp-conf/commit/ce5b8e0c4e09f24379a09748524a5fb58812170c

• switch to static network configuration for external interface eth0 · Whonix/whonix-gw-network-conf@670b928 · GitHub

• Works with VirtualBox.

• Likely breaks KVM. A solution for that has to be found. @HulaHoop

• Will make #physical_isolation configuration more difficult.

---

HulaHoop

2016-09-26 22:57:29 UTC

With libvirt a user can create another NAT network besides the default - with the same IP range. So another GW would have its own dedicated NAT without conflicts.

---

Patrick

2016-09-26 23:21:52 UTC

Can you emulate these changes, use that static IP? What will need changes? KVM documentation?

---

HulaHoop

2016-09-27 03:29:37 UTC

Can you emulate these changes, use that static IP?

Tested. Not working but thats expected since the ip range on the “default” network supports 192.168.122.2 - 192.168.122.254

What will need changes? KVM documentation?

Yes. I’ll have to add steps for creating new NAT networks. I hope we can ship a static IP that works for both instead of adding another network creation step to KVM whonix default GW-WS install pair.

Works with VirtualBox.

By working you mean in multi-GW usecase too?

---

Patrick

2016-09-27 14:28:40 UTC

By working you mean in multi-GW usecase too?

Yes.

---

HulaHoop

2016-09-27 15:59:19 UTC

So can we move to something static in the 192.168.122.2 - 192.168.122.254 range (depends on VBox choking or not) or should I include another network file with the whonix-libvirt package?

---

Patrick

2016-09-27 22:19:44 UTC

192... will be a huge generator of FUD “conflicts with my router”. Long time ago we moved away from that exactly for that reason.

Shipping a network configuration file in whonix-libvirt won’t work, since that would break VirtualBox.

Remember Non-Qubes-Whonix 13.0.0.1.0 X issues? Relevant summary:

• downloadable builds created by me
• raw image contains both VirtualBox and KVM packages
• therefore it was not possible to ship a KVM specific gui config package and not break VirtualBox at the same time

Same here. Unless you find a way to active that network configuration file when it detects being run inside KVM only. You’d also have to conditionally - at run time - hide whonix-gw-network-conf /etc/network/interfaces.d/30_non-qubes-whonix. Very complex and hacky.

Changing VirtualBox static IP range may be possible, but would break connectivity for all users who do Whonix 13 → Whonix 14 upgrades and they’d have to run some obscure commands on the host to fix this. Would be a support hell.

---

HulaHoop

2016-09-28 13:58:23 UTC

My mistake I was not clear. By network configuration I mean yet another XML to create a new separate network as an alternative to “default” (like how I do it now with whonix internal network for KVM). It has nothing to do with GW files at all. No changes have to be made there.

Now my only question is: What is the subnet range you would like me to include for that custom network?

---

Patrick

2016-09-28 15:10:41 UTC

Same as VirtualBox.

whonix-gw-network-conf/etc/network/interfaces.d/30_non-qubes-whonix at 670b928231327fc858e4cc588c5be58d281b4f25 · Whonix/whonix-gw-network-conf · GitHub

It has to work if Whonix-Gateway tries to use this static configuration:

auto eth0
iface eth0 inet static
        address 10.0.2.15
        netmask 255.255.255.0
        gateway 10.0.2.2

---

HulaHoop

2016-09-28 16:44:55 UTC

What I meant was subnet range using the CIDR calculator:

IP Calculator / IP Subnetting

Network: 10.0.2.0/24 00001010.00000000.00000010 .00000000 (Class A)
Broadcast: 10.0.2.255 00001010.00000000.00000010 .11111111
HostMin: 10.0.2.1 00001010.00000000.00000010 .00000001
HostMax: 10.0.2.254 00001010.00000000.00000010 .11111110
Hosts/Net: 254

I created a test network with 10.0.2.0/24 and no matter what I did the GW could not connect.

I enabled broadcasts in interfaces.d, disabled and enabled the test network DHCP with no result. So the current code will completey break KVM Whonix connectivity unfortunately.

---

The only sane solution IMO is to reinstate dhcp-client while wrapping it in an apparmor profile that forbids raw socket creation by denying the cap. That way iptables can blocks connections to it on internal-net.

---

Patrick

2016-09-28 17:02:59 UTC

I doubt it is possible to successfully use a dhcp client with raw sockets disabled. It may be possible to develop such a thing in theory, but I don’t think it exists.

Scratch the “dhcp kvm” search term. The proper search term is “kvm static network configuration”. Start with a Debian VM, then port the working solution to Whonix for debugging purposes.

dhcp - KVM/libvirt: How to configure static guest IP addresses on the virtualisation host - Server Fault

---

HulaHoop

2016-09-28 23:42:31 UTC

dhcp - KVM/libvirt: How to configure static guest IP addresses on the virtualisation host - Server Fault

These steps were not needed at all. Once I selected non-conflicting settings everything worked. Some changes to the netmask and gateway will need to be made to interfaces.d

This works:

address 10.0.2.15
netmask 255.255.252.0
gateway 10.0.0.1

---

netmask calculator:

IP Calculator / IP Subnetting

Address:   10.0.2.15             00001010.00000000.000000 10.00001111
Netmask:   255.255.252.0 = 22    11111111.11111111.111111 00.00000000
Wildcard:  0.0.3.255             00000000.00000000.000000 11.11111111
=>
Network:   10.0.0.0/22           00001010.00000000.000000 00.00000000 (Class A)
Broadcast: 10.0.3.255            00001010.00000000.000000 11.11111111
HostMin:   10.0.0.1              00001010.00000000.000000 00.00000001
HostMax:   10.0.3.254            00001010.00000000.000000 11.11111110
Hosts/Net: 1022 

I compared it to internal network’s netmask to make sure no conflicts:

Address:   10.152.152.10         00001010.10011000.10 011000.00001010
Netmask:   255.255.192.0 = 18    11111111.11111111.11 000000.00000000
Wildcard:  0.0.63.255            00000000.00000000.00 111111.11111111
=>
Network:   10.152.128.0/18       00001010.10011000.10 000000.00000000 (Class A)
Broadcast: 10.152.191.255        00001010.10011000.10 111111.11111111
HostMin:   10.152.128.1          00001010.10011000.10 000000.00000001
HostMax:   10.152.191.254        00001010.10011000.10 111111.11111110
Hosts/Net: 16382                 (Private Internet)

---

The nat network settings that worked. What other name do you suggest besides “test”?

<network>
  <name>test</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr2' stp='on' delay='0'/>
  <domain name='test'/>
  <ip address='10.0.0.1' netmask='255.255.252.0'>
    <dhcp>
      <range start='10.0.2.0' end='10.0.3.254'/>
    </dhcp>
  </ip>
</network>

---

HulaHoop

2016-09-28 23:44:58 UTC

Various documentation changes:

Add import steps for new network

Change the side by side instructions to account for static addresses and vibr numbers.

Include a warning somewhere on the multi-WS pages against using the same gateway for different WS trust-levels

---

Patrick

2016-09-29 00:28:18 UTC

! In T559#10494, @HulaHoop wrote:
address 10.0.2.15
netmask 255.255.252.0

This works for VirtualBox.

gateway 10.0.0.1

This breaks VirtualBox.

What other name do you suggest besides “test”?

Whonix-Gateway? Perhaps even more specifically, it being for the external network interface.

---

Patrick

2016-09-29 00:31:23 UTC

Looks like libvirt supports a gateway= keyword. Does that work?

---

HulaHoop

2016-09-29 00:50:50 UTC

Looks like libvirt supports a gateway= keyword. Does that work?

No. Its applicable to static routing networks not NAT, however I will test with another subnet range. It will likely work but the choice of 10.0.2.15 must be changed.

---

HulaHoop

2016-09-29 01:10:55 UTC

10.0.2.0/24

Works:

address 10.0.2.128
netmask 255.255.255.0
gateway 10.0.2.1


<network>
  <name>nat</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr2' stp='on' delay='0'/>
  <domain name='nat'/>
  <ip address='10.0.2.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.2.128' end='10.0.2.254'/>
    </dhcp>
  </ip>

---

Patrick

2016-09-29 03:17:08 UTC

! In T559#10500, @HulaHoop wrote:
address 10.0.2.128
netmask 255.255.255.0

Works with VirtualBox.

gateway 10.0.2.1

Changing the gateway IP does not work for VirtualBox.

---

HulaHoop

2016-09-29 12:42:09 UTC

Then we have reached an impasse because nothing I can put in the network configuration can change the gateway IP. Its not KVM’s fault as its the norm to have gateway IPs of x.x.x.1 for a given subnet. Because some idiot on the VBox team chose .2 compatibility is impossible.

The only network type where it can be changed is in static routing but it would require users to change their physical LAN settings to run Whonix - not gonna happen.

---

HulaHoop

2016-09-29 13:20:52 UTC

A very ugly hack:

Use a realtime network packet editor like netsed to change any outgoing packet ip destination headed for 10.0.2.2 to 10.0.2.1.

netsed uses scapy which uses raw sockets but at least you can bind it to a single interface instead of being forced to have this on all interfaces as with dhcpclient.

https://packages.debian.org/jessie/netsed

---

Patrick

2016-09-29 17:33:08 UTC

Can you redirect these packages using route? (Try in a Debian VM first to exclude Whonix firewall from interfering.)

---

HulaHoop

2016-09-29 20:41:35 UTC

Can you redirect these packages using route? (Try in a Debian VM first to exclude Whonix firewall from interfering.)

Thought about it some more.

My last idea doesn’t make sense since it will do this for both versions so it will still break VBox.

I see no easy solution to this…

---

Patrick

2016-09-29 21:03:29 UTC

We’re using ConditionVirtualization=kvm elsewhere already.
(shared-folder-help systemd unit file) Should be doable to reuse it for
the route command also.

---

HulaHoop

2016-09-30 03:19:17 UTC

We’re using ConditionVirtualization=kvm elsewhere already.(shared-folder-help systemd unit file) Should be doable to reuse it for the route command also.

Since this is an option how about shipping a 30_non-qubes-vbox or just 30_vbox and a 30_kvm and have a service that conditiobally activates one or the other based on detecting the vm type - it can do this by deleting the unnecessary file upon boot. This makes sure that even if the package updates reintroduce the file it gets cleaned up again.

The advantage is I can include a static ip config in 30_kvm that just works with the default network without needing yet more steps to setup kvm whonix.

---

Tested that multiple GWs using the same static ip works. No documentation changes needed for that.

---

Patrick

2016-09-30 13:24:19 UTC

Seems like an awful hack. Last resort. If it somehow by some update (by ifupdown) is run after ifupdown, it breaks connectivity.

What about ConditionVirtualization=kvm + route?

---

HulaHoop

2016-09-30 14:58:10 UTC

OK I will try route but I need some help with commands.

Is it possible to overwrite everything to be in the 192.168.122.0/24 subnet?

---

Patrick

2016-09-30 15:05:42 UTC

No idea. But we should probably stay on the subnet we have.

---

HulaHoop

2016-09-30 19:36:46 UTC

Great news! This config works without hacks. You can keep 10.0.2.15 unchanged too. Turns out the gateway ip address was just called “ip address”…

<network>
  <name>external</name>
  <forward mode='nat'/>
  <bridge name='virbr1' stp='on' delay='0'/>
  <ip address='10.0.2.2' netmask='255.255.255.0'/>
</network>

Must change Whonix (internal) to virbr2

Thinking about changing naming of network files to internal/external to be more clear.

subnet range not conflicting:

Address:   10.0.2.0              00001010.00000000.00000010 .00000000
Netmask:   255.255.255.0 = 24    11111111.11111111.11111111 .00000000
Wildcard:  0.0.0.255             00000000.00000000.00000000 .11111111
=>
Network:   10.0.2.0/24           00001010.00000000.00000010 .00000000 (Class A)
Broadcast: 10.0.2.255            00001010.00000000.00000010 .11111111
HostMin:   10.0.2.1              00001010.00000000.00000010 .00000001
HostMax:   10.0.2.254            00001010.00000000.00000010 .11111110
Hosts/Net: 254  

---

Patrick

2016-09-30 21:24:21 UTC

That’s great! So whonix-gw-network-conf/etc/network/interfaces.d/30_non-qubes-whonix at master · Whonix/whonix-gw-network-conf · GitHub can stay as is?

But existing KVM users need to change their config after upgrading to Whonix 14?

---

HulaHoop

2016-10-01 03:10:48 UTC

Yes it can stay as it is.

But existing KVM users need to change their config after upgrading to Whonix 14?

With Whonix 14 there will be no direct upgrade path on KVM at least since the interfaces.d will break connectivity. Also with 64bit compatibility this means the repo paths have changed.

Not a big problem as long as we warn people about it. The inevitable support threads can be redirected to a sticky topic.

---

Patrick

2016-10-01 15:31:16 UTC

Also with 64bit compatibility this means the repo paths have changed.

No such issue. 32 bit users do not have to do anything. Just users who
download Whonix 14 images will be 64 bit users. Users who used Whonix 13
or before that was 32 bit can continue to use that. This is because
Whonix packages are platform _all.

State of offical 64 bit builds - #16 by Patrick

At some point however when Debian kills 32 bit, they will have to
download a new image. Not sure if it will be possible to upgrade i386 in
place to amd64 or if that would result in a too huge mess.