Bitcoin Core onion-grater profile

nyxnor · October 3, 2022, 11:55am

Ok. So I don’t believe there is a problem with my onion-grater profile, but something strange happens.

retry add onion after tor's controller closed connection

opened 10:35PM - 02 Oct 22 UTC

Bug

I am running Bitcoind on Qubes-Whonix. Whonix-Gateway is where tor resides, and every tor controller's command made by Whonix-Workstation to Whonix-Gateway is filtered by a proxy called onion-grater (upstream development is made by Tails OS) From this point on I will be referring to tor's controller simply by "control" or "controller". If the control connection is closed by restarting onion-grater to apply new rules (even not regarding the bitcoind.yml) file, bitcoind will remove the current onion and won't be able to set it as address again. ``` RemoveLocal(myonion.onion:8333) tor: No supported authentication method ``` and `bitcoin-cli -netinfo` ``` Local addresses: n/a ``` and of course none of my nodes listening to this address will continue working. ADD_ONION command works the first time but not the second. The problem occurs because bitcoind does not give time for the controller to restart the second time, so the authentication is not "available yet". In fact, tor is not being reloaded or restarted, only the proxy, which is what bitcoind sees as the controller. This happens because I am using bitcoind tor controller feature to send ADD_ONION. If I set the onion service manually on the gateway, I wouldn't have this problem, but then the problem would be: - it wouldn't be pragmatically deleted, although I am rewriting it with `Flag=DiscardPK` anyway - it would stay as a file on the Gateway and the onion address would need to be manually inserted into the bitcoin.conf **Expected behavior** I expect that after bitcoind can't authenticate to the controller, it tries again after some time. This way, I can restart my controller without needing to restart bitcoind for it to get a onion listening address again. **Actual behavior** Bitcoind removes current listening onion hostname and not retry to authenticate to the controller. **To reproduce** This is difficult to reproduce on normal systems. First, Qubes may not interfere, so you could try to reproduce on Whonix KVM or Virtualbox or Tails OS (probably). If you can't test on that setup, I would try to simple close the controller connection on a normal system and then see what bitcoind does to the listening onion address. Also you need to add the bitcoind onion-grater profile, I don't know if Tails has, but for Whonix, I am using https://github.com/Whonix/onion-grater/blob/9addd1d6dd9671b18e5415a196b92d7f6ee5846c/usr/share/doc/onion-grater-merger/examples/40_bitcoind.yml It is not the current Whonix onion-grater profile but it is the correct one and will be merged upstream soon. If that is the case, add that profile to `/usr/local/etc/onion-grater-merger.d/40_bitcoind.yml` Also set in the `bitcoin.conf` the onion binding to `bind=0.0.0.0:8334=onion` and not `127.0.0.1`. This makes the service listen on the Whonix internal interface and does not affect security. Restart bitcoind to apply changes. You will also have to open the port on the Whonix-Workstation firewall. ``` $ sudo mkdir -p /usr/local/etc/whonix_firewall.d $ echo "EXTERNAL_OPEN_PORTS+=" 8334 " | tee -a /usr/local/etc/whonix_firewall.d/50_user.conf` $ sudo whonix_firewall ``` To view onion-grater logs, you need to enable debug mode on the Whonix-Gateway: ``` $ echo "[Service] ## Clear onion-grater default file '/lib/systemd/system/onion-grater.service'. ExecStart= ## Enable debug mode. ExecStart=/usr/lib/onion-grater --listen-interface eth1 --debug " | tee /usr/lib/systemd/system/onion-grater.service.d/50_user.conf $ sudo systemctl daemon-reload $ sudo systemctl restart onion-grater ``` Then after adding the profile, enabling onion-grater debug mode, restarting onion-grater and bitcoind, test if the address is reachable externally. Use the bitcoid option `connect=address.onion`, so you are sure it is connecting. If it is, restart onion-grater. ``` $ sudo systemctl restart onion-grater ``` On the bitcoin debug.logs, you should see the same error I saw above about removing onion address and not able to authenticate to the controller. But that is not entirely correct, because I can issue an ADD_ONION manually and it will accept, but bitcoind does not try to issue ADD_ONION a second time. ``` host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): -> PROTOCOLINFO 1 host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): <- 250-PROTOCOLINFO 1 host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): <- 250-AUTH METHODS=NULL host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): <- 250-VERSION Tor="0.4.7.10" host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): <- 250 OK host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): -> AUTHENTICATE host onion-grater[149562]: 10.137.0.31:54044 (filter: 30_autogenerated): <- 250 OK ``` The controller actually lets bitcoind authenticate, as seen from the onion-grater logs above. But I don't understand why bitcoind says it can't authenticate. The ip doesn't matter, it is internal ip. What you should expect from onion-grater logs is the following: ``` host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): -> AUTHENTICATE host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): <- 250 OK host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): -> ADD_ONION NEW:ED25519-V3 Port=8333,0.0.0.0:8334 host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): rewrote command: host onion-grater[149562]: ADD_ONION NEW:ED25519-V3 Port=8333,0.0.0.0:8334 host onion-grater[149562]: to: host onion-grater[149562]: ADD_ONION NEW:ED25519-V3 Port=8333,10.137.0.31:8334 Flags=DiscardPK host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): <- (multi-line) host onion-grater[149562]: 250-ServiceID=myonionhostnamewithoutonion host onion-grater[149562]: 250 OK host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): -> QUIT host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated): <- 250 closing connection host onion-grater[149562]: 10.137.0.31:43566 (filter: 30_autogenerated) disconnected: client quit ``` You could try on Whonix with `tor-ctrl` to authenticate after restarting onion-grater and it will work: ``` $ tor-ctrl ADD_ONION NEW:ED25519-V3 Port=8333,0.0.0.0:8334 ``` and it will authenticate. But for some unknown reason bitcoind does not reauthenticate to the controller. I understand that once the contoller connection closes, the commands send to the controller are not valid anymore. But bitcoind should be able to reauthenticate once it loses connection. **System information** System information was already described above. I understand this may be difficult debug because it requires a system with onion-grater. But it does not rely on Whonix or Tails or onion-grater package to fix this, as they are only restarting to read the new configuration, but bitcoind understand that as a connection that must be closed forever. If you read all this and is open to make bitcoind retry to authenticate cleanly, I am open to help debug and instruct how to reproduce.

Sometimes when restarting onion-grater, bitcoind gets a ServiceID, sometimes it can’t authenticate although onion-grater replied correctly that it could authenticate.

Patrick · October 3, 2022, 12:15pm

I am not sure upstream will accept the bug report because of the mention of onion-grater. Their position could be “that needs to be fixed in onion-grater than”. If the issue could be re-produced without onion-grater and only with Tor, then the argument be much stronger. Could this be reproduced by simply reloading or restarting Tor?
From my experience, bug reports generally, with such complexity have very low chance of being acted on by upstream.

nyxnor · October 3, 2022, 12:22pm

Unfortunately no. I tested on a debian qube running bitcoind with local tor, this does not occurr.

I tried to make the bug “simpler” but it does not happen without onion-grater. This does not mean its onion-grater fault, because it replies correctly. But maybe bitcoind doesn’t wait for it to reply? Not sure.

Tested on debian by restarting tor, as that is the way to close the control connection as there is no proxy. Bitcoind can always add onion. But on Whonix with onion-grater, it does not always works, don’t know why.

nyxnor · October 3, 2022, 1:22pm

This makes me think that if you want to be a listening node to contribute to the network, you should set listenonion=0 and set the onion serivce manually on the gateway. This way there is no need to worry about onion-grater.

Everything has pros and cons.

Patrick · October 3, 2022, 2:25pm

If there’s an inconsistency between Tor’s and onion-grater’s behavior it could be blamed on onion-grater. Could be demanded to act exactly as Tor.

nyxnor · October 3, 2022, 2:59pm

The problem is onion-grater replies correctly, so I don’t know how to open that issue.
If I use tor-ctrl to add the onion it works, but with onion-grater + bitcoind, sometimes it works, sometimes not, which is pretty weird.

bitcoind tor and onion-grater logs

nyxnor · October 3, 2022, 3:01pm

I mean, maybe someone could try this on Tails, but bitcoin core is not installed by default and they may say “Unsupported” or whatever. But onionshare is installed on Tails, so someone might try to test onionshare and then restart onion-grater and see what happens to the onion.

qubenix · October 9, 2022, 11:00pm

I absolutely do not mind being removed as maintainer, but “did not reply in time” after only 20 days is retarded.

nyxnor · October 10, 2022, 9:06am

It’s been a while since we’ve seen qubenix — their last post was 2 years ago.

First, this is what I see above your post ^^.
So great that you are back, I’ve been following your guide, excellent.
Last post 2 years ago, last time active 1 year ago.

Great, I just did not wanted to be blocked by having to wait a long time for your reply.

s/retarded/not enough time for me
From the point I made the PR was 20 days, compared to now its been a month.

Did I wait a long time, I don’t know, that is relative, how much time do you want me to wait next time? Do you have any complains against the onion-grater profile?

Patrick · October 23, 2022, 10:44pm

Thanks, merged!

Patrick · August 1, 2023, 12:08pm

Patrick · August 1, 2023, 12:20pm

Created minimal documentation just now:
Bitcoin Core - Whonix chapter Tor Control Protocol in Whonix wiki

Patrick · August 9, 2023, 10:29am

unfazed375 · August 10, 2023, 2:36pm

I’ve made a new PR this time changing the onion-grater profile for bitcoind. The problem with previous profile is that it incorrectly assumes that bind=0.0.0.0:8334=onion needs to be used when bind=10.152.152.XX:8334=onion works perfectly fine. This change will allow for bitcoin-cli -netinfo to correctly classify incoming onion connections as onion instead of npr.

I also escaped the dots in the patterns.

https://github.com/Whonix/onion-grater/pull/7

Patrick · August 10, 2023, 3:38pm

Comments?

Patrick · August 22, 2023, 10:25pm

@qubenix are you still available as a maintainer for the Bitcoin Core onion-grater profile?

nyxnor · September 4, 2023, 12:04am

0.0.0.0 works on Qubes-Whonix and Non-Qubes-Whonix, while 10.152.152.X is Non-Qubes-Whonix.

As you allowed both addresses, I’m fine with that.

Untested.

unfazed375 · September 4, 2023, 9:03am

Good point. It would not matter if bitcoind detected those connections correctly as onion connections. It might be possible to match the Qubes IP range likewise.

@Patrick

I found typos in my PR. On each line with \d{1.3}, it should be \d{1,3}