tor-ctrl - Tor control port command line tool

nyxnor · January 16, 2022, 7:30pm

issue is socat on Whonix, it does not occurs on debian.

report here https://github.com/nyxnor/tor-ctrl/issues/9#issuecomment-1013928468

commit here to change dependency s/socat/netcat-openbsd/ debian/control

netcat will be found before socat as the script was designed tor-ctrl/tor-ctrl at ab8f9d2b86f84bdc82402112f5bbe282df43e36e · nyxnor/tor-ctrl · GitHub

Patrick · January 16, 2022, 7:46pm

This seems difficult to debug and fix. Could be an issue with socat and/or onion-grater.

https://github.com/Whonix/anon-gw-anonymizer-config/issues/18

Patrick · February 4, 2022, 10:29am

Would you be interested to take-over maintenance of anon-verify and anon-info? These tools seem quite useful outside of Whonix.

(Ideally these could be renamed. Would making migrating them from Whonix anon-gw-anonymizer-config to another package easier packaging wise. Always hard if another package takes over a file with the exact same file path. In Whonix, I’d add compatibility symlinks.)

Note: Adding tor to project names may or may not be an issue: Reference:

Or tor-ctrl getting a more generic name to allow other utilities being added?

nyxnor · February 4, 2022, 11:43am

Would you be interested to take-over maintenance of anon-verify and anon-info? These tools seem quite useful outside of Whonix.

Anon-verify is great and even though there is a debian file dep on that systemd service, it can be ignored.
Anon-info is just for debian systems, the installation of it can be optional

Before I can say it can be added to tor-ctrl, why it should be merge the interaction of tor conroller with scripts to verify the torrc and print debian sources.

I would be much more inclined to add anon-verify to vitor, as it is related to modifying tor configuration files.

anon-verify is too good to not be on upstream tor package, I don’t blame you for not submitting issues upstream as your account is blocked on gitlab.torproject.org.

Or tor-ctrl getting a more generic name to allow other utilities being added?

suggestions? it is the interaction directly with the tor’s controller, the scripts related to that will need to have a similar name. The repository and therefore the package name can be more generic.
I just like the tor- because it makes easier to find tor related packages

tor                            tor-ctrl-circuit               tor-gencert                    tor-prompt                     
tor-circuit-established-check  tor-ctrl-onion                 torify                         tor-resolve                    
tor-ctrl                       tor-ctrl-stream                tor-print-ed-signing-cert      torsocks

nyxnor · February 4, 2022, 11:51am

latest script tor-ctrl-onion helps:

creating ephemeral onion services, with all flags possible (DiscardPK, AuthV3, MaxStreamsCloseCircuit etc)
deleting those services and also listing their ids.
adding client-side v3 onion authentication credentials, as well as removing them or viewing them

creating a service has two detach methods, from detaching to not detaching.

If not detaching, script will hang and the service will be active until user press Ctrl+C.
if detaching, flag Detach will be used and the service will be active until tor is halted, terminated, quit.

github.com/nyxnor/tor-ctrl

tor-ctrl-onion-client: explore `ONION_CLIENT_AUTH_{ADD,REMOVE,VIEW}`

opened 02:41PM - 03 Feb 22 UTC

closed 10:34PM - 03 Feb 22 UTC

nyxnor

- ONION_CLIENT_AUTH_ADD - https://gitweb.torproject.org/torspec.git/tree/control…-spec.txt#n1990 - ONION_CLIENT_AUTH_REMOVE - https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2033 - ONION_CLIENT_AUTH_VIEW - https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2051 It is not the same key format used inside `ClientOnionAuthDir/file.auth_private`, as it is in base32 and the controller requires it to be in base 64. Read [TPO Client-Auth](https://community.torproject.org/onion-services/advanced/client-auth/) To generate a key, you need `openssl` (not libressl as it does not contains alg x25519) and `basez`, as it contains base32 and bas64 scripts to encode and decode. ## Create certificate: ```sh openssl genpkey -algorithm x25519 -out /tmp/k1.prv.pem ``` ## Generate base32 keys: **private key:** ```sh grep -v " PRIVATE KEY" /tmp/k1.prv.pem | base64pem -d | tail -c 32 | base32 | tr -d "=" > /tmp/k1.prv.key.base32 ``` **public key:** ```sh openssl pkey -in /tmp/k1.prv.pem -pubout | grep -v " PUBLIC KEY" | base64pem -d | tail -c 32 | base32 | tr -d "=" > /tmp/k1.pub.key.base32 ``` # Keys ## Generate base64 keys: **private key:** ```sh grep -v " PRIVATE KEY" /tmp/k1.prv.pem | base64pem -d | tail -c 32 | base64 | tr -d "=" > /tmp/k1.prv.key.base64 ``` **public key:** ```sh openssl pkey -in /tmp/k1.prv.pem -pubout | grep -v " PUBLIC KEY" | base64pem -d | tail -c 32 | base64 | tr -d "=" > /tmp/k1.pub.key.base64 ``` ## Convert key from base64 to base32: **private key:** ```sh base64 -d /tmp/k1.prv.key.base64 2>/dev/null | base32 | tr -d "=" > /tmp/k1.prv.key.base32 ``` **public key:** ```sh base64 -d /tmp/k1.pub.key.base64 2>/dev/null | base32 | tr -d "=" > /tmp/k1.pub.key.base32 ``` ## Convert key from base32 to base64: **private key:** ```sh printf '%s====\n' "$(cat /tmp/k1.prv.key.base32)" | base32 -d | base64 | tr -d "=" > /tmp/k1.prv.key.base64 ``` **public key:** ```sh printf '%s====\n' "$(cat /tmp/k1.pub.key.base32)" | base32 -d | base64 | tr -d "=" > /tmp/k1.pub.key.base64 ``` # Syntax `serviceId` is the `hostname` without `.onion`. `serviceId=HSAddress` ## ONION_CLIENT_AUTH_ADD `serviceId x25519:privateKeyInBase64` `Flags=Permanent` - This client's credentials should be stored in the filesystem. If this is not set, the client's credentials are ephemeral and stored in memory. ## ONION_CLIENT_AUTH_REMOVE `serviceId` ## ONION_CLIENT_AUTH_VIEW `[serviceId]` Tells the connected Tor to list all the stored client-side v3 client auth credentials for "HSAddress". If no "HSAddress" is provided, list all the stored client-side v3 client auth credentials.

the controller only accept keys in base64, while the configuration files can be in base32, so the issue above guides to convert keys from one to another (test before trusting, I’ve done by hand.)

Patrick · February 4, 2022, 1:02pm

Sure.

I don’t like anon- prefix instead of tor- prefix myself. Just couldn’t come up with a better idea, though. It’s not really whonix- specific either.

tor- obviously the most convenient, best and easiest to discover naming prefix but I don’t know if trademark issues could become an issue. (The Tor Project Trademark versus TorBOX / Whonix)

According to Tor Project | Trademark

Can I use the word “Tor” as part of the name of my product or my domain name?

Please don’t use Tor in your product name or domain name. Instead, find a name that will accurately identify your products or services. Remember that our goal is to make sure that people aren’t confused about whether your product or project is made or endorsed by The Tor Project. Creating a new brand that incorporates the Tor brand is likely to lead to confusion, and commercial confusion is a sign of trademark infringement.

Patrick · February 20, 2022, 3:43pm

github.com/nyxnor/tor-ctrl

tor-observe - tool to see to which destinations Tor is connecting to

opened 03:42PM - 20 Feb 22 UTC

closed 06:16PM - 21 Feb 22 UTC

adrelanos

Sometimes users request a feature to observe connections between Whonix-Workstat…ion and Whonix-Gateway. But it's not really limited to Whonix. Also users that are using Tor Browser would like to check what connections Tor is actually creating. (related: https://github.com/nyxnor/tor-ctrl/issues/27) In https://github.com/nyxnor/tor-ctrl/issues/13#issuecomment-1017007037 a Tor Browser regression was found, Tor Browser (Firefox) phoning home to firefox.settings.services.mozilla.com: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/31575 Which you then reported at: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40788 Including some potential other unwanted connections. There might be many more such privacy violations. But these are difficult to spot when these are routed over Tor. Therefore it would be useful to have some tool that users can run which shows everything where Tor is resolving DNS and connecting to. * Output should be safe to copy/paste into public places such as mailing lists, forums, IRC without de-anonymizing oneself. Therefore Tor entry guard, middle relay and exit relay and even onion rendezvous relay should be omitted. * Duplicates might often be not helpful, but sometimes helpful to see the how many there really are or guess what is causing it? Default probably best to not show duplicates? * Simple to start monitoring by just staring 1 utility? * Similar to tor-ctrl-stream but simpler and OK for copy/paste? * If connecting to IPs, get the reverse DNS of the IP and add it to the script output to make it easier for the user to find out what kind of connection it is? * Live. Show connections when they are created. Better usability than having to press another button to actually see what happened. * Not sure if there is a way to find out which application was responsible for the traffic?

Patrick · February 20, 2022, 3:58pm

The Tor Project feature request:

Patrick · February 20, 2022, 11:10pm

Patrick · February 20, 2022, 11:10pm

github.com/nyxnor/tor-ctrl

tor-observe - tool to see to which destinations Tor is connecting to

opened 03:42PM - 20 Feb 22 UTC

closed 06:16PM - 21 Feb 22 UTC

adrelanos

Sometimes users request a feature to observe connections between Whonix-Workstat…ion and Whonix-Gateway. But it's not really limited to Whonix. Also users that are using Tor Browser would like to check what connections Tor is actually creating. (related: https://github.com/nyxnor/tor-ctrl/issues/27) In https://github.com/nyxnor/tor-ctrl/issues/13#issuecomment-1017007037 a Tor Browser regression was found, Tor Browser (Firefox) phoning home to firefox.settings.services.mozilla.com: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/31575 Which you then reported at: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40788 Including some potential other unwanted connections. There might be many more such privacy violations. But these are difficult to spot when these are routed over Tor. Therefore it would be useful to have some tool that users can run which shows everything where Tor is resolving DNS and connecting to. * Output should be safe to copy/paste into public places such as mailing lists, forums, IRC without de-anonymizing oneself. Therefore Tor entry guard, middle relay and exit relay and even onion rendezvous relay should be omitted. * Duplicates might often be not helpful, but sometimes helpful to see the how many there really are or guess what is causing it? Default probably best to not show duplicates? * Simple to start monitoring by just staring 1 utility? * Similar to tor-ctrl-stream but simpler and OK for copy/paste? * If connecting to IPs, get the reverse DNS of the IP and add it to the script output to make it easier for the user to find out what kind of connection it is? * Live. Show connections when they are created. Better usability than having to press another button to actually see what happened. * Not sure if there is a way to find out which application was responsible for the traffic?

nyxnor · February 22, 2022, 11:34pm

Move tor-ctrl from deprecated section to currently used.
I tried editing the wiki, but couldn’t find how to edit the whole page in a stance, would need to make two separate edits apparently.

Note: Vidalia and tor-ctrl have been deprecated and are no longer packaged in Debian.

Deprecated

nyxnor · February 23, 2022, 12:11am

I was thinking of using tor-ctrl SETEVENTS GUARD but figuring out exactly what to do on what events would require a lot of testing.

TPO does not document that on purpose, not only how to react to guards, but how to interact with the controller. They don’t teach you how to that for whatever reason they restrict people who has spent hours reading the code to make the interaction with the controller works. The specs control.spec.txt does not works for this because it doesn’t teach what the tor events means, it is just sent and up to you to decide.

For GUARDS reaction, there is GitHub - mikeperry-tor/vanguards: Vanguards help guard you from getting vanned..., but the debian package is almost 3 years old now.

Would be good to document all the guards important events.

Raspberry Pi TorBox log_check_config.py and log_check.py is an event based reaction to tor logs, it can restart tor or drop guards. The same could be done with the controller.

I was thinking, if we could aggregate enough info on tor guards, circuits events, we could take a better action towards a better connection.

BTW, on my experience, if I see guards problem for a long time that couldn’t make a connection even after restarting tor, I send the command tor-ctrl DROPGUARDS, to change guards. They seem to want to remove this option and I am not advising anyone to use it, just saying. If one day, they remove dropguards commands, it would still be possible to change guards by deleting some files on DataDirectory.

nyxnor · February 23, 2022, 12:21am

The Tor Project feature request:

Already posted here on the correct thread:

Patrick · February 23, 2022, 11:05am

That is very much OK. We’re accepting even 10 tiny edits each just fixing a single letter typo. There aren’t requirements such as “pretty git history” or “good edit comment”.

For tor-ctrl-observe I was even wondering about creating a dedicated wiki page. Anything appropriate to maximize usefulness for users.

Great!
Yeah. Ideally would be nicely documented. Soon tor-ctrl will be pre-installed. I forgot adding it in this point release but it’s done already in git.

There is a arrow down symbol. Called “supermenu”.

Direct link to edit all just in case not easy to find:

https://www.whonix.org/w/index.php?title=Tor%20Controller&action=edit

Please let me know if there are other wiki edit questions.

Would simplify Manual Rotation of Tor Guards.

nyxnor · February 23, 2022, 12:21pm

Thanks, found the supermenu.

For tor-ctrl-observe I was even wondering about creating a dedicated wiki page. Anything appropriate to maximize usefulness for users.

Made the edit but yet, it feels like needing a separate wiki page.

nyxnor · February 23, 2022, 3:51pm

tor-ctrl should probably have an onion-grater profile if anyone wants to create services from the Workstation.
Different than onionshare, target ports are not limited, so don’t know how to limit the profile with this.

I don’t have any reason to limit target port, it is ok for onionshare as it is designed around a one click installation and the server is configured for you. But for tor-ctrl, you probably will configure your server and you need to specify the target address that will be listened to by the webserver.

github.com

Whonix/onion-grater/blob/master/usr/share/doc/onion-grater-merger/examples/40_onionshare.yml

## Copyright (C) 2016 - 2022 ENCRYPTED SUPPORT LP <adrelanos@whonix.org>
## See the file COPYING for copying conditions

#### meta start
#### project Whonix
#### category tor-control
#### description
## Shipped but not enabled by default onion-grater profile.
#### meta end

## tested with onionshare v0.9.2-67-g12acb89
## tested with tor 0.2.9.8-2~d80.jessie+1

## Tor's reply ClientAuth
## 250-ClientAuth=onionshare:xxxKEYBLOBxxxxxxxxxxxx
## is left intact because onionshare's button 'Copy HidServAuth' coppies the
## following into the clipboard, which is a usability feature.
## HidServAuth xxxdomainxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.onion xxxKEYBLOBxxxxxxxxxxxx
## Otherwise the user would have to extract it from onion-grater
## logs.

This file has been truncated. show original

Patrick · February 24, 2022, 11:29am

A general tor-ctrl profile would be difficult. In that case, one could also set onion-grater to complain mode which lets through everything.

I don’t see the use case yet. Adding onion services isn’t a frequent task?
Anyhow. This would be contributions welcome. Not trivial to implement but easy on my side, it 1 more example profile in the onion-grater-merger examples folder.

Patrick · February 26, 2022, 9:18am

Created an empty tor-ctrl-observer - Tor Connection Destination Viewer just now. (The page name can certainly be modified/moved if there are better suggestions.)

nyxnor · March 18, 2022, 12:17pm

I was looking tor command line tools and found carml.
It is very nice the monitor command, the log messages are easy to read not raw, even though you can still see raw with events command.
Carml is built upon txtorcon, both by Meejah.

Unfortunately carml does not have a debian package, but txtorcon the library has, so possible that in the feature he provides pushes it to debian.org.

interesting commands:

monitor → beautify events
onion → create onion services
newid → newnym
cmd → any controller command

Thoughts:

Much better than tor-ctrl of course in terms of user friendliness, interpreting events, has a whole library.
Not available on deb.debian.org so one would need to maintain the package. Has some dependencies not available on debian repos such as ansicolors.

It is a great tool and I hope it becomes available on debian soon, but it it on version 21.1.0, so maybe there is something blocking.

Docs

https://carml.readthedocs.io/en/latest/

Patrick · June 19, 2022, 9:32am

github.com/nyxnor/tor-ctrl

messages improvement

opened 09:32AM - 19 Jun 22 UTC

closed 05:15PM - 19 Jun 22 UTC

adrelanos

tor-ctrl-observer > tor-ctrl-observer: [info]: subscribed to Tor stream even…ts, as soon as streams are created, output will be shown below. Good. > tor-ctrl: [notice] connecting to socket with command: nc -U /run/tor/control Good. > attempting to connect to [1] attempting to connect to... To what? And did it succeed? I guess so. But still a bit confusing. > stop listening with Ctrl+C Maybe rather write `To stop listening and show a summary, please Ctrl+C.` ? User could further think, yeah, sure, with Ctrl+C I can exit this. Thanks for the tip. But instead, I could also just close the window. But that's wrong, the Ctrl+C will actually show additional information. > connecting to socket with connecting to socket with... With what? And did it succeed? I guess so. But still a bit confusing. Similar to [1].