Improve Onion Service Usability by Script / GUI

Development
1

I don’t approve of the attitude of the forum thread starter but this again highlights usability issues.

While usability for onion services has room for improvement, on the other thand this applies to other aspects of Whonix too. Understanding what Whonix is and Installation of Whonix is still something most people fail at. Whonix-Host will help with usability since Whonix becomes a “standard conformant” Linux distribution available as iso. It also helps with security since then host security improvements are easy to deploy as probably few follow all host hardening instructions.

I was thinking about this.

There are some issues with scripts for improved usability. These add a level of mysticism as people start to think the functionality is dependent on the functionality of the scripts. Things the script does not ever then starts to seem impossible. Technical background that was previously simple gets mystified. People stop looking at the actual generated configuration files.

For example for Onion Services Authentication which is probably too difficult to manually set up, I created some scripts:

But in case that breaks in future or doesn’t work in some case, can someone look at the actual config files or even send bug fixes for these scripts?

Also the bug of duplicate parsing of torrc in this thread wasn’t caught by anyone which was caused by anon-connection-wizard modifying Tor config files.

An onion services usability script… Where does it start and where does it end? Should it support just 1 thing. Onion web server? Server on non-standard ports? Onion services authentication? I am wondering what’s the best way to implement this? Related to Focus on low-effort maintainability.

  • Implement declarative? I.e. Tor config file snippets which get copied [or symlinked if that is possible]) once the script runs? That is easier code, easier to review/understand but more limited as only a few use cases could be simplified that way before too many config snippets get hard to manage / too much.
    • sudo apt install anon-gw-onion-server-web - that package shipping the required config snippets? That would allow for an easy/clean implementation but there can’t be thousands of packages supporting all use cases.
    • If going that way, there then could be a companion package sudo apt install anon-ws-onion-server-web.
  • Implement functional? I.e. similar to anon-auth-autogen? Then any non-standard things (non-standard ports, something other than default webserver port) requires the script to be run with a parameter. And this script will become more complex/lengthy since it has to handle these variables.
    • Similar to sudo virtport=80 hsport=80 hsname=hidden_service client=1 anon-auth-autogen.
    • Users would then more or less mindlessly copy/paste such commands from the wiki. But would that effort be useful? If it’s a copy/paste job anyhow, could instead could also just provide one-liners in the wiki. See to use or to not use One-Liners - this is the question. And then the effort for the script could be saved.

As for onion services usability GUI… Well, I regard a GUI a frontend. Before frontend development makes sense, there needs be a backend, the script. If the script isn’t easy to use, then it’s also not easy to write a mostly bug free GUI. By separating backend and frontend also debugging gets easier and hopefully things are also easier to understand.

1 Like
2

I would suggest something basic. Reason being is that different users have different requirements. Creating and hardening a functional onion service takes a lot of time and careful planning. Time that may be taken away from Whonix’s other priorities. Add to that the almost endless permutations of configurations, and soon the task becomes formidable.

2 Likes
3

I have some usability scripts on onionjuggler - Manual

I have not made any release, so it should be considered beta, there is a warning on the readme about it, so use it on a testing machine.

It is platform agnostic and it has a prepared /etc/onionjuggler/debian.conf

It does not have a GUI which would be nice, but it has a TUI (dialog, whiptail).

Features:

  • it modifies a temporary copy of the torrc, which can be set on onionjuggler.conf to an alternative file for Whonix users. Then, it verifies if the modified copy will let tor start, if it doesn’t, it dismiss the copy (remove it) and error out with informative message. This avoids breaking tor. If the config is ok, then the copy is placed on the original file and tor is reloaded.
  • the values are checked if they are valid, such as the service name not having special characters, port being an integer and target not being already in use.
  • onion authentication client and server side (you can generate a key pair or use the private or public part to be inserted in the respective config).
  • webserver - it works with nginx and apache2 and work is being made to work with openbsd-httpd. It requires on nginx and apache2 to have the sites-enabled folder, documented on compatibility.md, but that is already default for debian nginx. The webserver configuration is minimal, as it is not possible to cover all user cases.
  • not only related to onion services, but vanguards installation script. The debian package is too old, from 2019, and I prefer to be running from git.

I can add a debian folder to make it a debian package if needed, but for now, the installation method is with the configure.sh script.

To create a service named terminator, it is as easy as possible:

onionjuggler-cli on -s xmpp -S tcp

But can be as advanced as specifying all the parameters:

onionjuggler-cli on --service xmpp --socket tcp --version 3 --port "5222 5269 5280 5281"

If you use 5222,127.0.0.1:5222, the virtport is the first part and the target is the second, but it is unecessary to specify if mapped to the same port from virtport. The script will insert the target anyway because later it will be checked if the target is already in use by another hs.

Add clients to your onion service:

onionjuggler-cli auth-server --on --service ssh --client bob,jane
#onionjuggler-cli auth-server -n -s ssh -c bob,jane

Add credentials as the client:

onionjuggler-cli auth-client --on --onion <ONION>] [--client-priv-key <CLIENT_PRIV_KEY>]

This command can be improved by copying a file .auth_private as it is done by anon-auth-autogen, so it can be shorter.

Add webserver configuration for your onion:

onionjuggler-cli web --on --service some_service --folder website_folder

List onion services, the configuration lines and all the clients names:

## list all services
onionjuggler-cli list -s @all
## list only specified services
onionjuggler-cli list -s ssh,xmpp
## list but without generating qr code of the hostname
onionjuggler-cli list -s ssh,xmpp --quiet

More options can be seen on the manual or running with -h.

The idea why I made this repo was to be an OnionShare but for data written to disk, so it is a completely different use case.

Questions someone might have:

Why is the script so big?
It does a lot of checking before even attempting to make any changes, and after the changes are done to temporary file, it is checked if it valid, and a lot of checks.

Requirements?
Few: tor grep sed tar openssl basez git python3-stem qrencode whiptail nginx

Tar is only necessary for backups
Basez and Openssl for onion client auth
python3-stem for Vanguards
QRencode is used to show the hostname in qr format
nginx for the webservice but it can be apache2 or I can maybe not make it mandatory

About abstracting from the user to edit configuration files:

  • the modification is printed to the screen and the file that was changed is also printed, so they can check later if they think it is not right
  • modifying manually is error prone
    • no downtime by rejecting invalid configuration before applying them to be used. what good does it make if you don’t know how to edit configuration files correctly, better to do it automatically than manually
    • complete uniformity. Standards are important.
    • graphical interface to help newbies. If they are too afraid, they can use the TUI, or later if someone helps me with GTK or whatever works would be cool.

About taking too much time to develop:

  • to write yes, but it enhanced over the months
  • to maintain, only if the checks fails to get an incorrect parameter
    • there is no known bug on debian
    • on whonix, untested, and probably default to this ip 10.152.152.11 if whonix marker is detected
    • on openbsd, it is lacking the webserver part and vanguards for rc.d
    • on other platforms, untested, because of time.
4

Started with 100 lines of code, but as there were no checks, tor would fail if target was repeated, service name contained special characters and many other failures.

The permutation of configuration

If an user puts hidden services on different tor configuration files, the script will not get erros before applying the configuration, because it would not detected repeated service names or targets. So it needs a singled torrc to be edited.

5

It can now be build as a debian package and has a whonix.conf, plus the target ip being detected if being qubes whonix, non-qubes, and optionally enforcing 127.0.0.1 for services of the gateway. Will test on Whonix soon.

1 Like
6

Still not ready because the webserver should be on the workstation, not on the gateway, but progress is being documented on

1 Like
7

I haven’t read all yet.

Thank you for working on this!

Not sure how that would be best implemented. Both has advantages and disadvantages.

  • A) “Ephemeral” Tor onion services (created through Tor control protocol) have the advantage that no commands on the gateway need to be run except extending onion-grater whitelist. Disadvantage is that the workstation has a copy of the onion service private key. Therefore an onion service running in a compromised workstation could be impersonated.
  • B) Setting up an onion service through Tor config files configuration on the gateway would require two commands. First, setting up the onion service. And second, running corresponding commands to set up the server on the workstation.

Ideally a single command somewhere would all set it up. In Qubes-Whonix it might be possible by using qrexec. Running the command in the gateway and then the gateway asking the workstation for permission to set up the web server through qrexec, to be confirmed through a dom0 question popup, perhaps. For Non-Qubes-Whonix I have no idea how to further improve usability for onion service setup.

related:

1 Like
8

A) Ephemeral

That is one big disadvantage loosing whonix protection of host separation. Also not trivial not only because it would circumvent whonix protection, but because tor’s controller doe not show the port service is running on. Example: setup an ephemeral onion service (tor-ctrl-onion -A -l 80) and try to get any information about it, if I send GETINFO onions/detached or tor-ctrl-onion -L, it only show the hostnames. This means that the webserver script cannot consult the controller to find the port it is running on, the hs pk would need to be saved along side its port for this to work, and a format to be defined. Related:

  • maybe a --permanent flag to be used so tor-ctrl-onion saves the PK and the ports and flags to a file, but every time tor receives and INT signal, the user would need to re add those files to the controller, possibly by a systemd script that send these commands after tor is (re)started.
  • this is the easy but less secure option, but does this usability justify the compromise? I mean, almost non whonix user is already running with their PKs on the same host unless they have enough knowledge to configure a remote host controller, and making whonix onion services vulnerable to attacks that already happen on other systems is not a gain on my view, even though the permanent option can still be done because it might be useful anyway.

B) config files

  • qubes-whonix → qrexec as mentioned above
  • non-qubes-whonix → --web option would need to circumnvent if check if service exists and the ports would need to be provided via command line, and yes, would be one command on the gateway and one on the workstation. GW onionjuggler-cli --activate -s web -p 80, WS onionjuggler-cli --web -s web -n -p 80 -w /var/www/folder --no-check-service. There is no such option yet, would need to be checked if this is viable.

Anyway, the web option is not magic, it is a sample nginx virtualhost configuration file for static webpages, advanced uses will always need to be done manually.

1 Like
9

B) config files

Seems more appropriate in case of Whonix sacrificing a bit usability for better onion service private key protection.

10

@patrick sorry to ping but can the gateway now the workstation qube ip?

I copied the code from anon-gw-anonymizer-config/anon-auth-autogen at c0d9e66dd3d5d84bdce9caeab6d4c1c5624dbfe0 · Whonix/anon-gw-anonymizer-config · GitHub which I thought was supposed to be ran on the GW according to http://www.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Onion_Services#Step_3:_Configure_Onion_Services_Authentication, but the qube ip read would be from the GW, not from the WS.

Edit: I guess not because of multiple GWs and WSs having to be configured per vm way to find out the name of gateway from inside the VM - qubesdb-read /qubes-gateway-name · Issue #4117 · QubesOS/qubes-issues · GitHub and Multiple Whonix-Workstation ™

do it is not possible to get the WS qube IP from the qubesdb-read command, I couldn’t find a way. What remains is using a variable on whonix.conf to set the WS IP, but this also has to be done manually or the user would need to specify the target every time.

Specifying a different domain is only available on dom0

qubesdb-read -d anon-whonix /qubes-ip

but this does not guarantee that the user is using the anon-whonix as the WS.

Well, seems target for qube-whonix is manual job as in http://www.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/wiki/Onion_Services#Step_2:_Edit_Tor_Configuration:
on WS run:

qubesdb-read /qubes-ip

on GW run:

onionjuggler-cli --activate -s web -p 80,{ws_qube_ip}:80

on WS run (target_port=80):

onionjuggler-cli --web -s web -p 80 -n --no-check-service -w /var/www/folder

Now possible to set the WS qube IP address on /etc/onionjuggler/conf.d/*.conf:

tor_hiddenserviceport_target_addr=WS-QUBE-IP-ADDR

But the command line target if specified will overwrite this option.
so after setting the above configuration option, it is possible to create services for the WS on the GW with:

onionjuggler-cli --activate -s web -p 80

and if running qubes whonix and the option tor_hiddenserviceport_target_addr is still 127.0.0.1 and the service is not for the gateway, it will fail with an informative error message to set the WS qube ip address on the configuration or obviously possible to specify on the command line.

1 Like
11

I don’t know if there’s a reliable/sane way for that.

Only that part of most of anon-auth-autogen? Btw most of anon-auth-autogen would be good if upstreamed, different name, and not part of anon-gw-anonymizer-config.

Indeed.

Also I guess if we had ⚓ T947 Qubes-Whonix eth1 static networking that might help.

I doubt that exists.

1 Like
12

Yes, I only copied that part (but removed, as it was not working as explained above because it reads the GW ip, not the WS), the rest, onionjuggler-cli --auth-server and --auth-client already had, except installing from a file.

So onionjuggler-cli will eventually substitute that script if implemented. So upstreaming onionjuggler-cli later would be better.

1 Like
13

The setup currently is very easy. Should opening firewall ports be automated to /usr/local/etc/whonix_firewall.d/45_onionjuggler.conf?

WS

Configure web service:

sudo onionjuggler-cli-web --on -s <MY_SITE> --folder <SITE_FOLDER> -p 80

Allow port from the firewall:

sudo mkdir -p /usr/local/etc/whonix_firewall.d
echo "EXTERNAL_OPEN_PORTS+=\" 80 \"" | sudo tee -a /usr/local/etc/whonix_firewall.d/50_user.conf
sudo whonix_firewall

GW

Configure service:

sudo onionjuggler-cli --activate -s <MY_SITE> -p "80 <WS_QUBE_IP>"
1 Like
14

Yes, sure would be nice. Bonus points for transparently pointing out what systemd changes onionjuggler is applying (where, what kind of changes).

I don’t think there is any convention in Debian or elsewhere how applications that create configs for users should operate as per configuration file location such as in /etc or /var and naming.

45_onionjuggler.conf looks good.

Not sure about the numbering:
config files numbering convention

15

For users or developers?

  • If for users, then I’d print an info about the whonix firewall port that was opened.
  • if for developers, then it would only be on the docs.

Not sure about the numbering:

Should it use 40 like: /usr/local/etc/torrc.d/40_tor_control_panel.conf does?

1 Like
16

I want to print the ws ip and the port allowed, how can I find the WS ip inside the WS for Non-Qubes-Whonix?

for qubes-whonix is qubesdb-read /qubes-ip, but for non-whonix I don’t know how to find its ip.

1 Like
17

For users. Point out the path to the config file too? I think it would be a good idea.

The rationale behind this has been documented just now:

Good comparison. Sounds good.

Probably required parsing ifconfig but requires root afaik but I guess that is available in this situation anyhow. There’s hopefully a number of helpful posts on:

bash how to find out local IP

on stackexchange etc. I found some answers but I never liked the parsing of ifconfig. Was hoping there’s a nicer API or tool for this.

1 Like
18

Ok.
But I’d rather users not change the config file manually because it may break the script.
Anyway, I can document this.
Also, there is a simple way to see configuration files manages by onionjuggler:

$ onionjuggler-cli --getconf

openssl_cmd="openssl"
webserver="nginx"
webserver_conf="/etc/nginx/sites-enabled"
website_dir="/var/www"
vanguards_commit="10942de93f6578f8303f60014f34de2fca345545"
tor_daemon="tor@default"
tor_user="debian-tor"
tor_conf_dir="/usr/local/etc/torrc.d"
tor_conf_user_group="root:root"
tor_conf="/usr/local/etc/torrc.d/40_onionjuggler.conf"
tor_data_dir="/var/lib/tor"
tor_data_dir_services="/var/lib/tor/services"
tor_data_dir_auth="/var/lib/tor/authdir"
tor_control_port="9051"
tor_backup_dir="/var/lib/onionjuggler/backup"

missing the variable for the whonix_firewall conf, but it might be added later because for now it is default to use /usr/local/etc/wonix_firewall.d/40_onionjuggler.conf

I just asked because I thought there was a whonix way.

ip a | grep "inet .* eth0$" | sed "s/.*inet //;s/\/32.*//"

ip r get 1.1.1.1 | grep 1.1.1.1 | sed "s/.*src //;s/ uid .*//"

sudo ifconfig | sed -n "/eth0/,/inet/p" | grep inet| sed "s/.*inet //;s/  netmask .*//"
1 Like
19

Yes. Most users shouldn’t do it. Not supposed to be manually changed. Just useful in case there’s ever a bug then advanced users already get the idea to look into that file and report it.

20

This was done where I could catch on recent commits.

Auth scripts now have some activation options

  • import file (servers/clients)
  • import config (servers/clients)
  • import onion+priv key (clients)
  • import pub key (servers)
  • generate keys (servers/clients)

These options are almost the same, it is just to facilitate for the user, maybe he has the config as a file, or maybe the config is on the clipboard, or maybe the client just sends the server the his pub key or vice verse, or maybe a new key needs to be generated and automatically imported.

Another thing missing but it is a non-blocker for whonix, is removing invalid auth files if tor fails to parse it improve verify_config_tor() parser and safe_edit() file mangement · Issue #66 · nyxnor/onionjuggler · GitHub

Missing separate debian packages, work is being done.

1 Like