torrc.d is comming

iry · February 7, 2018, 10:23pm

Here is what I did:

sudo /usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --allow-missing-torrc --ignore-missing-torrc -f /etc/tor/torrc -f /usr/local/etc/torrc.d/40_anon_connection_wizard.conf -f /usr/local/etc/torrc.d/50_user.conf --RunAsDaemon 0

The configurations in /usr/local/etc/torrc.d/40_anon_connection_wizard.conf does not take effect. I guess this approach does not work?

iry · February 7, 2018, 10:25pm

That is really nice to have. But I am afraid these two commands are only available as commandline parameters.

iry · February 7, 2018, 10:28pm

All we want to do is to make sure /usr/local/etc/torrc.d/40_anon_connection_wizard.conf and /usr/local/etc/torrc.d/50_user.conf exists.

So shall we do:

ExecStartPre=/usr/bin/touch /usr/local/etc/torrc.d/40_anon_connection_wizard.conf

Patrick · February 8, 2018, 12:23am

iry:

Patrick:

First line we keep unmodified. Second and third line we extend with

-f /usr/local/etc/torrc.d/40_anon_connection_wizard.conf
-f /usr/local/etc/torrc.d/50_user.conf

iry:

Writing all the files in the command line may not work neither because Tor only accepts one torrc behind -f parameter

Here is what I did:

sudo /usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --allow-missing-torrc --ignore-missing-torrc -f /etc/tor/torrc -f /usr/local/etc/torrc.d/40_anon_connection_wizard.conf -f /usr/local/etc/torrc.d/50_user.conf --RunAsDaemon 0

The configurations in /usr/local/etc/torrc.d/40_anon_connection_wizard.conf does not take effect. I guess this approach does not work?

I speculate -f can only be used once.

Patrick · February 8, 2018, 12:27am

iry:

All we want to do is to make sure
/usr/local/etc/torrc.d/40_anon_connection_wizard.conf and
/usr/local/etc/torrc.d/50_user.conf exists.

So shall we do:

ExecStartPre=/usr/bin/touch /usr/local/etc/torrc.d/40_anon_connection_wizard.conf

Idea: Great!

iry:

All we want to do is to make sure /usr/local/etc/torrc.d/40_anon_connection_wizard.conf and /usr/local/etc/torrc.d/50_user.conf exists.

So shall we do:

ExecStartPre=/usr/bin/touch /usr/local/etc/torrc.d/40_anon_connection_wizard.conf
Implementation:
Wouldn’t work. Also mkdir -p beforehand would be required. Anyhow.

systemd tmpfiles.d is the proper mechanism for that. To find some examples:

find . -type f -not -iwholename '*.git*'

(Got a shortcut for that.)


./packages/sdwdate/usr/lib/tmpfiles.d/sdwdate.conf


./packages/bootclockrandomization/usr/lib/tmpfiles.d/bootclockrandomization.conf


./packages/anon-ws-disable-stacked-tor/usr/lib/tmpfiles.d/anon-ws-disable-stacked-tor.conf

./packages/anon-shared-helper-scripts/usr/lib/tmpfiles.d/anon-shared-helper-scripts.conf

./packages/rads/usr/lib/tmpfiles.d/rads.conf


./packages/usability-misc/usr/lib/tmpfiles.d/50_openvpn_unpriv.conf


./packages/whonixcheck/usr/lib/tmpfiles.d/whonixcheck.conf

When copying and pasting a solution like this, it’s also useful to grep
a few filenames to see other places where they may have to be
referenced. See sdwdate.conf etc.

mygrep -r /usr/lib/tmpfiles.d/bootclockrandomization.conf
+ exec grep --exclude=README.md --exclude=GPLv2 --exclude=GPLv3
--exclude=COPYING --exclude=changelog.upstream-old1 --exclude-dir=mnt
--exclude-dir=qubes-src/linux-template-builder/mnt
--exclude=changelog.upstream --exclude-dir=.git
--exclude-dir=chroot-debian --exclude-dir=chroot-jessie -r
/usr/lib/tmpfiles.d/bootclockrandomization.conf
packages/bootclockrandomization/debian/bootclockrandomization.postinst:
systemd-tmpfiles --create
/usr/lib/tmpfiles.d/bootclockrandomization.conf >/dev/null || true

I.e. looking into for example into
bootclockrandomization/debian/bootclockrandomization.postinst would
reveal the required postinst snippet.

Patrick · February 8, 2018, 12:33am

There is one disadvantage of the systemd tmpfiles mechanism.

These files would be created inside a Qubes-Whonix-Gateway TemplateVM as
well as in a Qubes TemplateBased Whonix-Gateay ProxyVM. Not the cleanest
solution. But I guess acceptable.

On a second thought, it has to be seen if systemd tmpfiles works in
/usr/local. I doubt it has been used there before. But we’ll find out.
Otherwise we think more about ExecStartPre. There is a distinction for
systemd unit drop-ins between override and extend, if I remember
correctly? We wouldn’t want to override for simplicity (in case the
upstream ExecStartPre changes and we don’t notice, so don’t catch up,
therefore introduce unwanted/unaware differences). We’d want to extend
so we don’t interfere.

iry · February 19, 2018, 6:22am

Hi Patrick!

I am sorry for the delay of my response. And thank you so much for teaching me on the working flow. I do find it really helpful.

I tested with the following configurations (Let’s call it torrcd.conf for now):

d /usr/local/etc/torrc.d/ 0755 root staff
f /usr/local/etc/torrc.d/40_anon_connection_wizard.conf 0755 root staff
f /usr/local/etc/torrc.d/50_user.conf 0755 root staff

user@host:~$ cat /etc/torrc.d/95_whonix.conf 
## Do not edit this file!
## Please create and add modifications to the following file instead:
## /usr/local/etc/torrc.d/50_user.conf
%include /usr/local/etc/torrc.d/40_anon_connection_wizard.conf
%include /usr/local/etc/torrc.d/50_user.conf

It partly works as expected but here are some problems:

The systemd-tmpfile will only auto execute once at boot time, which means when /usr/local/etc/torrc.d/50_user.conf is deleted somehow after the boot up, and then when user would like to restart Tor again, Tor will fail to start, unless they reboot or manually execute systemd-tmpfile. Therefore, it would be helpful if we could find a way to make sure those directory and files exist by using ExecStartPre in /lib/systemd/system/tor@default.service.d/40_torrcd-workaround.conf.
If we decide to go for tmpfiles rather than ExecStartPre, we should put the torrcd.conf in a Whonix-Gateway-Only package instead of anon-connection-wizard package. Because even if it is related to the anon-connection-wizard, we should assume Whonix without anon-connection-wizard installed should also work.

It seems postinst snippets are all containing other components for example:

source /usr/lib/anon-shared-helper-scripts/torsocks-remove-ld-preload

case "$1" in
   configure)
      adduser --home /nonexistent --quiet --system --group whonixcheck || true

      ## Compatibility with anon-ws-disable-stacked-tor.
      addgroup debian-tor 2>/dev/null || true

      ## Add whonixcheck to group debian-tor so it can read
      ## /var/run/tor/control.authcookie which is required to check for Tor
      ## bootstrap test.
      addgroup whonixcheck debian-tor

Do we also need somethign similar to this part? Or do we just need:

## workaround for 'dh_installinit should run systemd-tmpfiles if a
## /usr/lib/tmpfiles.d/ snippet gets shipped for systemd-only packages
## also' - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795519
# In case this system is running systemd, we need to ensure that all
# necessary tmpfiles (if any) are created before starting.
if [ -d /run/systemd/system ] ; then
        systemd-tmpfiles --create /usr/lib/tmpfiles.d/torrcd.conf >/dev/null || true
fi

true "INFO: debhelper beginning here."

iry · February 19, 2018, 6:27am

I agree that we should extend instead of overriding, for ExecStartPre, putting setting in service.d should be extending, not orveriding I guess?

Yes, this will not work as epected. Is there another way to make sure certain files and directory exist? I searched online, but have not found a solution.

Thank you very much!

Patrick · February 19, 2018, 4:36pm

iry:

Patrick:

Otherwise we think more about ExecStartPre. There is a distinction for systemd unit drop-ins between override and extend, if I remember correctly? We wouldn’t want to override for simplicity (in case the upstream ExecStartPre changes and we don’t notice, so don’t catch up, therefore introduce unwanted/unaware differences). We’d want to extend so we don’t interfere.

I agree that we should extend instead of overriding, for ExecStartPre, putting setting in service.d should be extending, not orveriding I guess?

Yes.

ExecStartPre= is needed to overwrite all previous ones.

ExecStartPre=something extends.

iry:

So shall we do:

ExecStartPre=

iry:

All we want to do is to make sure /usr/local/etc/torrc.d/40_anon_connection_wizard.conf and /usr/local/etc/torrc.d/50_user.conf exists.

So shall we do:

ExecStartPre=/usr/bin/touch /usr/local/etc/torrc.d/40_anon_connection_wizard.conf

Yes, this will not work as epected. Is there another way to make sure certain files and directory exist? I searched online, but have not found a solution.

Sounds good!

Patrick · February 19, 2018, 4:41pm

Do we also need somethign similar to this part? Or do we just need:

## workaround for 'dh_installinit should run systemd-tmpfiles if a
## /usr/lib/tmpfiles.d/ snippet gets shipped for systemd-only packages
## also' - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795519
# In case this system is running systemd, we need to ensure that all
# necessary tmpfiles (if any) are created before starting.
if [ -d /run/systemd/system ] ; then
        systemd-tmpfiles --create /usr/lib/tmpfiles.d/torrcd.conf
/dev/null || true
fi

true "INFO: debhelper beginning here."

Only that is required.

iry:

The systemd-tmpfile will only auto execute once at boot time, which means when /usr/local/etc/torrc.d/50_user.conf is deleted somehow after the boot up, and then when user would like to restart Tor again, Tor will fail to start, unless they reboot or manually execute systemd-tmpfile. Therefore, it would be helpful if we could find a way to make sure those directory and files exist by using ExecStartPre in /lib/systemd/system/tor@default.service.d/40_torrcd-workaround.conf.

Makes sense.

If we decide to go for tmpfiles rather than ExecStartPre, we should put the torrcd.conf in a Whonix-Gateway-Only package instead of anon-connection-wizard package. Because even if it is related to the anon-connection-wizard, we should assume Whonix without anon-connection-wizard installed should also work.

Theoretical: Yes, into anon-gw-anonymizer-config package.

Practical: ExecStartPre touch method is better.

Actually: we need probably both… tmpfiles and ExecStartPre.
ExecStartPre for reasons you explained.

tmpfiles to create the folder. Because without the folder existing,
ExecStartPre touch would fail.

Patrick · February 19, 2018, 4:47pm

While we are at it…

What about ExecStartPre running a script that checks if
/usr/local/etc/torrc.d/50_user.conf is non-existing and if so,
populates it with some content from a template file from
/usr/share/anon-gw-anonymizer-config/user_torrc_template.conf (or so)?

I have a report from a user why user torrc is empty and that creating
confusion.

What about /usr/local/etc/torrc.d/50_user.conf by default looking like
this?

## Tor user specific configuration file

iry · February 19, 2018, 4:58pm

Sounds great to me. I will start working on the script right now.

Patrick · February 19, 2018, 6:11pm

(Somehow my reply by e-mail was truncated.)

## Tor user specific configuration file
##
## Add user modifications below this line:
##########

iry · February 19, 2018, 7:13pm

I tried the approach as follows, but it does not work as expected:

There are two problems:

The script needs root, but systemd does not run it with root privilege
The extended ExecStartPre in /lib/systemd/system/tor@default.service.d/40_torrcd-workaround.conf will only be executed after all the ExecStartPre in /lib/systemd/system/tor@default.service have been done. However, in /lib/systemd/system/tor@default.service, there is a line to check if Tor will work or not: ExecStartPre=/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0 --verify-config, and it will be executed (and then fail) before the script is executed.

Patrick · February 19, 2018, 9:39pm

iry:

Patrick:

What about ExecStartPre running a script that checks if /usr/local/etc/torrc.d/50_user.conf is non-existing and if so, populates it with some content from a template file from /usr/share/anon-gw-anonymizer-configuser_torrc_template.conf (or so)?

I tried the approach as follows, but it does not work as expected:
Attempt to make sure all torrc files exist · irykoon/anon-gw-anonymizer-config@ae80ee0 · GitHub

There are two problems:

The script needs root, but systemd does not run it with root privilege

This might be fixable using User= or so, but we shouldn’t do it. Too
intrusive since not our service. Could lead to bugs.

The extended ExecStartPre in /lib/systemd/system/tor@default.service.d/40_torrcd-workaround.conf will only be executed after all the ExecStartPre in /lib/systemd/system/tor@default.service have been done. However, in /lib/systemd/system/tor@default.service, there is a line to check if Tor will work or not: ExecStartPre=/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0 --verify-config, and it will be executed (and then fail) before the script is executed.

I see. I doubt that can be solved directly.

So we need our own systemd unit file that uses systemd directive
Before=tor.service that runs make-sure-torrc-exist.

/lib/systemd/system/anon-gw-anonymizer-config.service or so.

(Or even Before=tor@default.service - only if that is required.)

Could you please move /usr/bin/make-sure-torrc-exist to
/usr/lib/anon-gw-anonymizer-config/make-sure-torrc-exist?

Otherwise we’d have to create and ship a man page (otherwise lintian
warning). And users wouldn’t run this command so be safe all this effort
by moving it to a more appropriate directory.

Could implement that please?

Hints for new systemd unit file packaging implementation:

In debian/control add to Build-Depends: dh-systemd.
In debian/rules, change dh $@ to dh $@ --with systemd.
There are quite a few systemd unit files in Whonix source code as
examples.

myfind . | grep systemd | grep \.service | grep --invert-match \\.d | grep --invert-match \\.in

iry · February 19, 2018, 11:44pm

Patrick Schleizer:

Could implement that please?

Yes, definitely! I consider it as a great learning opportunity. Thank
you so much for your instructions, Patrick!

Patrick · February 20, 2018, 3:20pm

Qubes-Whonix
Now, that we no longer have to modify file /etc/tor/torrc, folder /etc/tor can and should be removed from bind-dirs?

github.com

Whonix/qubes-whonix/blob/dcda4a24d2dba03a3677726db5dcbb7e7a443cc5/usr/lib/qubes-bind-dirs.d/40_qubes-whonix.conf#L12


      
          ## Copyright (C) 2012 - 2018 ENCRYPTED SUPPORT LP <adrelanos@riseup.net>
          ## See the file COPYING for copying conditions.
          
          
binds+=( '/var/lib/tor' )
          binds+=( '/var/lib/whonix' )
          binds+=( '/var/lib/whonixcheck' )
          binds+=( '/var/cache/whonix-setup-wizard' )
          binds+=( '/var/cache/anon-base-files' )
          binds+=( '/var/cache/qubes-whonix' )
          binds+=( '/var/cache/tb-updater' )
          binds+=( '/etc/tor' )
          
          
mkdir -p /usr/local/etc/torrc.d

bind-dirs reference:

Patrick · February 20, 2018, 4:16pm

Patrick · February 20, 2018, 4:17pm

iry · February 21, 2018, 10:44pm

Again, thank your very much for your instructions, Patrick!

Done:

https://github.com/Whonix/anon-gw-anonymizer-config/pull/9

I set User=root in /lib/systemd/system/anon-gw-anonymizer-config.service. Otherwise, I do not know how to write to /use/local.