control-port-filter-python.service exits pre-maturely

[phabricator-migrator]

Information

ID: 273
PHID: PHID-TASK-5mimphcah6bqytwj725s
Author: nrgaway
Status at Migration Time: resolved
Priority at Migration Time: Normal

Description

It seems control-port-filter-python exits pre-maturely. I am unaware of its expected behaviour since this seems to be a new modules for Whonix 10.

If I manually restart the control-port-filter-python after booting the gateway, it seems to run fine, otherwise it exits with no error and therefore Whonix Workstation can not connect.

Status after booting
====================
root@host:/etc/init.d# systemctl status control-port-filter-python 
control-port-filter-python.service - LSB: Control Port Filter Proxy
   Loaded: loaded (/etc/init.d/control-port-filter-python)
   Active: active (exited) since Thu 2015-04-23 12:42:08 UTC; 6min ago
  Process: 1250 ExecStart=/etc/init.d/control-port-filter-python start (code=exited, status=0/SUCCESS)

Apr 23 12:42:08 host systemd[1]: Starting LSB: Control Port Filter Proxy...
Apr 23 12:42:08 host systemd[1]: Started LSB: Control Port Filter Proxy.

Restart service
===============
root@host:/etc/init.d# systemctl restart control-port-filter-python 

Status after restarting service
===============================
root@host:/etc/init.d# systemctl status control-port-filter-python 
control-port-filter-python.service - LSB: Control Port Filter Proxy
   Loaded: loaded (/etc/init.d/control-port-filter-python)
   Active: active (running) since Thu 2015-04-23 12:49:37 UTC; 46s ago
  Process: 11024 ExecStop=/etc/init.d/control-port-filter-python stop (code=exited, status=0/SUCCESS)
  Process: 11037 ExecStart=/etc/init.d/control-port-filter-python start (code=exited, status=0/SUCCESS)
   CGroup: name=systemd:/system/control-port-filter-python.service
           └─11054 /usr/bin/python /usr/lib/control-port-filter-python/cpfp.py

Apr 23 12:49:37 host systemd[1]: Starting LSB: Control Port Filter Proxy...
Apr 23 12:49:37 host systemd[1]: Started LSB: Control Port Filter Proxy.

Comments

---

Patrick

2015-04-23 20:13:46 UTC

As of Whonix 10.0.0.5.5 / control-port-filter-python (0.4-1), on wheezy, the sysvinit script and #control-port-filter-python seems to work very stable. I guess it “just” needs a proper #systemd unit file.

---

nrgaway

2015-04-23 21:23:02 UTC

I can write a systemd unit file, but that does not explain why the cpfp.py process is exiting in the first place.

This seems to be my only blocking item to being able to release Whonix 10 workstation.

Please share some additional information that will assist in the creation of a systemd files such as:

• At what point should it be started? Before or after Tor?
• Does it need to be restarted or reloaded if Tor is restarted or reloaded
• Any security considerations
• Does cpfp log its output? to syslog?
• Are there any conditions to starting the process?
• Anything else you may think I may find useful

---

Patrick

2015-04-23 21:50:27 UTC

! In T273#3833, @nrgaway wrote:
I can write a systemd unit file

Yes, please.

but that does not explain why the cpfp.py process is exiting in the first place.

Indeed.

Related, FYI:
Whonix Forum

• At what point should it be started? Before or after Tor?

Not defined at the moment. Doesn’t matter. If Tor is not yet started, it will [hopefully] behave as if Tor is not yet started. No issue there. whonixcheck waits for it up to 30 seconds, seems sufficient. Probably not [reverse]dependency in sysvinit / systemd required. sdwdate-plugin-anon-shared-con-check waits as long as needed.

• Does it need to be restarted or reloaded if Tor is restarted or reloaded

No need.

• Any security considerations

None that I can see.

• Does cpfp log its output? to syslog?

At the moment using python logging, using /var/log/control-port-filter-python.log. @troubadour might change this in future (T243).

• Are there any conditions to starting the process?

Not that I know. Just a usual daemon. Not too important. Just what the current sysvinit script does. Nothing fancy.

• Anything else you may think I may find useful

Nope.

---

nrgaway

2015-04-24 04:13:19 UTC

I have completed the systemd unit file. There are a few deficiencies we may want to consider for cpfp.py for the future:

1. cpfp.py should daemonize itself and write the PID to the proper location
2. cpfp.py should implement a watchdog (sd_notify) function

I have impleemnted a work-around in systemd unit file to use start-stop-daemon to deamonize cpfp since otherwise the PID would not be created and whonixcheck raised a concern.

The watchdog feature would be nice to have, which would restart the service if it appeared to be running, but became unresponsive.

Here is the systemd unit file:
/lib/systemd/system/control-port-filter-python.service

# control-port-filter-python.service -- Filters out dangerous control port 
# messages in Tor control port.
#
# See also related configuration file:
# /etc/tmpfiles.d/control-port-filter-python.conf

[Unit]
Description = Control Port Filter Proxy
ConditionPathExists = /usr/lib/control-port-filter-python/cpfp.py
After = remote-fs.target syslog.target network.target

[Service]
Type = forking
User = debian-tor
Group = debian-tor

## Run ExecStartPre with root-permissions
PermissionsStartOnly=true
ExecStartPre = /usr/lib/anon-shared-helper-scripts/torsocks-remove-ld-preload
ExecStartPre = /bin/touch /var/log/%p.log ; /bin/chown --recursive debian-tor:debian-tor /var/log/%p.log
PIDFile = /var/run/%p/pid

## Run ExecStart with User=debian-tor / Group=debian-tor
# ExecStart = /usr/lib/control-port-filter-python/cpfp.py
#
## TODO: cpfp.py should daemonize itself and handle pid file creation
ExecStart = /sbin/start-stop-daemon \
      --start \
      --quiet \
      --background \
      --make-pidfile \
      --pidfile /var/run/%p/pid \
      --chuid debian-tor:debian-tor \
      --exec /usr/lib/control-port-filter-python/cpfp.py

##
## General options
##

TimeoutSec = 30
Restart = always
StandardOutput = syslog

## TODO: Watchdog disabled.  cpfp.py would need to implement the sd_notify protocol
# WatchdogSec = 1m

##
## Hardening
##

PrivateTmp=yes

[Install]
WantedBy=multi-user.target

And related configuration file:
/etc/tmpfiles.d/control-port-filter-python.conf

d /var/run/control-port-filter-python 02750 debian-tor debian-tor

---

nrgaway

2015-04-24 04:18:43 UTC

! In T273#3835, @Patrick wrote:

! In T273#3833, @nrgaway wrote:
but that does not explain why the cpfp.py process is exiting in the first place.

Indeed.

The issue was related to IP addresses being of the old value 10.152.152.10 and the init file not restarting on that error. As an init file it must have been starting before the search and replace function; before network was started maybe.

Anyway, with the new systemd unit file in place it seems to be working correctly. I am currently re-building both the gateway and workstation for a final test.

---

Patrick

2015-04-24 05:03:13 UTC

Nice!

Can you add this to git please?

• copyright?
  Fork https://github.com/Whonix/control-port-filter-python?

Other TODO:

• add to debian/control
• add to debian/rules
• remove syvinit script

I don’t mind who is doing this.

! In T273#3852, @nrgaway wrote:
There are a few deficiencies we may want to consider for cpfp.py for the future:

1. cpfp.py should daemonize itself and write the PID to the proper location
2. cpfp.py should implement a watchdog (sd_notify) function

Created T274 for it.

---

Patrick

2015-04-26 22:01:17 UTC

@nrgaway?

---

nrgaway

2015-04-26 23:46:48 UTC

! In T273#3929, @Patrick wrote:
@nrgaway?

You rang? Are you asking me to commit it to repo?

---

Patrick

2015-04-27 00:02:38 UTC

! In T273#3939, @nrgaway wrote:

! In T273#3929, @Patrick wrote:
@nrgaway?

You rang?

Yeah!

Are you asking me to commit it to repo?

Not necessarily. I am not employing you, so I cannot ask you for anything.

I was hoping you address https://phabricator.whonix.org/T273#3867. To re-state…

Can you fork and commit the systemd unit file to control-port-filter-python? You plan on doing it, prefer doing it yourself or prefer me doing it? I actually prefer if you do it, because then you get the git commit and credit in git history and licensing will be sorted out. Since this is a volunteer based thing, it’s also fine for me to take your systemd unit file as is if you wish and do the commit myself.

And what about the other TODO mentioned in my previous comment? Same here. If you wish to do it, that’s fine. If you prefer me doing it, that’s also fine.

Was just ringing to find out if you plan on doing it or if I should do it.

---

nrgaway

2015-04-27 00:07:38 UTC

! In T273#3941, @Patrick wrote:

! In T273#3939, @nrgaway wrote:

! In T273#3929, @Patrick wrote:
@nrgaway?

You rang?

Yeah!

Are you asking me to commit it to repo?

Not necessarily. I am not employing you, so I cannot ask you for anything.

I was hoping you address https://phabricator.whonix.org/T273#3867. To re-state…

Can you fork and commit the systemd unit file to control-port-filter-python? You plan on doing it, prefer doing it yourself or prefer me doing it? I actually prefer if you do it, because then you get the git commit and credit in git history and licensing will be sorted out. Since this is a volunteer based thing, it’s also fine for me to take your systemd unit file as is if you wish and do the commit myself.

And what about the other TODO mentioned in my previous comment? Same here. If you wish to do it, that’s fine. If you prefer me doing it, that’s also fine.

Was just ringing to find out if you plan on doing it or if I should do it.

Yes I can do it. I would prefer to get Whonix11 build going so I can test it within that environment. It should just work, but I do prefer to test on the targeted platform before I commit something.

Hopefully I can figure out why I am getting package problems building Whonix11 tonight

---

nrgaway

2015-04-27 09:35:20 UTC

Do you want me to keep both systemd and sysv init files in the package. Both of them could go in the /usr/share/control-port-filter-python directory and be available to anyone that may want to manually install the sysv init scripts. At the same time a copy can be made of the systemd unit file to be placed in the /lib/systemd/system directory (since linking may not work)

Then at some point you could even consider automatically detecting the init manager to install the appropriate configuration files.

I figure once this is package, I may as well use it in Whonix 10, then there will not be any possibility of conflict down the road. Same for whonixcheck; maybe tor as well

---

Patrick

2015-04-27 10:18:36 UTC

I would like to do it the Debian way. Like how other Debian packages handle this. Using the standard way. I want to figure that out today. And only if the standard way of doing this seems inappropriate I would cook up a custom solution. Maybe you get to find this out before me? debian-mentors IRC?

---

Patrick

2015-04-27 13:58:38 UTC

I’ve got an unrelated lintian error for an existing sysvinit script.

Since the effort to keep sysvinit still somewhat supported is too high for too little gain, let’s deprecate all sysvinit scripts for Whonix 11. Let’s move those into the https://github.com/Whonix/deprecated-code repository and then say “patches welcome” if someone wants to reintroduce/contribute sysvinit support.

As for proper systemd unit files, for packages developed under the Whonix umbrella (where we are upstream), let’s move them into the standard location /lib/systemd/system?

Does that sound alright? @nrgaway?

---

debian/rules

	dh $@

→

	dh $@ --with systemd

• remove dh_installinit stuff.

---

And related configuration file:
/etc/tmpfiles.d/control-port-filter-python.conf

Wrong location as per lintian.

E: control-port-filter-python: systemd-tmpfiles.d-outside-usr-lib etc/tmpfiles.d/control-port-filter-python.conf
N: 
N:    The package ships a systemd tmpfiles.d(5) conf file outside
N:    /usr/lib/tmpfiles.d/
N:    
N:    Severity: serious, Certainty: certain
N:    
N:    Check: systemd, Type: binary
N: 

---

Please use lintian to check for other systemd related issues. They look easy to fix.

---

nrgaway

2015-04-27 22:26:19 UTC

! In T273#4007, @Patrick wrote:
I’ve got an unrelated lintian error for an existing sysvinit script.

Since the effort to keep sysvinit still somewhat supported is too high for too little gain, let’s deprecate all sysvinit scripts for Whonix 11. Let’s move those into the https://github.com/Whonix/deprecated-code repository and then say “patches welcome” if someone wants to reintroduce/contribute sysvinit support.

As for proper systemd unit files, for packages developed under the Whonix umbrella (where we are upstream), let’s move them into the standard location /lib/systemd/system?

Does that sound alright? @nrgaway?

Yes

---

debian/rules

	dh $@

→

	dh $@ --with systemd

• remove dh_installinit stuff.

---

And related configuration file:
/etc/tmpfiles.d/control-port-filter-python.conf

Wrong location as per lintian.

E: control-port-filter-python: systemd-tmpfiles.d-outside-usr-lib etc/tmpfiles.d/control-port-filter-python.conf
N: 
N:    The package ships a systemd tmpfiles.d(5) conf file outside
N:    /usr/lib/tmpfiles.d/
N:    
N:    Severity: serious, Certainty: certain
N:    
N:    Check: systemd, Type: binary
N: 

---

Please use lintian to check for other systemd related issues. They look easy to fix.

Ok.

How do I run your makefile scripts manually to build package and produce these warnings? Currently I do not use your Makefiles, so am un-aware of the process.

---

Patrick

2015-04-27 22:37:11 UTC

You don’t necessarily need Whonix’s makefiles. Although I hope they make things real comfortable and fast to test/develop.

Just cd into any package. Then run.

make deb-pkg

It builds the package and automatically runs lintian in the process.

There is also a separate make lintian.

Or you could just build the package your way, and then manually run lintian.

lintian --pedantic --info --display-info --fail-on-warnings

Or just.

lintian --pedantic

To cleanup (delete temporary files and produces artifacts in the parent directory), you can comfortably use make deb-cleanup.

There is a lot more. See make help.

---

nrgaway

2015-04-27 22:44:15 UTC

Thanks, I need to bookmark those instructions now

Is there already a forum topic on this?

---

Patrick

2015-04-27 22:49:11 UTC

Perhaps just remember make help. I guess you get used to the others quite soon. Forum… This one…
make / make help / makefile / package build process changes:
Whonix Forum
It’s an older thread. I don’t mind if we continue to use that one or create a new one.

---

nrgaway

2015-04-28 03:57:19 UTC

BTW, and as a note to self

/etc/tmpfiles.d/control-port-filter-python.conf is an acceptable location for config file if it’s not installed by the pacakge that owns it. For this package the correct location would be /usr/lib/tmpfiles.d/control-port-filter-python.conf

[[ tmpfiles.d(5) - Linux manual page | tmpfiles.d man page ]]

Files in /etc/tmpfiles.d override files with the same name in
       /usr/lib/tmpfiles.d and /run/tmpfiles.d. Files in /run/tmpfiles.d
       override files with the same name in /usr/lib/tmpfiles.d. Packages
       should install their configuration files in /usr/lib/tmpfiles.d.
       Files in /etc/tmpfiles.d are reserved for the local administrator,
       who may use this logic to override the configuration files installed
       by vendor packages. 

---

Patrick

2015-04-30 13:19:39 UTC

What’s next here?

@nrgaway Can you please post all your existing systemd units for packages developed under the Whonix umbrella somewhere? Such as the one in the comment of https://phabricator.whonix.org/T273#3852 ? + Add the license header? That would be important. It’s currently a blocker before I can continue with most Whonix 11 development tasks. I would like to integrate those into packages and make those systemd-only. I guess I am getting to that before you?

Just finished making whonix-initializer systemd-only. Once the systemd unit is ready, the required remaining changes in debian/control and debian/rules are minimal and simple.

---

Patrick

2015-04-30 15:28:20 UTC

Related, see also:
Whonix Forum

---

Patrick

2015-05-03 22:34:55 UTC

! In T273#4175, @Patrick wrote:
What’s next here?

@nrgaway Can you please post all your existing systemd units for packages developed under the Whonix umbrella somewhere? Such as the one in the comment of https://phabricator.whonix.org/T273#3852 ? + Add the license header? That would be important. It’s currently a blocker before I can continue with most Whonix 11 development tasks. I would like to integrate those into packages and make those systemd-only. I guess I am getting to that before you?

Just finished making whonix-initializer systemd-only. Once the systemd unit is ready, the required remaining changes in debian/control and debian/rules are minimal and simple.

@nrgaway

Above is important to proceed.

---

Patrick

2015-05-14 23:57:58 UTC

This has been fixed in T274. control-port-filter-python has a working systemd unit file now. Closing. Please re-open if you still see an issue here.