Control Port Filter Proxy Python (cpfpy) / anon-ws-disable-stacked-tor

Development
21

[quote=“troubadour, post:19, topic:533”]- create a configuration file.

  • create a hash file (sha1) of the configuration file (installed along the configuration file).
  • when CPFP is started, it runs the hash function of the configuration file and compares it to the hash file content.[/quote]
    Why do you need a hash of the config anyway? I don’t know the rationale behind that yet. Why not just parse it every time cpfpy starts?
I overlooked that (a daemon should not have root privileges).
Probably easiest to run the daemon as the same user as Tor, as user debian-tor. Because that user has access to Tor's files (control cookie) because it's member of the debian-tor group. (But that user does not have a /home folder by default.)
I do agree on the usability of the .d folders. Yes, the problem is parsing.
Please find out. Never figured that out myself. Otherwise maybe I would have written more scripts in python.
What you are doing in bash is very neat (although I do not fully understand the magic).
[code] [...] ## Check if folder /etc/controlportfilt.d exists. if [ -d /etc/controlportfilt.d ]; then ## Use bash (or also sh's? nevermind) globbing feature. It matches all files in that folder. (Except for i in /etc/controlportfilt.d/*; do if [ -f "$i" ]; then ## [...] ## Check for syntax errors. (Would exit 0 if no syntax errors, otherwise 1.) ## (And non-zero exit codes would be caught by the error handler and the whole script would exit.) bash -n "$i" ## "source" (same as ".") the folder. Imagine you'd take the content of the whole file "$i" and include it at this point in the script. source "$i" fi [...] [/code]
It would be a whole different game to parse the controlportfilt.d directory in Python and make out what the real configuration is.
Looks like there is no python library for doing this?
The configuration file I am working with: [code] [CONFIGURATION] CONTROL_PORT_FILTER_LIMIT_GETINFO_NET_LISTENERS_SOCKS = True CONTROL_PORT_FILTER_LIMIT_STRING_LENGTH = True CONTROL_PORT_FILTER_EXCESSIVE_STRING_LENGTH = 128

[WHITE_LIST]
1 = GETINFO net/listeners/socks
2 = SIGNAL NEWNYM
3 = GETINFO status/bootstrap-phase
4 = GETINFO status/circuit-established
[/code]

Looks like a long way to go to make it overrulable? :expressionless:

22

May I suggest you use SHA256 as a minimum or better yet SHA512? SHA1 is pretty much toast at this point for people with serious computing power, even Google is moving away from SHA1 for certs in Chrome. I mean there isn’t a whole lot of data we are hashing anyway, slow downs will hardly be noticeable or matter to a serious degree.

It would be sweet if we can use Keccak (the new SHA-3 finalist) but I don’t know if its available in the library you are using.

23

The hash for detecting “has local config changed” on the local disk is irrelevant for security. Even md5 would do. Because someone in position to modify that file, doesn’t need to break hashes anymore. But I haven’t yet understood the need for hashing the config file anyway. If hashes were used for something else like certs of fingerprints, we really should avoid sha1.

24
The hash for detecting "has local config changed" on the local disk is irrelevant for security.

I’m not quite sure how the hashing is being used in this design but if we are comparing the hash of the trusted whitelist config file with that of the untrusted one, then the algorithm we use is important because a skilled attacker can craft a file with a different/malicious input that could give the same hash as the trusted one.

25

That’s for sure, but that’s not what this hash is about.

26

Yes, it is. I was not clear in my first description, too new to the field.

For a better understanding, the would be CPFP package file list (not for .d configuration environment, obviously):

   /etc/init.d/control-port-filter
   /usr/bin/controlportfilt
   /usr/lib/cpfp.py
   /etc/cpfp.conf
   /somewhere-root/cpfpconf.hash
   /usr/lib/cpfp_configuration_editor

Would be design:

When the daemon is [re]started, “cpfp.py” computes the hash (sha512, it’s available) of “/etc/cpfp.conf”, compares the result (untrusted) to the one stored in “cpfpconf.hash” (trusted). If the strings do not match, CPFP issues a warning and stops, or runs with a default hardcoded configuration.

“cpfp_configuration_editor” is a GUI plain text editor that opens “/etc/cpfp.conf”. In the context, it is the mandatory mean for updating the configuration.
It’s opened as root (“kdesudo /etc/usr/lib/cpfp_configuration_editor”). On closing, after the changes from the user, it saves the file, computes the new hash, updates “cpfpconf.hash” (root) and possibly, issues SIGHUP to the daemon to restart it. We’ve come full circle.

27
If the strings do not match, CPFP issues a warning and stops, or runs with a default hardcoded configuration.
Why that? Users should be able to change the config.
"cpfp_configuration_editor" is a GUI plain text editor that opens "/etc/cpfp.conf". In the context, it is the mandatory mean for updating the configuration.
Why invent the first *nix daemon, that requires a GUI to make config changes? Also what about CLI users? Seems like a unnecessary limitation to me.

[hr]

I must say, I don’t like the idea of not having a .d config folder at all. Also the mandatory GUI editor. But nevermind…

Fortunately, I had the idea to search for “python source bash”. What I wanted to know from search engines was “how to use something like bash’s source in python”. And voila, it looks like there is a quite simple solution. :slight_smile:

Method 1:

  • cpfp.py could at the very beginning run a bash helper script, that does sourcing of the .d config folder. Then python could read the variables set by th bash helper script. Sounds difficult? I don’t think it is. See: Emulating Bash 'source' in Python - Stack Overflow

Method 2:

  • There are other ways to implement this when searching along such search terms. Also one could ask in python specific help channels on how to parse a .d folder. But I don’t think that will be necessary. The above method looks okay to me.
28
I must say, I don't like the idea of not having a .d config folder at all. Also the mandatory GUI editor. But nevermind...

Let’s forget it. That was the only way I could see to check the input from Workstation (in a GUI environment, I forgot the cli users in Gateway). Good exercise, though.

Fortunately, I had the idea to search for "python source bash". What I wanted to know from search engines was "how to use something like bash's `source` in python". And voila, it looks like there is a quite simple solution. :)

There is always a solution :), but I could not get Method 1 working (did not try method 2). As far as I know, the script in Emulating Bash 'source' in Python - Stack Overflow can be written that way:

#! /usr/bin/env python

#import os
import pprint
#import subprocess

#command = ['bash', '-c', 'source init_env && env']

#proc = subprocess.Popen(command, stdout = subprocess.PIPE)

#for line in proc.stdout:
#  (key, _, value) = line.partition("=")
#  os.environ[key] = value

#proc.communicate()

pprint.pprint(dict(os.environ))

# for a variable
print os.environ("PATH")

Whatever you try to source (script uncommented) gives the same “pprint” output. The last line explains why I was asking for CPFP configuration in the environment. The variables are easy to import in Python if they are in the environment.

Nevermind, “/etc/controlportfilter.d” is properly parsed (I believe) in GitHub - troubadoour/control-port-filter-python. There are three commits:

  • new format in “/etc/controlportfilter.d/30_controlportfilt_default”
  • commented the lines sourcing “/etc/controlportfilter.d/” in “/usr/bin/controlportfilt”
  • added the parsing code in “/usr/lib/cpfp.py”

I was able to substitute variables values by adding files in the directory, for example modifying the list of commands in “CONTROL_PORT_FILTER_WHITELIST”. Please check.

Note. At some stage, I got mixed up with some git commands (not unusual), so I deleted my repository in github and restarted fresh, cloning yours.

29

Looking at the diff:

(Very small issues for now since it’s still developing. Just mentioning.)

  • Removed the copyright by mistake?
  • Please set your editor to remove leading spaces when saving. Or make them visible and remove them before saving. And please use diff before commit, so we don’t add leading spaces.

Will the CONTROL_PORT_FILTER_WHITELIST variable be extensible or just overrule? I guess it is the latter?

The ability to extend variables is one of the core features of .d config folders. Sorry for picking on this one, but it will be useful in future when additional usability packages may be added such as “whonix-gw-hidden-webserver” (a package that sets up all settings on Whonix-Gateway to set a hidden web server). In that case it would be much easier if that package could just drop a config snippet.

For example…

/etc/controlportfilt.d/30_controlportfilt_default sets the default:

CONTROL_PORT_FILTER_WHITELIST=SIGNAL NEWNYM,GETINFO net/listeners/socks,GETINFO status/bootstrap-phase,GETINFO status/circuit-established

/etc/controlportfilt.d/40_hidden_web_server without knowing what any before it did adds “GETINFO something” so it looks like:

CONTROL_PORT_FILTER_WHITELIST=SIGNAL NEWNYM,GETINFO net/listeners/socks,GETINFO status/bootstrap-phase,GETINFO status/circuit-established, GETINFO something

And finally /etc/controlportfilt.d/40_onionshare without knowing what any before it did adds “GETINFO whatever” so it looks like:

CONTROL_PORT_FILTER_WHITELIST=SIGNAL NEWNYM,GETINFO net/listeners/socks,GETINFO status/bootstrap-phase,GETINFO status/circuit-established, GETINFO something,GETINFO whatever

If we must change the /etc/controlportfilt.d/30_controlportfilt_default format, we should probably use another folder name. I didn’t like /etc/controlportfilt.d anymore. Maybe we find something better? /etc/cpfpy.d? (So users who in past used cpfp-bash won’t end up with a broken config folder. Also both packages can coexist for a while when there are no file name conflicts (not using same file names two packages at once).

30
- Removed the copyright by mistake?
Yes
- Please set your editor to remove leading spaces when saving. Or make them visible and remove them before saving. And please use diff before commit, so we don't add leading spaces.
I run "dIff --cached" after each addition anf "diff origin master" after commit. This one was pushed in a kind of a hurry, and Kate was not set properly.

Both fixed.

Will the CONTROL_PORT_FILTER_WHITELIST variable be extensible or just overrule? I guess it is the latter?

The ability to extend variables is one of the core features of .d config folders. Sorry for picking on this one, but it will be useful in future when additional usability packages may be added such as “whonix-gw-hidden-webserver” (a package that sets up all settings on Whonix-Gateway to set a hidden web server). In that case it would be much easier if that package could just drop a config snippet.

From your examples with hidden services and onionshare, CONTROL_PORT_FILTER_WHITELIST is extensible. You can try adding “40_controlportfilt_address” with

CONTROL_PORT_FILTER_WHITELIST=SIGNAL NEWNYM,GETINFO net/listeners/socks,GETINFO status/bootstrap-phase,GETINFO status/circuit-established,GETINFO address

The daemon still works normally. Now

$ telnet 10.152.152.10 9052
Trying 10.152.152.10...
Connected to 10.152.152.10.
Escape character is '^]'.
GETINFO address
250-address=176.xxx.xxx.xxx

(yes, it’s my host’s address, scary)

The other variables values can also be changed that way.

Because I had some problems when giving arbitrary names to the files in .d (even while respecting the numbering convention), they are now sorted. sort files in /etc/controlportfilt.d · troubadoour/control-port-filter-python@20e904e · GitHub

If we must change the /etc/controlportfilt.d/30_controlportfilt_default format, we should probably use another folder name. I didn't like /etc/controlportfilt.d anymore. Maybe we find something better? /etc/cpfpy.d? (So users who in past used cpfp-bash won't end up with a broken config folder. Also both packages can coexist for a while when there are no file name conflicts (not using same file names two packages at once).

“/etc/cpfpy.d” sounds good, and yes, it does not break the original package, in case… If you install the Python version for testing, the only change to revert to the bash version would be in /usr/bin/controlportfilt, running tcpserver instead of cpfp.py.

31

Pushed the change (/etc/cpfpy.d).

32

That’s not what I mean by extensible. I mean extending a variable without knowing what predecessors already have added and without caring what successors will add (or even remove).

Example.

## 30_default
x="first, second"

## 40_controlportfilt_address
x="$x, third, fourth"

## result
echo "$x"
first, second, third, fourth

Otherwise if 40_controlportfilt_address needs to know, that 30_default already added “SIGNAL NEWNYM” and needs to copy that, then that defeats the purpose of .d config files.

33

In many situations, using popen is not good idea because it triggers parts of the shell that are directly affected by shellshock:

to my understanding this means that calls like:

os.popenx([‘executable’, ‘some’, ‘arguments’])

are safe because no shell is invoked, while commands of the form:

os.popenx(‘executable some argument’)

are vulnerable because a shell will be used to interpret the string as a command line.

Also note that all popen functions are deprecated since python2.6 and should have been avoided since then. The subprocess module provides a much better interface to launching subprocesses which is shellshock safe by default. If you don’t pass the shell=True argument your program wont launch subshells and hence wont be affected by shellshock.

34

Doesn’t apply for what he had in mind here - opening a file on local hdd that contains the config, that needs to be non-malicious so or so.

35

There have been some warnings on popen before shellshock (I do not remember exactly what about). Trying to avoid it.

36

[quote=“Patrick, post:32, topic:533”]That’s not what I mean by extensible. I mean extending a variable without knowing what predecessors already have added and without caring what successors will add (or even remove).

Example.

## 30_default
x="first, second"

## 40_controlportfilt_address
x="$x, third, fourth"

## result
echo "$x"
first, second, third, fourth

Otherwise if 40_controlportfilt_address needs to know, that 30_default already added “SIGNAL NEWNYM” and needs to copy that, then that defeats the purpose of .d config files.[/quote]

I misread/misunderstood your post Whonix Forum.

It should be better in CONTROL_PORT_FILTER_WHITELIST extensible · troubadoour/control-port-filter-python@210a6f5 · GitHub.

With the way it is parsed, a boolean or any single value variable has to be changed in the default file (the values are added to a list, changing ‘true’ to ‘false’ would result in ‘true, false’.)

37

Doesn’t run if there is no config folder.

Traceback (most recent call last): File "usr/lib/cpfp.py", line 60, in <module> if LIMIT_STRING_LENGTH: NameError: name 'LIMIT_STRING_LENGTH' is not defined

(cpfp-bash can’t do that either, but this is a bug as per Debian policy. So it would be great if it would just work with the built in defaults then.)

38

Yes, now I have to expand the error handling for all situations (if the design of cpfp.py looks sound).

39

Yeah. I am currently testing config handling.

Can you make it support parallel connections?

40
Can you make it support parallel connections?

It can be done, surely. Can I make it? It might take some time, as it involves threading, most likely. By the way, in which situations would that be necessary?