Control Port Filter Proxy Python (cpfpy) / anon-ws-disable-stacked-tor

Patrick · October 4, 2014, 8:32pm

When using multiple workstations and/or multiple clients that use cpfpy and/or in advanced configurations. Not that advanced though. Some custom automated controller, that does stuff + Tor Browser New Identity.

Patrick · October 4, 2014, 8:35pm

It’s also required in simpler situations. Because we already have multiple clients using it. whonixcheck uses it (runs in background, and first it checks is Tor bootstrap status). sdwdate-plugin-anon-shared-con-check uses Tor bootstrap status. tb-updater uses Tor bootstrap status. Tor Browser uses New Identity. It’s not that likely, that these happen simultaneously, but it’s also not impossible. It could result in a race condition, Heisenbug.

Patrick · October 4, 2014, 8:37pm

Config parsing looks good so far. There is a small issue. Stuff might get duplicated.

But I guess there is some easy python sort --unique function?

Patrick · October 4, 2014, 8:39pm

Pushed some minor comment fixes.

troubadour · October 4, 2014, 9:00pm

[quote=“Patrick, post:43, topic:533”]Config parsing looks good so far. There is a small issue. Stuff might get duplicated.

But I guess there is some easy python sort --unique function?[/quote]

It should not do that. Could you post your cpfpy.d content giving this output?

Patrick · October 4, 2014, 9:01pm

Pushed a small change to make IP and PORT configurable. Please check my commits carefully.

Patrick · October 4, 2014, 9:07pm

[quote=“troubadour, post:45, topic:533”][quote author=Patrick link=topic=560.msg4641#msg4641 date=1412455036]
Config parsing looks good so far. There is a small issue. Stuff might get duplicated.

But I guess there is some easy python sort --unique function?
[/quote]

It should not do that. Could you post your cpfpy.d content giving this output?[/quote]

I am using default config. (Because during development, I often just do “sudo make install”.)

Also for debuging I added “print WHITELIST” below “#print MAX_LINESIZE” so it looks like this.

#print MAX_LINESIZE
print WHITELIST

/etc/cpfpy.d/50_user contains.

/etc/cpfpy.d/51_user also contains.

Result:
[‘SIGNAL NEWNYM’, ‘GETINFO net/listeners/socks’, ‘GETINFO status/bootstrap-phase’, ‘GETINFO status/circuit-established’, ‘GETINFO dormant’, ‘GETINFO network-status’, ‘GETINFO dormant’, ‘GETINFO network-status’]

troubadour · October 4, 2014, 9:31pm

Pushed a small change to make IP and PORT configurable. Please check my commits carefully.

Checked and tested in real situation. OK.

/etc/cpfpy.d/50_user contains.
CONTROL_PORT_FILTER_WHITELIST=GETINFO dormant,GETINFO network-status
/etc/cpfpy.d/51_user also contains.
CONTROL_PORT_FILTER_WHITELIST=GETINFO dormant,GETINFO network-status
Result:
[‘SIGNAL NEWNYM’, ‘GETINFO net/listeners/socks’, ‘GETINFO status/bootstrap-phase’, ‘GETINFO status/circuit-established’, ‘GETINFO dormant’, ‘GETINFO network-status’, ‘GETINFO dormant’, ‘GETINFO network-status’]

The list is not sorted, but what is the point of having two identical configuration files?

Patrick · October 4, 2014, 9:50pm

Packages such as whonix-gw-torsion and whonix-gw-onionshare might want to add same/similar additions to cpfp. The similarities should not be duplicated in WHITELIST.

troubadour · October 4, 2014, 9:54pm

Pushed “remove duplicates in WHITELIST”. remove duplicates in WHITELIST · troubadoour/control-port-filter-python@9956b8f · GitHub

troubadour · October 4, 2014, 9:57pm

There are many ways in Python. :).

Patrick · October 4, 2014, 11:00pm

Three more changes by me. Please feel free to refactor the hell out of them if you don’t like the variable names or way of implementation.

Do you mind me making some simple changes as I am doing at the moment? I want to offload some feature requests from my side from you and I am also learning more python as I do.

Would we benefit from using python-stem for authentication? I guess probably not. I’ll ask Damian Johnson (python-stem maintainer).

And perhaps please explain if anything, stylistic or security wise or else… At the moment it’s this.

[...]
                if line.startswith('CONTROL_PORT_FILTER_PORT'):
[...]
                    PORT = int(value.strip())
[...]
    server.bind((IP, PORT))
[...]

What about changing that to.

[...]
                if line.startswith('CONTROL_PORT_FILTER_PORT'):
[...]
                    CONTROL_PORT_FILTER_PORT = int(value.strip())
[...]
    server.bind((IP, CONTROL_PORT_FILTER_PORT))
[...]

To make the code a little simpler?

troubadour · October 5, 2014, 12:00am

Three more changes by me. Please feel free to refactor the hell out of them if you don't like the variable names or way of implementation.

Just one thing. Why that ?

        first_word = request.split(' ', 1)[0]

        # Authentication request from Tor Browser.
        if first_word == "AUTHENTICATE":

The original from Tails was

        if request.startswith("AUTHENTICATE"):

which performs the same action. I changed to “==‘AUTHENTICATE password’” because it is the exact string sent by TBB before a request. We could revert to the original.

Do you mind me making some simple changes as I am doing at the moment? I want to offload some feature requests from my side from you and I am also learning more python as I do.

That is what collaborative development is all about, I think. Don’t worry about putting some load on me.

Would we benefit from using python-stem for authentication? I guess probably not. I'll ask Damian Johnson (python-stem maintainer).

I don’t know, but in my opinion, there is something illogical in the way we send the requests to controlport. We use stem in the client (whonixcheck, sdwdate…), a python socket on the server side of cpfp, and another python socket to put the request to controlport. It would make sense to do it the other way around, that is, stem in cpfp (client side). I do not know if it’s feasible (there is an issue passing the request string to stem), but it would have some advantages, for error handling, firstly.

And perhaps please explain if anything, stylistic or security wise or else... At the moment it's this.
[...]
                if line.startswith('CONTROL_PORT_FILTER_PORT'):
[...]
                    PORT = int(value.strip())
[...]
    server.bind((IP, PORT))
[...]
What about changing that to.
[...]
                if line.startswith('CONTROL_PORT_FILTER_PORT'):
[...]
                    CONTROL_PORT_FILTER_PORT = int(value.strip())
[...]
    server.bind((IP, CONTROL_PORT_FILTER_PORT))
[...]
To make the code a little simpler?

Yes. Why not

     server.bind((CONTROL_PORT_FILTER_IP, CONTROL_PORT_FILTER_PORT))

Patrick · October 5, 2014, 12:18am

Just one thing. Why that ?

        first_word = request.split(' ', 1)[0]

        # Authentication request from Tor Browser.
        if first_word == "AUTHENTICATE":

The original from Tails was

        if request.startswith("AUTHENTICATE"):

which performs the same action. I changed to “==‘AUTHENTICATE password’” because it is the exact string sent by TBB before a request. We could revert to the original.[/quote]
Revert to original sounds good. (request.startswith(“AUTHENTICATE”) The reason is, the “AUTHENTICATE password” thing is a Whonix-only thing. Other clients might use other passwords. And teaching them to use literal “password” seems odd. Accepting any password seems more appropriate here. By accepting any password, it can be used by others. Possible Whonix-Custom-Workstation with other passwords. More likely inclusion into Debian.

[Or maybe someone some day wants to publish a real password authentication feature pull request, for whatever that would be worth.]

[quote]Do you mind me making some simple changes as I am doing at the moment? I want to offload some feature requests from my side from you and I am also learning more python as I do.[/quote]
That is what collaborative development is all about, I think. Don’t worry about putting some load on me.

Patrick · October 5, 2014, 2:03am

Sounds like we should go for stem.

I see. Yup, Stem could simplify the authentication. All you're aiming to do is pass along raw whitelisted controller commands so this is pretty simple...
% cat scrap.py
from stem.control import Controller

with Controller.from_port() as controller:
controller.authenticate()
print controller.msg(‘GETINFO version’)

% python scrap.py
version=0.2.6.0-alpha-dev (git-b408125288ad6943)
OK

Cheers! -Damian

On Sat, Oct 4, 2014 at 5:33 PM, Patrick Schleizer adrelanos@riseup.net wrote:

Yes, the readme isn’t up to date yet.

For more info, please see…

Package description:

control-port-filter-python/debian/control at master · adrelanos/control-port-filter-python · GitHub

Longer description:

onion-grater, a Tor Control Port Filter Proxy

Python server code:

https://github.com/adrelanos/control-port-filter-python/blob/master/usr/lib/cpfp.py

Damian Johnson:

Hi Patrick. From the name I would guess so but first I’d need to know
more about what you’re desiring for it to do. The readme on the second
link oddly didn’t describe what it does so much as a variety of
packaging and development details.

And yup, I don’t mind if you forward answers I give to the whonix forum.

On Sat, Oct 4, 2014 at 4:03 PM, Patrick Schleizer adrelanos@riseup.net wrote:

Dear Damian,

we’re working on control-port-filter-python [1] [2].

I was wondering, if we could benefit from using python-stem (for
authentication or what else)?

And would you mind if I posted your answer in the forums?

Cheers,
Patrick

[1] Whonix Forum
[2] GitHub - troubadoour/control-port-filter-python

troubadour · October 6, 2014, 11:26pm

I have started with stem in cpfp. Nearly there except for whonixcheck in Gateway, the problem being that it is writing directly into tor controlport, I believe. Is there a particular reason for that, or could we use 10.152.152.10 ? It’s too long to explain why in details now, but it would make my life easier.

Patrick · October 6, 2014, 11:34pm

whonixcheck on Whonix-Gateway indeed does not use CPFP and directly uses Tor’s control port. So does arm / Vidalia. This is on purpose. It should not cause any issues.

No idea what you mean by “could we use 10.152.152.10”.

troubadour · October 6, 2014, 11:43pm

No idea what you mean by "could we use 10.152.152.10".

I mean using CPFP instead of Tor control port directcly.

Patrick · October 6, 2014, 11:47pm

Not sure I am following.

I would prefer if whonixcheck on Whonix-Gateway could continue using Tor’s control port directly. Interesting for those who do not wish to use CPFP.

Also I don’t see the advantage. CPFP shouldn’t break direct access to Tor’s control port.

troubadour · October 7, 2014, 4:21pm

Created a new branch https://github.com/troubadoour/control-port-filter-python/tree/CPFP-stem.

The major changes:

tor_bootstrap_check.py and tor_circuit_established_check.py are rewritten. They use a socket to talk to CPFP and Tor control port. It is necessary because stem being in CPFP, the Workstation was using stem over stem.
CPFP is handling stem exceptions. All exceptions are returned to the client, so may be some of them could be parsed in whonixcheck.

Also I don't see the advantage. CPFP shouldn't break direct access to Tor's control port.

If all requests were channeled through CPFP, the same, more robust exception handling would be used.