Control Port Filter Proxy Python (cpfpy) / anon-ws-disable-stacked-tor

Development
61

I think if we have to bend existing scripts that are users of python-stem just because we rewrote CPFP, then we’re doing something fatally wrong. That seems to mix up abstraction levels. CPFP is supposed to be as transparent as possible. So any application that makes use of Tor ControlPort, using python-stem or not, at best won’t even notice it. Any scripts / applications not developed by Whonix (ex: onionshare or whatever) should not need any further changes than the required white list additions.

It is necessary because stem being in CPFP, the Workstation was using stem over stem.
What's the issue with using python-stem over python-stem? If it's all rightly implemented, those implementations shouldn't notice or care?
If all requests were channeled through CPFP, the same, more robust exception handling would be used.
If we want the same error handling everywhere, we could go for a shared python file.
All exceptions are returned to the client, so may be some of them could be parsed in whonixcheck.
In it's default mode, it should mimic Tor's real ControlPort as closely as possible for 100% compatibility and not send extra messages. Non-standard messages cause confusing error messages (for example in Tor Browser). For applications that know CPFP (from anon-shared-helper-scripts), they could perhaps authenticate using a special authentication password/extra keyword or so, and then they could use the "extended protocol" with extra messages.
62

Points taken. I realize that the modifications are going too far. We can fully mimic Tor control port. without extra error messages returned, and probably add some extended protocol for the anon-share-helper-scripts.

I am back to the original branch. Pushed a small modification (“request.startswith” for authentification). request.startswith() instead of first_word · troubadoour/control-port-filter-python@6792958 · GitHub.

What could be done from now:

  • check that the config folder exists.
  • handle parallel connections.
  • stem over stem (needs some testing).
  • extended protocol (to be defined), if we need any.
63

Sounds good.

I guess stem over stem will work just fine. If not, I would really wonder and that should be considered a bug.

64
I guess stem over stem will work just fine. If not, I would really wonder and that should be considered a bug.

stem over stem should work. It seems to be a matter of what command to use. “controller.GET_INFO(“request”)” does not give the same output format as “controller.msg(“GETINFO request”)”.

65

The one is in parsed format, the other (from Damian’s mail) is raw format?

Anyhow. Existing anon-shared-helper-scripts from Whonix 9.x shouldn’t require changes because of replacing cpfp-bash with cpfp-python. Any improvements to them would be off-topic bonuses.

66
The one is in parsed format, the other (from Damian's mail) is raw format?

No, I rephrase: the format is the same, but the one from Damian adds a line “OK”, which confuses the anon-share-helper-scripts. But I’m not there yet.

If the configuration folder does not exist, do we stop CPFP or do we run it with a default hard coded configuration, the one in “30_controlportfilt_default”?

67

No, I rephrase: the format is the same, but the one from Damian adds a line “OK”, which confuses the anon-share-helper-scripts. But I’m not there yet.[/quote]
The “OK” line is Tor ControlPort’s raw answer. stem is only relaying this, I think.

If anon-share-helper-scripts (python-stem) gets confused by it, this is a bug in cpfp-python.

For debugging such bugs one developers-only option might help: disable all filtering. We can either add this as an option or temporary disable the related code during development phase ourselves.

As said earlier, the ultimate test for any cpfp for relying bug freeness is running arm trough it. (While having filtering temporarily disabled.) Arm is a much better test candidate than anon-shared-helper scripts, because arm is not written by us and because it is quite picky (knowing this from cpfp-bash development) (about things like a minor missing \r) and using a lot Tor ControlPort commands.

[Let cpfp listen on gateway only while having filtering disabled for better security during development.]

Using hardcoded defaults is the proper and policy conform way to design (Debian) Linux daemons. (This is a small bug in cpfp-bash.) And as a bonus perhaps making a note in the log as part of the startup log message.

68

I changed the socket to a TCP server on the server side of CPFP. It should accept parallel connections.

Using hardcoded defaults is the proper and policy conform way to design (Debian) Linux daemons. (This is a small bug in cpfp-bash.)

Going to add the hardcoded defaults.

And as a bonus perhaps making a note in the log as part of the startup log message.

Do you mean a note telling the configuration used (default or from file)? And we do not have a log as such, it’s only a “print” statement.

69

Yes.

Not yet, but it would be useful to have something similar to /var/log/controlportfilt.log.

70

Pushed Default hardcoded configuration · troubadoour/control-port-filter-python@1ddf5df · GitHub (hardcoded configuration).

71

I think there might be an issue with the “lie” around here:
https://github.com/troubadoour/control-port-filter-python/blob/master/usr/lib/cpfp.py#L75
A socket has been opened above, but because of “return(‘250-net/listeners/socks=“127.0.0.1:9150”\n’)” it will never be closed below. I think moving the lie/return right below " with open(AUTH_COOKIE, “rb”) as f:" would solve this.

72

I distinguish between points,

  1. we really should do to have a robust implementation so cpfp-py can replace cpfp-bash, and
  2. things that are minor/bonus, possible further work/patches welcome.

[hr]

One minor/bonus point:
This is a minor/bonus point, because cpfp-bash doesn’t do this either.

What is non-ideal at the moment, that for an open connection from a client, we read that line by line, and for each line that passes the white list we open a Tor ControlPort connection, authenticate, pass the request, read the answer, replay the answer to the client, then and close that ControlPort connection.

To be much more efficient, much better emulate Tor ControlPort, we would only open a Tor ControlPort connection once a client connects, and close that one once the client sends “QUIT” and/or disconnects. During that period we pass legitimate ControlPort commands to the already open and authenticated Tor ControlPort socket and relay the answer back to the client.

What would be the benefit? I think this is equired so clients could register for Tor ControlPort events. (Because I guess Tor does de-register the event as soon the related ControlPort connection is closed.)

When doing this, we would have to make sure that we are not mixing multiple client connections into the same Tor ControlPort connection.

Even if you don’t feel like implementing this, it would be good if we both understand what I am talking about here to better understand all this. Does that make sense?

73

[quote=“Patrick, post:71, topic:533”]I think there might be an issue with the “lie” around here:
https://github.com/troubadoour/control-port-filter-python/blob/master/usr/lib/cpfp.py#L75
A socket has been opened above, but because of “return(‘250-net/listeners/socks=“127.0.0.1:9150”\n’)” it will never be closed below. I think moving the lie/return right below " with open(AUTH_COOKIE, “rb”) as f:" would solve this.[/quote]

Yes, a design bug. I moved it before the “with…” block. Moved the "lie" in front of "with open(AUTH_COOKIE, "rb") as f:" block. · troubadoour/control-port-filter-python@2174f9e · GitHub

Before answering you last post, I think I will add a parameter for disabling filtering, so that we can run arm through control port filter, as suggested in Whonix Forum.

74

Sounds like a good approach.

75

Pushed Transparent mode CONTROL_PORT_FILTER_DISABLE_FILTERING=[true/false] · troubadoour/control-port-filter-python@1358a37 · GitHub.

Added CONTROL_PORT_FILTER_DISABLE_FILTERING. When set to ‘true’, all the request are passed to Tor control port and the answers are relayed back to the client transparently. I had to use ‘sock.recv’ instead the file like ‘sock.makefile’. When I started to test with arm, I was assuming that all the answers form control port where terminated with “250 OK”. It’s far from being the case, there are too many exceptions, hence ‘sock.recv’, which reads all the data available at once.

Except for an intermittent bug. Running several times “arm -i 10.152.152.10:9052” consecutively can crash arm. To reproduce it, you can run CPFP from kate, and run arm through CPFP several times. The requests and the answers are printed as they come.

76
Even if you don't feel like implementing this, it would be good if we both understand what I am talking about here to better understand all this. Does that make sense?

I believe I understand. First of all, I have removed the “QUIT” command form Tor control port side in CPFP. Everything works fine without it.

To make it by the book, I probably will have to send it from the anon-shared-helper-scripts, after the request. I 'll try to find out how TBB or arm handle that (how they supposedly de-register). For example, if a “QUIT” is sent when closing arm.

When doing this, we would have to make sure that we are not mixing multiple client connections into the same Tor ControlPort connection.

I guess the TCP server in CPFP is able to discriminate the connections. To be checked.

77
  • When clients write “QUIT”, CPFP should close the socket.
  • When clients close the socket without writing “QUIT”, CPFP should close the socket as well.
78
- When clients write "QUIT", CPFP should close the socket.

Since “QUIT” is in the whitelist, it’s passed to the control port as the other requests, closing the connection. But the “QUIT” catching in the handle function has to be removed.

            elif request == "QUIT":
                # Quit session (telnet...)
                self.wfile.write("250 Closing connection\n")
                break
- When clients close the socket without writing "QUIT", CPFP should close the socket as well.

The control port socket is closed after each connection prior to returning the answer to the client.

            if DISABLE_FILTERING:
                # Some answers are longer than 8 Kb (arm).
                answer = sock.recv(16384)
            else:
                answer = sock.recv(MAX_LINESIZE)
            ~
            ~
            sock.close()

            return answer

Writing “QUIT” looks redundant. It only returns “250 Closing connection” (with the above “QUIT” block commented out in “handle(self)”).

TBB and arm do not write “QUIT”.

79

Mistake, “QUIT” is not in the white list.

80

But the anon-share-helper-scripts write “QUIT”, because of stem GETINFO.

Edit: if the request was successful.