onion-grater development

This pr is not good enough in terms or merging profiles.

Multiple dirs is working fine.

Multiple matches is not, it is more difficult than what I expected.

Sorting files in lexical order does not make sense because onion-grater stops at the first, even it make in reverse, the first match could not be enough for all cases.

If for example the a file containing hosts: '*' is matched, every host will only use that file, and no other even if it contains hosts: '*' in the other files.

The same apply even if hosts are specified such as hosts: '10.137.0.15' and the host match, it will not use any other file even if the host could match, it stops at the first file.

Unfortunately this is not done already on the onion-grater side because it was not written with that in mind.

What could be better done is transforming onion-grater-merger to separate files per hosts.
Lets take for example these files:

  • a.yml - hosts: ‘10.10.10.10’
  • b.yml - hosts: ‘*’
  • c.yml - hosts ‘10.10.5.5’, ‘10.10.10.10’

The file b needs to be merged to all profiles.
The file a needs to be merged to the c profile
The file c needs to receive all profiles that either have all hosts or the matching hosts.

But this is just one aspect, the hosts, Tails uses apparmor-profiles and users, which lead to all sort of combinations of the qualifiers apparmor-profiles, users, hosts.

This means that there need to be a good way to merge in memory per the qualifiers, which gets more difficult with more qualifiers.
Lets take a bigger example know.

  • a.yml
    • apparmor-profiles:
      • ‘/usr/bin/onionshare’
      • ‘/usr/bin/onionshare-gui’
    • users:
      • ‘amnesia’
      • ‘user’
    • hosts:
      • ‘10.10.10.10’
      • ‘10.10.5.5’

The three qualifiers are lists, they can hold multiple values. And this is only one profile, if you have to merge them in a way that it always results well.

From my tests, this was accepted by onion-grater:

- apparmor-profiles:
    - '/usr/bin/onionshare'
  users:
    - 'amnesia'
  hosts:
    - '*'

it matched only the hosts.

From code comments

A filter is matched if for each of the relevant qualifiers at least one of the elements match the client. For local (loopback) clients the following qualifiers are relevant:

Upon tests:

Matchers: [('hosts', '10.137.0.44')]

Matched_filter: {'exe-paths': ['/usr/bin/onionshare'], 'users': ['amnesia'], 'hosts': ['*'], .........

So any qualifier matching, onion-grater will happily accept that. You won’t harden the profile by setting all three.

Profiles that accept every host should be merged with profiles accepting any host.
Profiles that have the same host, should be merged together.

On the other hand, by not specifying a qualifier, it will never match by that qualifier.

Match prefix is outdated for the merger, better to remove it.
And change exe-paths for apparmor profile.

1 Like

Broken links across the wiki because of wide search and replace.

In Whonix ™, there is only limited access to Tor’s control port (see [[Dev/onion-grater
|Control Port Filter Proxy]] for more information).

1 Like

Fixed.

Remember to separate the profiles per application, newnym is for TBB, bootstrap-phase for sdwdate etc, much better to understand which application needs what.

1 Like

Branch I am using GitHub - nyxnor/onion-grater at enhance

  • allow to match multiple profiles and merge them together per matcher basis
  • allow to set restrict-stream-events for remote hosts
  • allow alternative directories to be parsed

after review, I will pr upstream.

1 Like

Note to self: I still need to look into this.

1 Like

Hi.

1 Like

About merge:

This is rather complex. And there’s low user demand for it. Various levels:

  1. Usability challenges of even understanding what Whonix is due to it’s unique dual-VM design, getting it installed at al. Few users.
  2. The usual Linux usability issues. Even less users.
  3. Usability issues of using applications that require onion-grater. Even less users.
  4. Users that want more fine-grained onion-grater control.

Hence, it’s a very detail and that’s not really something I want to spend a lot time on. The issue is also that these changes can probably not be upstreamed.

Debugging user reports of onion-grater issues is time intensive and painful already. By making onion-grater more complex, this adds only to it.

So what could be done? “A + B”

A) Keep the “normal” onion-grater as currently is - /usr/lib/onion-grater-merger remains unchanged.
B) Add the forked version that you invented as a separate script - add a new script such as /usr/lib/onion-grater-merger-somename. Not ideal to have code duplication but at least no regression and easy beta testing, easily revertible.

What could be “somename”? /usr/lib/onion-grater-merger-granular? Something better?

Then in the wiki users who wish to use the more granular way to use onion-grater / beta test can opt-in for it. (Using a wiki tab controller or something.)

Thoughts?


review and PR upstream:
Didn’t test. Best not to wait for me to unlock your progress. Generally looks good but it would require lots of testing to make sure there’s no regressions.

(Just make sure any PR to upstream does not include textual string “Whonix”.)

Even though this decreases the chance of people using it in the beginning, I believe it is worth it as it changes the behaviour and therefore the output.

1 Like

A change that could be implemented directly to onion-grater-merger is restrict-stream-events to work for remote hosts (filters streams per client/host).

Unfortunately I did a bad commit practice and used 1 commit for many changes… enhance onion-grater · nyxnor/onion-grater@d67b76f · GitHub (lines 355-356 and 364-367).

1 Like

Big changes to the Tor control protocol upcoming.

This is now in the testers repository.

In context of SETCONF Tor control protocol command should not be used when system Tor is configured / TOR_SKIP_LAUNCH=1 is not honored (#42510) · Issues · The Tor Project / Applications / Tor Browser · GitLab maybe the following could be used…

    SETCONF:
      - pattern: 'UseBridges.*'
        replacement: 'non-existing-command'
        response:
        - pattern:     '.*'
          replacement: '250 OK'
Apr 09 10:46:36 host onion-grater[15915]: 10.138.28.23:60210 (filter: 30_autogenerated): -> SETCONF UseBridges=0 Bridge Socks4Proxy Socks5Proxy Socks5ProxyUsername Socks5ProxyPassword HTTPSProxy HTTPSProxyAuthenticator ReachableAddresses
Apr 09 10:46:36 host onion-grater[15915]: 10.138.28.23:60210 (filter: 30_autogenerated): rewrote command:
Apr 09 10:46:36 host onion-grater[15915]:     SETCONF UseBridges=0 Bridge Socks4Proxy Socks5Proxy Socks5ProxyUsername Socks5ProxyPassword HTTPSProxy HTTPSProxyAuthenticator ReachableAddresses
Apr 09 10:46:36 host onion-grater[15915]: to:
Apr 09 10:46:36 host onion-grater[15915]:     SETCONF non-existing-command
Apr 09 10:46:36 host onion-grater[15915]: 10.138.28.23:60210 (filter: 30_autogenerated): rewrote response:
Apr 09 10:46:36 host onion-grater[15915]:     552 Unrecognized option: Unknown option 'non-existing-command'.  Failing.
Apr 09 10:46:36 host onion-grater[15915]: to:
Apr 09 10:46:36 host onion-grater[15915]:     250 OK

Taking a command to be filtered (“SETCONF UseBridges=0 Bridge ...”), rewriting it to SETCONF non-existing-command, catch Tor’s response and rewrite to 250 OK.

…but I hope that won’t be needed.

Mentioning here because this might be useful one day for other applications.