sdwdate and sdwdate-gui development thread

Development
32
- Create a separate etc/sdwdate-python.d/30_sdwdate_default configuration file. Modifying the original crashes sdwdate bash (expected).
Let's please just keep /etc/sdwdate.d. Incompatibility with sdwdate-bash is fine. sdwdate-bash will be deprecated. Transition (Whonix 11 sdwdate-bash -> Whonix 12 sdwdate-python) will most likely be smoothly. The package management stops sdwdate-bash, replaces the configuration file and starts sdwdate-python. Changing the folder would generate extra work, require to also update the sdwdate plugins as well as qubes-whonix package.
33

This is a temporary solution. If I modify directly the existing default file, sdwdate bash does not work. As sdwdate-python package progresses, it should be easy to revert to the original /etc/sdwdate/ 30_sdwdate_default.

34

Okay. Nevermind. But do we need those co-installed for development purposes? Anyhow. Minor detail. Nevermind.

35
Ah. Yes. While we're at it, maybe those set by date vs sclockadj situations also need some scrutiny/rethinking.
Plunging into sneaky clock adjust. I'm probably missing something, but a remark nonetheless.

sclockadj is running more or less permanently in the background. Every second [SDWDATE_SCLOCKADJ_WAIT_MIN = $SDWDATE_SCLOCKADJ_WAIT_MAX], it add /subtract 500 microsecond [SDWDATE_SCLOCKADJ_MOVE_MIN = SDWDATE_SCLOCKADJ_MOVE_MIN] to the clock, that is 1.8 second per hour. No problem there. But, does that mean that all Whonix users share the same clock drift when sclockadj is running? That might be an advantage In case of clock skew fingerprinting, as the skew cannot be pinned to an individual user, but it seems that an attacker could easily tie it to Whonix.

36

With sclockadj, it’s “same as NTP”. Hopefully.

Quote from TimeSync: Whonix Time Synchronization Mechanism

For example standard Linux adjusts the time with a rate of 0.5ms per second.

With sclockadj2 we would be using the exact same function as NTP does, therefore be even closer to NTP:

37

Some (slow) progress with sclockadj. Looking into sclockadj2. What is the status with sclockadj2: initial implementation of sclockadj2 using adjtimex syscall by AvdN · Pull Request #4 · Kicksecure/sdwdate · GitHub ?

38

Stalled. (Already contacted the author.)

39

Pushed a batch of commits in GitHub - troubadoour/sdwdate: Secure Distributed Web Date; privacy, anonymity and Tor friendly; console time fetcher and daemon; optional graphical user interface etc. Website: https://www.whonix.org/wiki/sdwdate

Could you review the logic? At the moment, time setting mimics the original sdwdate:

  • set time using date → boot and timesync.
  • run sclockadj → after sleep period or sdwdate service restart.

Note. When installing the package from python branch, sdwdate bash is removed. For testing the behaviour of timesync (which reports a failure at this stage), I had to reinstall /usr/lib/sdwdate/restart_fresh.

40

Forget the note above.
missing files · troubadoour/sdwdate@907cd7c · GitHub installs the required files.

41

Yes, this is expected. When files are removed from the package and the package is reinstalled, then old files are removed. (With exceptions, /etc.)
The solution is to have these files in your branch in the correct location. Then next time they will be [re]installed. Seems like you did that already.

42

Please restore:
debian/sdwdate.postinst
debian/sdwdate.postrm

Not sure the move of sdwdate to /usr/sbin is good. Maybe /usr/bin is better. Maybe some day we want to support a oneshot, non-daemon operation mode? I.e. type “sdwdate” -> replies with time.

43

[quote=“troubadour, post:30, topic:1137”]- Made all the pool members “single line”.
https://github.com/troubadoour/sdwdate/commit/b3520602181e368ae563182b8b1c7032dc044d08[/quote]
I think this was a misunderstanding. Now previously grouped pool members, are not a group anymore. Therefore those members are chosen much more often. Thereby get more influence.

Not good:
meld etc/sdwdate.d/30_sdwdate_default etc/sdwdate-python.d/30_sdwdate_default

What I meant by ‘Make all pool members “multi lined”’…

Original sdwdate-bash:

SDWDATE_POOL_TWO=(
         "
            atlas777hhh7mcs7.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            compass6vpxj32p3.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            globe223ezvh6bps.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            bbbbbb6qtmqg65g6.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            pppppptkftqqnfsq.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
         "
         "w6csjytbrl273che.onion#Ljost[24][25] 	2012-September-30 	Transparency Activism 	w6csjytbrl273che.onion 	https://w6csjytbrl273che.tor2web.org/ 	Iceland"
         "ak2uqfavwgmjrvtu.onion#MagyarLeaks[26] 	2013-July-7 	Investigative Journalism 	ak2uqfavwgmjrvtu.onion 	https://ak2uqfavwgmjrvtu.tor2web.org 	Hungary"
         ...
)

Proposed if that helps parsing:

SDWDATE_POOL_TWO=(
         "
            atlas777hhh7mcs7.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            compass6vpxj32p3.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            globe223ezvh6bps.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            bbbbbb6qtmqg65g6.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
            pppppptkftqqnfsq.onion:80#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html
         "
         "
            w6csjytbrl273che.onion#Ljost[24][25] 	2012-September-30 	Transparency Activism 	w6csjytbrl273che.onion 	https://w6csjytbrl273che.tor2web.org/ 	Iceland
         "
         "
            ak2uqfavwgmjrvtu.onion#MagyarLeaks[26] 	2013-July-7 	Investigative Journalism 	ak2uqfavwgmjrvtu.onion 	https://ak2uqfavwgmjrvtu.tor2web.org 	Hungary
         "
         ...
)

In other words, from the initial random picking of a pool member “Thomas White”, a multi lined mirror, i.e. a pool member providing mirrors should only appear as one. Not many. And upon picking such a multi lined pool member “Thomas White”, some random mirror he is providing gets chosen.

44

typo: eaxh

                if len(self.urls) == 0:
                    ## Most likely, internet connection is down.
                    ## Raise eror, log.
                    message = ('No values returned from url_to_unixtime. Internet connection might be down.')
                    print(message)
                    logger.critical(message)
                    sys.exit()

Please don’t exit. The daemon must be able to cope up in case internet connection is down.

    def general_proxy_error(self, pools):
        '''
        This error occurs (at least) when Tor is not running.
        '''
        if (pools[0] == 'Connection closed unexpectedly' and
            pools[1] == 'Connection closed unexpectedly' and
            pools[2] == 'Connection closed unexpectedly'):
                ## Raise error, log, user warning.
                logger.critical('General Proxy Error')
                sys.exit(1)

Also no reason to exit.

The right fall back action in such special cases is falling back to the loop (which will then continue with a randomized sleep).

45

Bug: If set_new_time returns false, because median equals zero, then the first_success_file does not get created. However, such cases can be viewed as sdwdate having run successful. first_success_file will be used as an indicator by applications (gui) using state information provided by sdwdate.

46

[quote=“Patrick, post:42, topic:1137”]Please restore:
debian/sdwdate.postinst
debian/sdwdate.postrm[/quote]

Not sure the move of sdwdate to /usr/sbin is good. Maybe /usr/bin is better. Maybe some day we want to support a oneshot, non-daemon operation mode? I.e. type "sdwdate" -> replies with time.
Yes, moved it to /usr/bin. There are some minor changes in sdwdate that I forgot to commit earlier. https://github.com/troubadoour/sdwdate/commit/2142fd9a22cf4e857090d13c617465b205d083f6

Restored the multi-line entries (groups) in 30_sdwdate_default. The new config.py can read any number of groups. It appends one random member from each group to the pool list.
For consistency, the groups are separated by a line with one double-quote (could be anything), but the members have the same format as the single entries, starting with a double-quote. That lets us use the same regular expression operation for parsing the urls from single or multi entries.

The sys.exit() are only temporary solutions. Have yet to implement the error handling.

Bug: If set_new_time returns false, because median equals zero, then the first_success_file does not get created. However, such cases can be viewed as sdwdate having run successful. first_success_file will be used as an indicator by applications (gui) using state information provided by sdwdate.
Yes. Will modify that.
47

There is still a diff. Some stuff that I don’t understand.

kdiff3 etc/sdwdate.d/30_sdwdate_default etc/sdwdate-python.d/30_sdwdate_default

Please check.

Experimenting with a few inline comments:

48

The inline comments are quite useful.

I’ve been making some experiments with etc/sdwdate-python.d/30_sdwdate_default, hence the readability issue.

Pushed a new one which should better show the differences.

  • replaced the extraneous quote by square brackets. For parsing the groups, a marker is required, anyhow [the modified config.py is in a separate commit].
  • added the missing quotes.
  • some entries are redundant, or misspelled (missing .onion). Removed or edited.
  • the parsing script needs a consistent format for the urls, ex. "cwoiopiifrlzcuos.onion#. The ones with the port extension :80 are edited.
49

Pushed two commits:

  • error handling.
  • test .d configuration files with invalid pools.

There is error logging only so far, because I’m trying to find the way to pass messages from sdwdate to the timesync systray icon, which is running its own event loop. Some progress…

50

Moved the test configuration files to /usr/share, because they stop sdwdate (to be investigated).

51

Yes, we haven’t thought through yet how sdwdate would communicate with the tray icon. I think RPC can and should be avoided for simplicity. The tray icon should omit minor details. For example, if fetching some onion failed but time synchronization still succeeded, this information can be buried in sdwdate’s log.

I think ‘Sdwdate Connection Checker Plugin’ - https://github.com/Whonix/sdwdate-plugin-anon-shared-con-check - no longer needs to be a plugin. Because sdwdate depends on Tor now so or so. Not sure if it would still be useful to have the code that determines the status of connectivity would be configurable / replaceable by custom code (setting).

I made a first sketch how this interaction could work:

Please feel free to make bold changes. This needs some hard criticism, so sdwdate-tray can provide good usability.