sdwdate and sdwdate-gui development thread

Update:

In sdwdate:

  • if len(self.already_picked_index_pool) >= len(self.pool):
  • changed icons path.
  • logger.info in sigterm handler (changed logger level from DEBUG to INFO)
  • added python-gevent in debian/control

In sdwdate-gui:

  • added icons
  • added items in context menu
  • the EOFError looks like a race condition (was wondering about that, it should not). Added a short sleep before reading the status file.
  • show a status message on left click (does not show at startup, looking int it).
Can we make the passive tooltip text (mouse hover over) copyable? Only if not too much effort.

When you keep your mouse on that tooltip, the tooltip won’t change until you move your mouse away and there again. Is this something that could be made more dynamic?


This is true for the left click message too. Have to close and reopen it to show the change. At the moment, could not find a solution without the tooltip or the info message becoming a popup on each status change. Not sure it’s feasible…

Will be back for the remaining.

Fixed.

The sleep seems useful. (sleep in status_changed() to prevent race condition · troubadoour/sdwdate-gui@e0bb470 · GitHub)

with open(self.status_path, 'rb') as f:

However, I still think it should be encapsulated within exception handling. Anything can go wrong, will go wrong. Would be a pity to have sdwdate-gui crashing on some systems some times.

Not sure what exactly the exception handling should do in case of an exception. sdwdate-gui is just a single sequential thread? And will stay that way? And that’s simple and fine. So in that case perhaps wait 5 seconds, try reading again, and if still failing, give up by explaining the error. Eventually suggest to restart sdwdate-gui and/or sdwdate?

I have an idea to perhaps get ride of the technical term “clock jump”. What about this…

  • Gradually adjust the time.
    vs
  • Instantly adjust the time.
    What do you think?

The sleep seems to solve the python exception. It has been running smoothly from the cli for a few hours.

I will recreate the error, to check if it actually prevents the icon and message updates. I think not, but better double check.
Anyhow, we could at least catch the error, and perhaps issue a warning after a wait period, or after an error count if it does not affect the performance of sdwdate-gui (still updating).

I have an idea to perhaps get ride of the technical term "clock jump". What about this... - Gradually adjust the time. vs - Instantly adjust the time. What do you think?
Much better. "clock jump" can be scary.

[quote=“troubadour, post:81, topic:1137”][quote]
When you keep your mouse on that tooltip, the tooltip won’t change until you move your mouse away and there again. Is this something that could be made more dynamic?[/quote]
This is true for the left click message too. Have to close and reopen it to show the change. At the moment, could not find a solution without the tooltip or the info message becoming a popup on each status change. Not sure it’s feasible…[/quote]
It is feasible. The tooltip is updated on status change.

For testing, you can leave the mouse on the tray icon and restart sdwdate from a terminal.

Will try updating the left click status (otherwise it would not make sense).

Please only check bootclockrandomization [and timesanitycheck] if it’s actually installed. I am not sure how to best detect this. A Debian specific way would be:

dpkg-query --show --showformat='${db:Status-Abbrev}' "bootclockrandomization"

Which exits “0” and outputs ‘ii’ if properly installed. Otherwise exits “1” and outputs and error message (that should not be shown).

Checking the systemd status is one test. I am not sure it’s necessary, though.

A better way might be to run:

sudo /usr/share/timesanitycheck/status

Or respectively:

sudo /usr/share/bootclockrandomization/status

Those exit zero if everything is alright. Otherwise non-zero.

This is because don’t know if it’s the correct way to have the timesanitycheck systemd service enter the failed state if the check failed. That could be a bug. The purpose of systemd if to start daemons and scripts at the right time. I don’t know if having it reflect statuses would re-purpose it. Still wondering how it’s best implemented.

Checkig timesanitycheck and bootclockrandomization with dpk-query. “check_output” returns the ouput only if th command exits zero, otherwise raise CalledProcessError.

Using dpk-query on the assuption that if the package is not installed, the corresponding directory in /usr/share won’t exist.

Have to find the way to integrate the potential errors in the icon.

Sorry, just today while I worked on ⚓ T390 whonixcheck should test for unwanted packages such as chrony, ntp, ntpdate and other anon-banned-packages I learned, that dpkg-query exit codes are not reliable for this task. It could exit 0 for a package that is uninstalled but cached.

The following bash code translated to python however should do.

dpkg_query_result="$(dpkg-query --show --showformat='${Status}' "package-name" 2>/dev/null)" || true
if [ "install ok installed" = "$dpkg_query_result" ]; then

Alternatively we could also just check if /usr/share/timesanitycheck/status and/or /usr/share/bootclockrandomization/status are executable? Specifically when we run those scripts anyhow. That would be more generic, less dependent on Debian and/or systemd.

(If these are not installed, please don’t add it to the gui output. Too confusing. These are optional.)

Also didn’t we want to leave checking those to sdwdate-daemon rather than sdwdate-gui?

Also didn't we want to leave checking those to sdwdate-daemon rather than sdwdate-gui?
My understanding was that we would try to leave sdwdate-daemon free from anonymity stuff, for possible inclusion upstream. https://www.whonix.org/forum/index.php/topic,1301.msg9338.html#msg9338

But now sdwdate-daemon cannot be dissociated from sdwdate-gui. That’s why I was thinking of a separate [simple] package that would do the check when the GUI loads (reporting in any non intrusive fashion).

Right. Yes. Maybe I am also confused. With that post I wanted to explain why it was a plugin and whatnot. And failed to express that.

If we wanted to go for ideal upstream inclusion chances and separation, the ideal way would be to come up with a plugin infrastructure. But that seems needlessly difficult to me.

I guess it’s best to add it to sdwdate-daemon. If available, use it. Otherwise, silently skip it. [Unless more verbose.]

Timesanitycheck is something worthwhile to run before the first fetch as well as after setting the time. But wait… Why after setting the time? Let’s rethink…

Why check bootclockrandomization at all? While reimplementing this, we just copied the previous implementation. Not a great reason. That package bootclockrandomization is working pretty well for a long time now. Seeing it broken is unlikely. If we wanted, we could check bootclockrandomization in whonixcheck. Finding out if it was ever broken could be part of the usual Whonix development / testing work.

And as for timesanitycheck, actually that’s a pretty simple action. See:
https://github.com/Whonix/timesanitycheck/blob/master/usr/share/timesanitycheck/start

Summary:

  • After start: Gets a timestamp. It gets the current unixtime for use in script calculation and human readable date/time for pretty output.
  • Reads a few more existing time stamps. Build timestamp.
  • And expiration timestamp.
  • Then compares.
  • if BOOT_UNIXTIME is lower than BUILD_UNIXTIME → pretty your clock is fast log output
  • if BOOT_UNIXTIME is greater than EXPIRATION_UNIXTIME → pretty your clock is slow log output
  • creates status files as appropriate but that is not so important

I am wondering if the whole thing should be reimplemented in python? Or if I should just deprecate the package and make the scripts easily accessible from python? A timesanitycheck does have nothing to do with anonymity. Is useful.

If the clock is slow before fetching for the first time, and most likely therefore cannot be set, it’s a helpful information to fix the issue. And before setting the new timestamp it’s also useful to check if it’s within a sane range, that is between BUILD_UNIXTIME and EXPIRATION_UNIXTIME. Otherwise, drop the fetched time result as invalid.

I hope that makes a lot more sense?

(There is another sanity check, “sdwdate Tor Consensus Time Sanity Check” - ⚓ T151 sdwdate Tor Consensus Time Sanity Check, that can be added at a later time after the first release. Building such a sanity check into sdwdate is also independent from anonymity stuff, because sdwdate now depends on Tor anyhow. So why not use also Tor as a source for sanity checking. From that perspective, an external timesanitycheck package makes less sense.)

I hope that makes a lot more sense?
Yes. I summarize.
  • bootclockrandomization check in whonixcheck (would be OK from my point of view).
  • timesanitycheck is run before fetching for the first time → warning if the clock is either slow or fast.
  • timesanitycheck is run every time before setting the new date → re-fetch if off limits.

Even if we would have to (probably) reuse some bash commands, this could be reimplemented as a python module.

Before starting on sdwdate, I had a quick look at https://phabricator.whonix.org/T151. I believe that it could be the next step, fetching the tor consensus with stem in the Gateway, and using anondate in the Workstation (if I remember correctly the conclusions, will have to read through again).

Yes.

- timesanitycheck is run before fetching for the first time -> warning if the clock is either slow or fast.
Actually, when that state has been detected by sdwdate-daemon, you can show a red status message explaining that and give up. Fetching a new time then won't be possible anyhow. [And if it was, it would not be secure.] Fall back to the next sleep cycle.

As a related note, I am wondering if we should include a clock jump detection. If sdwdate is within the sleep cycle [or anywhere?] and a clock jump is detected [the user roughly fixing the clock], then stop the sleep cycle? But this can be done after the first release.

- timesanitycheck is run every time before setting the new date -> re-fetch if off limits.
Yes. (But also not unlimited retries. Going through the usual sleep circles also.)
Before starting on sdwdate, I had a quick look at https://phabricator.whonix.org/T151. I believe that it could be the next step, fetching the tor consensus with stem in the Gateway, and using anondate in the Workstation (if I remember correctly the conclusions, will have to read through again).
anondate won't work on the workstation. Anondate requires /var/run/tor/log. Anondate and Tor ControlPort (python-stem) have both pros and cons. Information from Tor's ControlPort is much more reliable and easier to parse. Requires no local disk access to /var/run/tor/log. While anondate can obtain more information. I'll update the ticket. Compare those and write feature requests for the missing information not obtainable from Tor's ControlPort. But this stuff is difficult design wise. HulaHoop and I have been researching and discussing these things for quite some time now. Can be implemented after the first release.

FYI, anondate vs Tor ControlPort is now documented:

Wrote a hack to update the message shown when left clicking the icon.

Some explanation. Could not find a way to know if the message is showing, so we allow updating the ballon for a period of 10 seconds after the click, the default showing time.

Works fine as long as the mouse is not left inside the balloon. In that case, the update happens if the fetching time is less than 10 seconds, otherwise the icon is frozen. That’s a default behaviour, the user has to click it to hide it, or click sdwdate-gui ican to update the message.

It’s an inconvenience, but the only spare solution would be popup GUI (right above the icon, but still rather ugly).

Can we make the passive tooltip text (mouse hover over) copyable? Only if not too much effort.
Could add a menu item: "Copy last message".

Could add a menu item: “Copy last message”.[/quote]
Hm. Dunno. Any opinions?

I don’t want to overload it. If it can be copied out after the left click action (the active popup or whatever the left click action is going to become). But it’s also useful. So let’s wait for other opinions or use your best judgment.

While you’re at the sdwdate rewrite, could you also implement please
“Only source configuration files that end with the .conf extension?” (⚓ T286 Only source configuration files that end with the `.conf` extension?)
? (Unless that seems like a bad decision for some reason.)

(Unless that seems like a bad decision for some reason.)
For the python code, cannot see any reason why not. It would make the code simpler. If we agree on that, I would do cpfpd too.

Since no one spoke up against it, I guess that’s fine.

workaround for ‘dh_installinit should run systemd-tmpfiles if a /usr/lib/tmpfiles.d/ snippet gets shipped for systemd-only packages also’ - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795519

That means, that the daemon now works right after installation. No reboot required anymore. This wasn’t a visible problem, because we are installing in chroot inside the image without starting services and then boot. After boot, the tmpfiles will be created. This matters for anyone installing sdwdate from a repository and/or as non-Whonix user on Debian installing the package from source.

Please add the fix also to the python branch.

git cherry-pick b66ca854ea1d2be60b738582723acc849921c830

git cherry-pick returns

Tried to change branch (origin/master), fetch, back to python branch, same result.