whonix-ws-16 Template fails to update due to timing issue

The different time zones might be an issue. dom0 / Templates are “normal” but Qubes-Whonix templates are in UTC.

For Debian 11 / bullseye in sdwdate a change was required (and introduced since the first Debian 11 based Whonix version 16) to set in python:

os.environ["LC_TIME"] = "C"
os.environ["TZ"] = "UTC"
time.tzset()

This python change may need to be added to any Qubes source code. For example the python based Qubes dom0 file /etc/qubes-rpc/qubes.GetDate might need it.

Furthermore, to improve the robustness (and perhaps even fix this bug) any invocation of date should be prefixed with --utc, i.e. date --utc for any Qubes shell/bash scripts.

Internally, programmatically Qubes should always handle time in UTC. The time shown to the user when manually running date or looking at the systray can remain as is in user local timezone, no problme.

The Whonix template is ~half a minute in the past. If it’s close enough that the 5 minutes makes a difference (date -s +5min is enough to fix), then I can very well believe that 30s in the past may be problematic at times too. Something clearly is making Whonix’s clock be in the past, and I’d say we should avoid it at all regardless if that’s 30s or 5min.

1 Like

added to wiki just now…

Qubes Time Synchronization Features:

Xen DomU’s get their initial time from Xen (Qubes dom0) at VM start.

  • qvm-sync-clock.service
  • qvm-sync-clock.timer

qvm-sync-clock gets time from Qubes ClockVM.

It is used because otherwise Xen DomU’s would start to clock drift more and more. This this answer by marmarek [archive].

qvm-sync-clock is unwanted in sys-whonix and anon-whonix because sdwdate runs there.

qvm-sync-clock is disabled in Qubes-Whonix ™ Templates until version Qubes-Whonix ™ 16.0.3.7. To be re-considered for later versions. Qubes-Whonix ™ get their time from dom0 at VM startup, which is then randomized using Boot Clock Randomization.

Future: qvm-sync-clock should be equally safe to run inside Qubes-Whonix ™ Templates, if passed though clock-random-manual-cli.


Now working on making timesync in Qubes-Whonix Template similar to Non-Qubes Templates.


https://github.com/Whonix/sdwdate/commit/6215a9ea996e9db970059c3b4ad58d17016b7483

Time to revise:

New plan:
As for Qubes-Whonix Templates, make it similar to /etc/qubes-rpc/qubes.SetDateTime.

I thought suspend/resume as well as long running Templates should be improved by modifying qvm-sync-clock for Qubes-Whonix. Started working on it:

Not in use yet. And was probably in vain. And probably low priority.

Because…


Then if this “small” difference is causing some much issues… The reason for that is probably:

If someone would like to try if that is the case, try the workaround of disabling boot clock randomization.

Related boot clock randomization development discussion:
Boot Clock Randomization - bootclockrandomization

As for usefulness of Boot Clock Randomization in Qubes-Whonix Template, see:

I am still not sure Boot Clock Randomization is the cause since then this issue should be equally happening to Non-Qubes-Whonix users and a lot more users. But to find out…

Now I am a bit more sure this is caused by Boot Clock Randomization. It’s not happening in Non-Qubes-Whonix since that is using sdwdate which has much higher accuracy than Boot Clock Randomization. Qubes-Whonix Templates however use only Boot Clock Randomization and do not use sdwdate at time of writing.

Still not sure why this isn’t happening more often to more users and wasn’t reported earlier.

In summary:

  • Non-Qubes-Whonix: Boot Clock Randomization + sdwdate
  • Qubes App Qubes: Boot Clock Randomization + sdwdate
  • Qubes Templates: Boot Clock Randomization only

Accuracy:

  • Boot Clock Randomization: +/ 180 seconds
  • sdwdate: quite good, often +/- 1 second

Should sdwdate be run inside Qubes Templates?

It would probably be easy to make sdwdate connect through Qubes UpdatesProxy - similar to how already APT and Tor Browser Downloader by Whonix ™ are using networking in Qubes-Whonix Templates.

A more secure solution might be something like ⚓ T387 Qubes-Whonix-Gateway as ClockVM, i.e. Qubes-Whonix Templates receiving a timestamp with sdwdate accuracy from an App Qube, but also quite harder to implement.

@Patrick & @marmarek : Thanks a lot for all your work on tracking down the root cause of the reported issue.

It is not clear to me, if at the moment you need additional information from my system.

FYI: Qubes Updater has (again) successfully updated several fedora templates - but - failed to update ‘whonix-ws-16’.

@Patrick : Is there an update available on when & how this issue might be fixed?

On Qubes R4.0 I’m still experiencing it with every update of a ‘whonix-*-16’ template.

No, if there is, this forum thread will be updated.

See my previous post.

That’s surprising because it never happens to me.

Hello Patrick,

That’s surprising because it never happens to me.

It just occurred again w/ “whonix-gw-16” template. - Let me know, if any of my own system setup details are important for you …

Hi, it just happened to me. Is there a fix for this issue, please? Thank you.

Hello jrnn,

jnrn https://forums.whonix.org/u/jnrn
July 25

Hi, it just happened to me. Is there a fix for this issue, please? Thank
you.

Not that I’m aware of. - In the meantime I’ve upgraded my system to Qubes
OS 4.1, but the problem still persists :frowning:

With kind regards,

Viktor

Documentation written just now:

InRelease is not valid yet error

(Whonix is based on Kicksecure.)

I’m resurrecting this thread, because the issue is still happens in Whonix 17. And is rather common with Debian fasttrack repository. Looking at automated tests results, about 30% of update attempts fails due to InRelease is not valid yet error. Example with specific time:

whonix-gateway-17:err: E:Release file for tor+https://fasttrack.debian.net/debian/dists/bookworm-fasttrack/InRelease is not valid yet (invalid for another 2min 37s). Updates for this repository will not be applied.

I’m not sure which component specifically causes this (boot time randomization? sdwdate? something else?) but can you make it to not set time into the past, but only (slightly) in the future? or maybe disable time randomization in templates at all (if that’s okay from privacy standpoint)?

Thank you for bringing this up as I wasn’t aware of this magnitude.

Quite probable.

Problem is sdwdate doesn’t (and probably shouldn’t) have connectivity in Template.

boot time randomization is too inaccurate.

The ClockVM ticket might be the solution but also hard to implement. (https://phabricator.whonix.org/T387)

Not ideal.

Technically probably easy to implement. (+ or - sign is random. Easy to hardcode to “+” in Qubes Templates.)

As for privacy implications, not sure yet. Might not be ideal but perhaps a compromise we have to make.


A wholly different approach would be commenting out fasttrack for Qubes by default as it isn’t very important for non-users of VirtualBox. (Fasttrack is primarily added to acquire VirtualBox Guest Additions which aren’t needed/useful/installed in Qubes(-Whonix).)

But the technical implementation as well as the delta between the various platforms would be messy.

This doesn’t open for me:

Unhandled Exception ("RuntimeException")
strlen(): Passing null to parameter #1 ($string) of type string is deprecated

Wouldn’t then the issue still happen on VirtualBox? I’m not using this setup, so I don’t know. But if time randomization is used in VirtualBox too, I’d expect updates failing there similarly often.

BTW, we’ve seen the opposite issue too - signature made in the future was rejected by qubes builder. Not applicable to templates really, but may be worth considering when changing how time randomization works. Maybe a better solution would be to reduce the range to +/- few seconds (including random miliseconds offset) instead of few minutes?

1 Like

Some mysterious phabricator issue. It’s here:

No, because in VirtualBox sdwdate will fix the clock to reasonable correctness.
Exactly this is failing in Qubes Template, because these are non-networked and because https://web.archive.org/web/20230603005622/https://phabricator.whonix.org/T387 isn’t implemented. (Asking sys-whonix for the time.)

Added here for consideration just now:

I couldn’t decide what amount of time should be randomized at boot in Qubes Template. Now i arbitarily settled for up to 1 second +/- nanoseconds. In other words:

+/- 1 or 0 seconds
+/- nanoseconds

Ok, I’m disabling the workaround now (setting time always +5min in Whonix) and will observe logs after collecting some data.

1 Like