whonix-ws-16 Template fails to update due to timing issue

Qubes-Whonix
12

Also suspend/resume time fix is disabled in Qubes-Whonix templates.
(sdwdate /usr/libexec/sdwdate/suspend-post)

Are you using suspend/resume? That might result in time in Qubes-Whonix Templates to become stale.

As a workaround, does it help to shutdown and restart Qubes-Whonix Templates?

Maybe Qubes-Whonix Templates should use qubes.GetRandomizedTime?

Reminds me of ⚓ T387 Qubes-Whonix-Gateway as ClockVM. I still didn’t get around implementing it since it’s kinda complex, not pretty and daunting to implement. More realistically might be using sdwdate inside the ClockVM (sys-net) or using a Kicksecure based sys-net which comes with sdwdate pre-installed.

13

All the above checks are just after startup, no suspend nor long uptime is involved.

1 Like
14

Looks good enough.
Please re-run in case this issue happens again. Or could you please add it to the Qubes Q/A scripts? Seems quite useful anyhow to have this information handy. Checking/comparing dom0/VM time before proceeding with any updates.

15

Yet another cause could be Debian indeed setting a wrong valid-from field.

16

When this issue is happening, could you please check this link/file?
https://fasttrack.debian.net/debian/dists/bullseye-fasttrack/Release

At time of writing the interesting fields are:

Date: Sat, 20 Nov 2021 15:30:10 UTC
Valid-Until: Sat, 27 Nov 2021 15:30:10 UTC

VM:

date --utc

Sat 20 Nov 2021 04:00:03 PM UTC

dom0:

date --utc

Sat Nov 20 16:00:29 UTC 2021

(Few seconds delay due to thinking, typing.)

17

The different time zones might be an issue. dom0 / Templates are “normal” but Qubes-Whonix templates are in UTC.

For Debian 11 / bullseye in sdwdate a change was required (and introduced since the first Debian 11 based Whonix version 16) to set in python:

os.environ["LC_TIME"] = "C"
os.environ["TZ"] = "UTC"
time.tzset()

This python change may need to be added to any Qubes source code. For example the python based Qubes dom0 file /etc/qubes-rpc/qubes.GetDate might need it.

Furthermore, to improve the robustness (and perhaps even fix this bug) any invocation of date should be prefixed with --utc, i.e. date --utc for any Qubes shell/bash scripts.

Internally, programmatically Qubes should always handle time in UTC. The time shown to the user when manually running date or looking at the systray can remain as is in user local timezone, no problme.

18

The Whonix template is ~half a minute in the past. If it’s close enough that the 5 minutes makes a difference (date -s +5min is enough to fix), then I can very well believe that 30s in the past may be problematic at times too. Something clearly is making Whonix’s clock be in the past, and I’d say we should avoid it at all regardless if that’s 30s or 5min.

1 Like
19

added to wiki just now…

Qubes Time Synchronization Features:

Xen DomU’s get their initial time from Xen (Qubes dom0) at VM start.

  • qvm-sync-clock.service
  • qvm-sync-clock.timer

qvm-sync-clock gets time from Qubes ClockVM.

It is used because otherwise Xen DomU’s would start to clock drift more and more. This this answer by marmarek [archive].

qvm-sync-clock is unwanted in sys-whonix and anon-whonix because sdwdate runs there.

qvm-sync-clock is disabled in Qubes-Whonix ™ Templates until version Qubes-Whonix ™ 16.0.3.7. To be re-considered for later versions. Qubes-Whonix ™ get their time from dom0 at VM startup, which is then randomized using Boot Clock Randomization.

Future: qvm-sync-clock should be equally safe to run inside Qubes-Whonix ™ Templates, if passed though clock-random-manual-cli.

Now working on making timesync in Qubes-Whonix Template similar to Non-Qubes Templates.

https://github.com/Whonix/sdwdate/commit/6215a9ea996e9db970059c3b4ad58d17016b7483

20

Time to revise:

New plan:
As for Qubes-Whonix Templates, make it similar to /etc/qubes-rpc/qubes.SetDateTime.

21

I thought suspend/resume as well as long running Templates should be improved by modifying qvm-sync-clock for Qubes-Whonix. Started working on it:

Not in use yet. And was probably in vain. And probably low priority.

Because…

Then if this “small” difference is causing some much issues… The reason for that is probably:

If someone would like to try if that is the case, try the workaround of disabling boot clock randomization.

Related boot clock randomization development discussion:
Boot Clock Randomization - bootclockrandomization

22

As for usefulness of Boot Clock Randomization in Qubes-Whonix Template, see:

I am still not sure Boot Clock Randomization is the cause since then this issue should be equally happening to Non-Qubes-Whonix users and a lot more users. But to find out…

23

Now I am a bit more sure this is caused by Boot Clock Randomization. It’s not happening in Non-Qubes-Whonix since that is using sdwdate which has much higher accuracy than Boot Clock Randomization. Qubes-Whonix Templates however use only Boot Clock Randomization and do not use sdwdate at time of writing.

Still not sure why this isn’t happening more often to more users and wasn’t reported earlier.

In summary:

  • Non-Qubes-Whonix: Boot Clock Randomization + sdwdate
  • Qubes App Qubes: Boot Clock Randomization + sdwdate
  • Qubes Templates: Boot Clock Randomization only

Accuracy:

  • Boot Clock Randomization: +/ 180 seconds
  • sdwdate: quite good, often +/- 1 second

Should sdwdate be run inside Qubes Templates?

It would probably be easy to make sdwdate connect through Qubes UpdatesProxy - similar to how already APT and Tor Browser Downloader by Whonix ™ are using networking in Qubes-Whonix Templates.

A more secure solution might be something like ⚓ T387 Qubes-Whonix-Gateway as ClockVM, i.e. Qubes-Whonix Templates receiving a timestamp with sdwdate accuracy from an App Qube, but also quite harder to implement.

24

@Patrick & @marmarek : Thanks a lot for all your work on tracking down the root cause of the reported issue.

It is not clear to me, if at the moment you need additional information from my system.

FYI: Qubes Updater has (again) successfully updated several fedora templates - but - failed to update ‘whonix-ws-16’.

25

@Patrick : Is there an update available on when & how this issue might be fixed?

On Qubes R4.0 I’m still experiencing it with every update of a ‘whonix-*-16’ template.

26

No, if there is, this forum thread will be updated.

See my previous post.

That’s surprising because it never happens to me.

27

Hello Patrick,

That’s surprising because it never happens to me.

It just occurred again w/ “whonix-gw-16” template. - Let me know, if any of my own system setup details are important for you …

28

Hi, it just happened to me. Is there a fix for this issue, please? Thank you.

29

Hello jrnn,

jnrn https://forums.whonix.org/u/jnrn
July 25

Hi, it just happened to me. Is there a fix for this issue, please? Thank
you.

Not that I’m aware of. - In the meantime I’ve upgraded my system to Qubes
OS 4.1, but the problem still persists :frowning:

With kind regards,

Viktor

30

Documentation written just now:

InRelease is not valid yet error

(Whonix is based on Kicksecure.)

31

I’m resurrecting this thread, because the issue is still happens in Whonix 17. And is rather common with Debian fasttrack repository. Looking at automated tests results, about 30% of update attempts fails due to InRelease is not valid yet error. Example with specific time:

whonix-gateway-17:err: E:Release file for tor+https://fasttrack.debian.net/debian/dists/bookworm-fasttrack/InRelease is not valid yet (invalid for another 2min 37s). Updates for this repository will not be applied.

I’m not sure which component specifically causes this (boot time randomization? sdwdate? something else?) but can you make it to not set time into the past, but only (slightly) in the future? or maybe disable time randomization in templates at all (if that’s okay from privacy standpoint)?