There is a modified standalone url_to_unixtime at https://github.com/troubadoour/url_to_unixtime-asynchronous.
The idea is to get the time from the pool’s remotes asynchronously, using some of the Python gevent magic. Besides an improved transparency of timesync, the rationale is that if we want to implement https://phabricator.whonix.org/T300 and block internet access after boot as long as the time synchronization has not finished, the shorter it takes, the better for the user.
In the script, for the sake of demonstration, the three pools are concatenated into a single one. After running it a sufficient number of times, you should have seen many combinations of errors (connect error, timeout, header parsing…). In all the cases, the duration of the acquisition is reduced drastically.
I do not know if this approach looks sound, but it would require some major changes in sdwdate. The pools being moved in url_to_unixtime, all the logic from sdwdate_pick and above (sdwdate_loop, sdwdate_get_time_from_remote and even sdwdate_build_median) would be done in the new module(s). Perhaps a starting point to port sdwdate to Python.