What is missing and hard to automate is correlating individual test failures with a specific change (for example which package in current-testing caused problem). We don’t have resources to run test with each package separately and it would also be undesirable, as in the final system all those updates are together.
For Whonix templates, the second point includes also proposed-updates repository.
I can easily add another Whonix repository to the test system. And also some more commands to run and check results. But if some more elaborate tests would be desired, someone would need to write them. I can help with technicalities.
…immediately clear to me that I would like to see the buster-developers repository added there too. If resources allow also all the other repositories, buster-testers and buster (stable).
If I am somehow notified that a change in the developers repository breaks something - the challenge here is that there are many different version combinations (also TemplateBasedVM vs standalone vs DispVM etc.) - then of course I will fix that first. Upload a fixed package to the developers repository. And never migrate any broken package (combination) to any other repository.
Weekly is good. Manual would be super useful too. If it’s either/or,
then I need manually more. If both are possible, I take that.
Any easy (little time effort on your side) way to trigger these from
command line (such as curl POST)? Then I could automate that with my
package build scripts.
how would you like be notified about results
Only one strong preference: I cannot follow (github watch) all of Issues · QubesOS/updates-status · GitHub since it generates way
too many e-mails. Or is there a way to github e-mail subscribe to 1
specific tag only?
A public place preferred over private - to not add needlessly
centralization into me.
Some dedicated github repository? Great!
I could create a new dedicated mailing list too if that is required.
(Would not want to mix with whonix-devel.)
E-mail me privately: least preferred way since not public.
Yes, this is basically how it works. There is openqa-client tool for that (looks like it’s packaged in Fedora, but not in Debian), it pulls in ~250 perl packages…
Generally what is does it making a single POST request with fairly simple content, but authentication headers would need to be prepared manually (hmac of API path and timestamp using API secret). If using openqa-client directly would be an issue, I can probably write some ~10 lines of python doing the same.
Credentials request for openQA access sent in private email.
That can be the simplest option for now. Do you want to create one under Whonix org?
BUILD - arbitrary build number, but needs to be unique and monotonic; it is listed on the main page, so should be meaningful. Doesn’t need to be fully numeric - for example some meaningful word as a suffix is fine
VERSION - qubes version to test on - currently supported 4.0 and 4.1
WHONIX_REPO - which repository to install updates from (appended to buster- or whatever base distro is there)
My own recommended command is against this requirement… All 4.0 tests are now considered older than all 4.1, in practice meaning only 4.1 test runs are visible on the main page…
Can you change it in subsequent builds to put date first and -4.0 or -4.1 as suffix?