Currently, I have a repo working for automated whonix integration testing. This is a continuation of the WATS senior capstone project, and will be integrated in to the new CI stuff I built.
I am creating a new thread to log my efforts and progress, so as to keep things readable and up to date.
@Patrick you had a list of desired test cases somewhere right? Can’t seem to find it at the moment.
You mean as in adding additional tests to wats?
This is a good start. I will try to get all (or most) of that running in CI and report back when it is running 
@Patrick can you look at this failed build? I am working on getting everything set up for GUI testing, and am having failing CI when I push a tag. Strange it seems it passed a couple days ago for you.
It says something about a bad SSL version when trying to grab a package 
https://github.com/Mycobee/derivative-maker/actions/runs/3342157416
Synchronizing submodule url for 'packages/whonix/onion-grater'
Synchronizing submodule url for 'packages/whonix/qubes-whonix'
Synchronizing submodule url for 'packages/whonix/uwt'
Synchronizing submodule url for 'packages/whonix/whonix-base-files'
Synchronizing submodule url for 'packages/whonix/whonix-firewall'
Synchronizing submodule url for 'packages/whonix/whonix-gw-network-conf'
Synchronizing submodule url for 'packages/whonix/whonix-welcome-page'
Synchronizing submodule url for 'packages/whonix/whonix-ws-network-conf'
+ git submodule update --init --recursive --jobs=200
+ true 'INFO: Updated git sub modules.'
+ sudo --non-interactive apt-get -o Acquire::http::Proxy=http://127.0.0.1:3142 -o Acquire::https::Proxy=http://127.0.0.1:3142 -o Acquire::tor::Proxy=http://127.0.0.1:3142 -o APT::Update::Error-Mode=any -o Acquire::Languages=none -o Acquire::IndexTargets::deb::Contents-deb::DefaultEnabled=false -o Apt::Install-Recommends=false -o Acquire::Retries=3 -o Dpkg::Options::=--force-confnew -o Dir::Etc::sourcelist=/home/ansible/derivative-maker/build_sources/debian_stable_current_clearnet.list -o Dir::Etc::sourceparts=- update
Get:1 http://HTTPS///deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:2 http://HTTPS///deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:3 http://HTTPS///deb.debian.org/debian bullseye-backports InRelease [49.0 kB]
Ign:4 http://HTTPS///fasttrack.debian.net/debian bullseye-fasttrack InRelease
Get:5 http://HTTPS///deb.debian.org/debian bullseye InRelease [116 kB]
Ign:4 http://HTTPS///fasttrack.debian.net/debian bullseye-fasttrack InRelease
Ign:4 http://HTTPS///fasttrack.debian.net/debian bullseye-fasttrack InRelease
Get:6 http://HTTPS///deb.debian.org/debian-security bullseye-security/non-free Sources [632 B]
Get:7 http://HTTPS///deb.debian.org/debian-security bullseye-security/main Sources [167 kB]
Get:8 http://HTTPS///deb.debian.org/debian-security bullseye-security/non-free amd64 Packages [528 B]
Get:9 http://HTTPS///deb.debian.org/debian-security bullseye-security/main amd64 Packages [193 kB]
Err:4 http://HTTPS///fasttrack.debian.net/debian bullseye-fasttrack InRelease
500 SSL error: wrong version number [IP: 127.0.0.1 3142]
Get:10 http://HTTPS///deb.debian.org/debian bullseye-updates/main Sources [4812 B]
Get:11 http://HTTPS///deb.debian.org/debian bullseye-updates/main amd64 Packages [14.6 kB]
Get:12 http://HTTPS///deb.debian.org/debian bullseye-backports/contrib Sources [2712 B]
Get:13 http://HTTPS///deb.debian.org/debian bullseye-backports/non-free Sources [4552 B]
Get:14 http://HTTPS///deb.debian.org/debian bullseye-backports/main Sources [345 kB]
Get:15 http://HTTPS///deb.debian.org/debian bullseye-backports/main amd64 Packages [356 kB]
Get:16 http://HTTPS///deb.debian.org/debian bullseye-backports/contrib amd64 Packages [4400 B]
Get:17 http://HTTPS///deb.debian.org/debian bullseye-backports/non-free amd64 Packages [14.3 kB]
Get:18 http://HTTPS///deb.debian.org/debian bullseye/non-free Sources [81.2 kB]
Get:19 http://HTTPS///deb.debian.org/debian bullseye/main Sources [8633 kB]
Get:20 http://HTTPS///deb.debian.org/debian bullseye/contrib Sources [43.2 kB]
Get:21 http://HTTPS///deb.debian.org/debian bullseye/contrib amd64 Packages [50.6 kB]
Get:22 http://HTTPS///deb.debian.org/debian bullseye/non-free amd64 Packages [97.7 kB]
Get:23 http://HTTPS///deb.debian.org/debian bullseye/main amd64 Packages [8184 kB]
Fetched 18.5 MB in 3s (6784 kB/s)
Reading package lists...
E: Failed to fetch http://HTTPS///fasttrack.debian.net/debian/dists/bullseye-fasttrack/InRelease 500 SSL error: wrong version number [IP: 127.0.0.1 3142]
E: Some index files failed to download. They have been ignored, or old ones used instead.
++ errorhandlergeneral ERR
++ last_failed_exit_code=100
++ last_failed_bash_command='$SUDO_TO_ROOT apt-get ${APTGETOPT[@]} -o Dir::Etc::sourcelist="$dist_build_sources_list_primary" -o Dir::Etc::sourceparts="-" update'
++ output_cmd_set
++ '[' -o xtrace ']'
++ output_cmd=true
++ true 'INFO: Middle of function errorhandlergeneral of ././build-steps.d/1120_prepare-build-machine.'
++ errorhandlerprocessshared ERR
++ last_script=././build-steps.d/1120_prepare-build-machine
++ trap_signal_type_previous=
++ '[' '' = '' ']'
++ trap_signal_type_previous=unset
++ trap_signal_type_last=ERR
++ dist_build_error_counter=1
+++ benchmarktimeend 1666921637
++++ date +%s
+++ benchmarktimeend=1666921642
+++ benchmark_took_seconds=5
++++ convertsecs 5
++++ local h m s
++++ (( h=5/3600 ))
++++ true
++++ (( m=(5%3600)/60 ))
++++ true
++++ (( s=5%60 ))
++++ printf '%02d:%02d:%02d\n' 0 0 5
+++ echo 00:00:05
++ benchmark_took_time=00:00:05
Not CI specific. Created
for it.
Awesome. Also just fyi your build failed today because I was testing some things. Apologies. Should probably figure out how to let multiple CI builds run in parallel, but not a priority at the moment
VPS has XFCE on it now with tightvnc configured for inspection. Working on building non-headless Whonix VMs in CI.
Next step is to run wats, which I have already done locally so it shouldnt be too bad. There is a PR that is open @Patrick , but it is not ready to merge yet (still iterating)
I will let you know once it is
Doing some testing today, might cause collisions of CI builds if you are doing the same @Patrick…just a heads up in case unexpected failures occur
(should probably figure out a way to fix this but how do you add more hours in a day?)
The easiest solutiion might be giving me access to CI builds on whonix repo, but not sure offhand how to set that up. It is set up to not do more than 1 build at a time per repo, but when mycobee repo and whonix repo are doing CI it collides 
Thanks for the head up! No worries.
Not sure what would be to do on my side.
Add me as a collaborator on Whonix/derivative-maker but set up rules where I can’t push to master, can open pull requests, and can’t merge my own pull requests
Then all CI builds can be commited directly to Whonix/derivative-maker but not the master branch.
Important to not give me access to master though, and set up rules that PR must be merged by an admin (you).
then all forks can have CI turned off in settings, and Whonix/derivative-maker ensures no more than 1 build at a time is ever running
this one is disco and ready to go @Patrick
It builds VMs now with the GUI setting and launches the VM
Next step I will have tag pushes automatically run WATS. This PR sets the stage for that to happen
Excellent! Thank you, merged.
This was done. Could you check please?
I have access. Will hopefully get WATS running after this weekend
Okay so WATS is almost running, a few changes left. I had to create a huge VPS in order for all the VMs to run with GUI (8gb of RAM).
So once I get WATS running, I will create new functionality to automatically provision servers.
Here is the design plan for 2 scenarios
--remote-derivative-packages
Doing this in a parrallel manner will speed things up, and hopefully get CI jobs for commits to under 30 minutes.
This will be a much slower process, probably almost 2 hours to fully test a VM tag with GUI. But this is shorter than manually testing it, and can happen in the background
A post was split to a new topic: GitHub Accelerator
Running in to problems with the GUI stuff.
I can successfully VNC in, open a terminal, and run WATS. But when I do it remotely via VBoxManage it isnt working because the shell (on the host) is not within the Workstation XFCE display.
Been a huge headache, took a LOT of hours. I am thinking my new strategy is going to be creating a .desktop file and transferring it to Workstation and set it to run WATS on boot. Then after X amount of seconds (I will average how long it takes to run wats and add maybe an extra minute for good luck), have the host grab the WATS logfile, parse, and throw an error if WATS does not pass.
It’s hacky and I am not proud of the strategy, but I am at a loss what to do otherwise. It has been quite a lift getting this running 