CI Roadmap
Moving forward with CI, here are some useful improvements to our CI
Preconfigured Runner VPS
We should utilize an existing img file in digitalocean instead of installing and configuring the box every time CI runs. We should split the ansible tasks out to have a build_machine_image role. The role should install all needed depedencies including virtualbox, and do all configuration unrelated to the actual code changes in derivative-maker
The role should conditionally run, ONLY when git changes are in the ./automated_builder directory. It should tag the new image in a sane way, and publish the image to digital ocean.
When a code change happens in any directory that is not ./automated_builder, the CI should load this image and run the steps to install the source code, and build the whonix images
Parallelize Builds
The workstation and the gateway do not need to run on the same machine. CI should spin up two VPS machines and run the builds in parallel. The logs should be updated to reflect these separate builds, and tagging of machines should be updated in digitalocean. Teardown should successfully destroy all these machines, and run regardless of any behavior.
We should assess the CPU and Memory usage of our images, and then decide whether or not we can scale down based on that. We should only use as much machine as we need to run the build efficiently, to save costs. CI is cheap as it is, but the cheaper we can make things, the better.
This might mean making different sized images for commit builds (headless), and tag builds (GUI + full depedencies)
Upon further research, both builds must be on same host. Investigate running these with gnu parallel on the same host, and determine how big of a box is needed to run as fast as possible without being over provisioned.
Documentation Overhaul
As CI becomes more leveraged, we need good documentation so that it can be more maintainable and understood. The wiki should be updated and we should ensure that we have some sort of visual representation of how the system works.
OpenQA Setup in CI
This one is going to be a big fucking task, but super important to the future of both Whonix and KS. We need a useful OpenQA setup. This probably will need to be split in to multiple subtasks.
Exploration of Further CI Usage
Take stock of other things we can automatically build with CI to ensure integrity
VirtualBox installer script? Etc. Some of this can likely be done with OpenQA, but there might be areas of our codebase we can utilize CI to make life easier for us all, and improve the development cycle.