“Diet” builds for commits are taking about 70 minutes…Full builds with tags are taking about 115 minutes.
Maybe once I get the GUI and WATs running, we can add some cores to the box and parralellize the builds so they dont take so long
2 cores build the workstation, 2 cores build the gateway or something like that
Thank you! Merged.
Now that we have CI for commits and tags, I am going to work on having WATS run every time a tag is pushed…then I will work on making WATS more robust
Planning to try to get some of that automated testing fired up this weekend. I might make a new repo based off the fork you have in your github @Patrick…will research more and hopefully get ya some automated tag testing built in in the not too distant future
Extremely useful! Very good contribution. Now I am confident I didn’t break the build since I would get an e-mail if something is broken (something as simple as a forgotten git push). Also a good feeling to have a different, more picky build environment to know other stuff such as the mount related stuff is solid.
Could you please update from https://github.com/derivative-maker/derivative-maker to https://github.com/Whonix/derivative-maker?
I think (could def be wrong) this should pretty much work out of the box. I dynamically set all the URLs with built in github environment variables. Maybe allow github actions on the new repo, and push a dummy tag, then I can look at the logs if something breaks?? For now it all seems okay, at least when I run grep -irn 'derivative' ./automated_builder I don’t see anything that should be problematic or break
working on getting a fork of WATS running today 
Woot woot! It is running !!! 
Now I have some cleaning up to do, then will get it to run via CI next, when tags are built 
I am wondering if we should keep the per commit building. At the moment I am pushing lots of commits for live-build (ISO) development. Those won’t influence VM builds. We could save some expenses for the CI server by not wasting builds. Build from tags should be enough for now.
I think it is good to have CI run any time a commit is pushed. CI server costs are very small even with lots of commits currently being pushed. I am spending around $5 USD a month on CI, which is totally fine with me.
Benefits of CI running any time a commit is pushed is fast feedback if something breaks the build, rather than compounding issues. For $5 I am happy to donate every month so you can have the information readily available.
I think currently since CI isn’t being used for GUI testing, I can cut that number in half to about $2.50 a month by resizing droplet. I plan to overhaul all this work eventually. It is on the horizon
All that to say, I do not think builds are “wasted” when they pass
With bumped changelog version · Whonix/derivative-maker@8756ec0 · GitHub we might have hit a github actions timeout issue after 6 hours? prepare_release log simply stops without error.
Yeah. Really long. I can invent ways to make that more efficient.
I am pretty sure thats a github problem. Every now and then, they have an issue where their runners wont pick up a job and then timeout
Major con of a managed service is when they drop the ball
Rerunning now. In the future if things go that long without the builder running, faster to cancel and rerun
Oh nvm…something else is clearly going wrong. I will investigate eventually. Why builds taking 5+ hours is beyond me.
This should be possible to mitigate. I will research
For each build for different flavors (whonix-gateway, …) the packages are rebuilt. Also the base raw image is rebuilt every time. That is inefficient. So I need to work on that. Build packages + base image only once. Then keep re-using it for all flavors. Doable.
OS image creation is really… Not easy at all… Tons of OS building tools have been created to cope with this complexity.
If there is anything we can run concurrently on separate VPS runners, let me know. That’s another optimization we can utilize to speed things up.
Only two flavors building right now tho right? Whonix Gateway XFCE and Whonix Workstation XFCE right? I think building from tag does not build CLI.
If we are building CLI, we can do CLI and XFCE builds on seperate servers