Derivative Maker Automated CI Builder

@Patrick PR up for ya

1 Like

Thank you! Merged.

Now that we have CI for commits and tags, I am going to work on having WATS run every time a tag is pushed…then I will work on making WATS more robust

Planning to try to get some of that automated testing fired up this weekend. I might make a new repo based off the fork you have in your github @Patrick…will research more and hopefully get ya some automated tag testing built in in the not too distant future

1 Like

@Patrick has the CI been useful for you yet? feedback/criticism is welcome :slight_smile:

1 Like

Extremely useful! Very good contribution. Now I am confident I didn’t break the build since I would get an e-mail if something is broken (something as simple as a forgotten git push). Also a good feeling to have a different, more picky build environment to know other stuff such as the mount related stuff is solid.

Could you please update from https://github.com/derivative-maker/derivative-maker to https://github.com/Whonix/derivative-maker?

I think (could def be wrong) this should pretty much work out of the box. I dynamically set all the URLs with built in github environment variables. Maybe allow github actions on the new repo, and push a dummy tag, then I can look at the logs if something breaks?? For now it all seems okay, at least when I run grep -irn 'derivative' ./automated_builder I don’t see anything that should be problematic or break

1 Like

working on getting a fork of WATS running today :slight_smile:

1 Like

Woot woot! It is running !!! :raised_hands:

Now I have some cleaning up to do, then will get it to run via CI next, when tags are built :slight_smile:

1 Like

I am wondering if we should keep the per commit building. At the moment I am pushing lots of commits for live-build (ISO) development. Those won’t influence VM builds. We could save some expenses for the CI server by not wasting builds. Build from tags should be enough for now.

1 Like

I think it is good to have CI run any time a commit is pushed. CI server costs are very small even with lots of commits currently being pushed. I am spending around $5 USD a month on CI, which is totally fine with me.

Benefits of CI running any time a commit is pushed is fast feedback if something breaks the build, rather than compounding issues. For $5 I am happy to donate every month so you can have the information readily available.

I think currently since CI isn’t being used for GUI testing, I can cut that number in half to about $2.50 a month by resizing droplet. I plan to overhaul all this work eventually. It is on the horizon

All that to say, I do not think builds are “wasted” when they pass :slight_smile:

1 Like

@Patrick CI fix

1 Like

With bumped changelog version · Whonix/derivative-maker@8756ec0 · GitHub we might have hit a github actions timeout issue after 6 hours? prepare_release log simply stops without error.

Yeah. Really long. I can invent ways to make that more efficient.

I am pretty sure thats a github problem. Every now and then, they have an issue where their runners wont pick up a job and then timeout

Major con of a managed service is when they drop the ball

Rerunning now. In the future if things go that long without the builder running, faster to cancel and rerun

Oh nvm…something else is clearly going wrong. I will investigate eventually. Why builds taking 5+ hours is beyond me.

This should be possible to mitigate. I will research

1 Like

For each build for different flavors (whonix-gateway, …) the packages are rebuilt. Also the base raw image is rebuilt every time. That is inefficient. So I need to work on that. Build packages + base image only once. Then keep re-using it for all flavors. Doable.

OS image creation is really… Not easy at all… Tons of OS building tools have been created to cope with this complexity.

If there is anything we can run concurrently on separate VPS runners, let me know. That’s another optimization we can utilize to speed things up.

Only two flavors building right now tho right? Whonix Gateway XFCE and Whonix Workstation XFCE right? I think building from tag does not build CLI.

If we are building CLI, we can do CLI and XFCE builds on seperate servers

1 Like

build targets:

  • Kicksecure ISO
    • Xfce
    • CLI
  • Kicksecure for VirtualBox
    • Xfce
    • CLI
  • Whonix ISO (future)
    • Xfce
    • CLI
  • Whonix for VirtualBox
    • Whonix-Gatway
      • Xfce
      • CLI
    • Whonix-Workstation
      • Xfce
      • CLI
  • Whonix Xfce for VirtualBox Windows Installer (building requires negligible time)

That’s ~ 10 build targets so far. (Currently - 2 because there is no Whonix ISO yet at time of writing.)

But building ISO CLI maybe makes limited sense anyhow. →

Maybe I could drop the CLI builds from the CI. That would half the number of build targets to 4 (or 5). It’s unlikely that that Xfce build would succeed but the CLI build would fail.

Then there is also ARM64 / UTM builds supported by derivative-maker. I’ve configured the CI for now to create ARM64 builds only and no AMD64 builds. This is because ARM64 builds are more likely to fail than AMD64 builds. There are few cases where the ARM64 build would succeed but the AMD64 build to fail. (Except for Tor Browser download.)

That makes sense now but will fail in the future when the CI would run automated tests of the VirtualBox VMs because at time of writing VirtualBox is available for AMD64 only.

If we created both, AMD64 + ARM64 builds, that would double the number from 4 (or 5) to 8 (or 10).

We could declare ARM64 not our problem. If someone wants to maintain an ARM64 or other architecture support, they should please run their own instance of the CI.

Where reasonable without increasing code complexity too much, I’ve already optimized the build process in so far that build steps that don’t need to be needlessly run over and over again are run once only. (Such as dependency installation, package creation and base raw image creation.) It could be optimized furhter in theory (have base images share more installed packages) but that would make derivative-maker more complex. Since there are few people already who can handle and/or are interested in build scripts, that is probably a bad idea.

I configured the CI to use developer-meta-files/usr/bin/dm-virtualbox-build-official at master · Kicksecure/developer-meta-files · GitHub. That is the script which I am using to create official builds. This I have done to avoid having to maintain build commands / targets separately for official builds versus CI builds.

That approach however isn’t compatible yet with:

  • A) running build targets concurrently on the same VPS runner, let alone
  • B) running build targets concurrently on separate VPS runners.

Next thing I could work on is A). In theory it should already work. All paths used during the build are already unique to the flavor and not shared. I just have to test it. But I have no idea how much time it would safe.

1 Like