Docker Container that builds Whonix Images

Patrick · June 17, 2025, 5:51pm

export has to be in a separate line. Cannot be only 1 line. Because export always exits 0. Hence, breaking error handling. So please first set tag, then export in separate line.

How would the user be sure that script is non-malicious? Same bootstrap issue. Unicode check + script review required. Outside of skillset of many users.

e-coin · June 17, 2025, 7:08pm

export has to be in a separate line. Cannot be only 1 line. Because export always exits 0. Hence, breaking error handling. So please first set tag, then export in separate line.

Done, thanks.

How would the user be sure that script is non-malicious? Same bootstrap issue. Unicode check + script review required. Outside of skillset of many users.

Hm ok. So the user must manually copy/paste from wiki and execute that initial verification code.

That seems ugly, but kind of impossible otherwise.

How about an interactive shell that runs on the first container start (writes a file in ~/binary-mnt with check value)…

lol same issue.

Yeah ok fine I guess.

prepare-build-machine
(including key import)

git_verify() {
  git verify-commit -- "${TAG}^{commit}"
  [ "$TAG" = "master" ] || git verify-tag -- "${TAG}"
}

Do you want this to only execute on a docker run?

Patrick · June 17, 2025, 7:50pm

Useful as a sanity test (in case tag signing fails let’s say due to gpg configuration issue). So good to keep.

Patrick · June 17, 2025, 11:32pm

Maybe best to avoid dockerhub remote docker upload? Doesn’t seem to provide much? Lower maintenance effort. More up-to-date. Maybe more secure as we avoid a huge binary image on a third-party remote server?

Would it make sense if the docker start script would run the docker build script when needed for better usability? Any docker convention for that?

Does the git fetcher + verifier be part of docker? Could be a standalone script that gets called by a docker script.

Scripts we might integrate with:

e-coin · June 18, 2025, 6:56am

Yeah, I didn’t consider the security concerns.
Dockerhub only hosts docker images (I believe). The image we’d push to hub.docker.com is the docker image that is built by derivative-maker-docker-image. So ~150MB, probably less with compression.

Building is already simplified with derivative-maker-docker-image.
Offering docker pull would be an even simpler way for users to get the docker image without building it, but I agree that it might not be appropriate here. I’ll scrap that from the readme.

Yes, that makes sense. This would pretty much be the equivalent of docker pull, because 0 user interaction is required to create the docker image. Less even.

I’ll add that.

Yes, much needed clean up of that part. I’ll take a look at those scripts.

What I’ve done so far is to create a volume that mounts the key location

--volume "${KEY_VOLUME}:/home/${DOCKER_USER}/.gnupg"

This ensures that the key only needs to be imported once, even on container restarts. [ -d "~/.gnupg" ] and f ~/.gnupg/pubring.kbx can now be conditioned too.

So in combination with --list-keys and pointing to a separate script that does fetch, checkout and verify is probably best.

Will do that today or tomorrow.

Patrick · June 18, 2025, 7:50am

git verify-commit only checks that a signature was created with a known key. It does not check which key. How to fix using a shell command? Ideally git verify-commit --fingerprint xxxxxx. But that does not exist at time of writing as far as I know. Without bash scripting (more error prone, dangerous). Here’s how:

git verify-commit HEAD
signed_by_fingerprint="$(git show --no-patch --pretty=format:%GF HEAD)"

(Replace HEAD with the actual commit or tag.)

e-coin · June 18, 2025, 2:37pm

Yes, works.

git verify-commit -- "master^{commit}"
gpg: Signature made Sat Jun 14 13:22:04 2025 UTC
gpg:                using RSA key 6E979B28A6F37C43BE30AFA1CB8D50BB77BB3C48
gpg:                issuer "adrelanos@whonix.org"
gpg: Good signature from "Patrick Schleizer <adrelanos@kicksecure.com>" [unknown]
gpg:                 aka "Patrick Schleizer <adrelanos@riseup.net>" [unknown]
gpg:                 aka "Patrick Schleizer <adrelanos@whonix.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 916B 8D99 C38E AF5E 8ADC  7A2A 8D66 066A 2EEA CCDA
     Subkey fingerprint: 6E97 9B28 A6F3 7C43 BE30  AFA1 CB8D 50BB 77BB 3C48

git show --no-patch --pretty=format:%GF "master^{commit}"
6E979B28A6F37C43BE30AFA1CB8D50BB77BB3C48

I’ll add the sub_key_fingerprint to buildconfig.d/30_signing_key.conf
And then just do a check against signed_by_fingerprint.

derivative_signing_key_sub_fingerprint="6E979B28A6F37C43BE30AFA1CB8D50BB77BB3C48"

Ok added to the list. I’ll have to check out those scripts later so that I can finish derivative-maker-docker-start

e-coin · June 19, 2025, 8:07am

OK, I checked it out.

I’ll put /usr/bin/dm-git-tag-checkout-latest in help-steps, modify it slightly and move the entire git stage from derivative-maker-docker-start to prepare-build-machine by calling this script.

derivative-maker-docker-start will only export $TAG, $GIT and other similar env stuff. Strictly to be passed on.

derivative-maker/help-steps/git-gpg-verify can be used, but this is not wanted.

   export GNUPGHOME="$binary_build_folder_dist/temp_gpg"

Whatever you prefer.
I can change this and call it or use the current code in prepare-build-machine

   if ! gpg --quiet --list-keys -- "${derivative_signing_key_fingerprint}" &>/dev/null; then
    true "INFO: Importing signing key ${derivative_signing_key##*/}"
    gpg --keyid-format long --import --import-options show-only --with-fingerprint -- "${derivative_signing_key}"
    gpg --import -- "${derivative_signing_key}"
    gpg --check-sigs -- "${derivative_signing_key_fingerprint}"
   fi

Edit: there is still a bunch of stuff to do, maybe cancel the pull for now.

e-coin · June 19, 2025, 10:41pm

Tested and working. Just fine tuning now.

e-coin · June 23, 2025, 7:55am

I had a simpler idea regarding key handling.

Why not just do curl -O https://www.whonix.org/keys/derivative.asc "${KEY_VOLUME}/derviatvie.asc" in derivative-maker-docker-run?

That provides the key through --volume mount, making it available before submodule sync. Then you just do the import in the container and verify normally.
buildconfig.d/30_signing_key.conf will then simply say derivative_signing_key="~/.gnugp/derivative.asc"

I wrote something else where the user has to manually download and gpg import the key to "${KEY_VOLUME}" if it’s empty, but this seems better.

Patrick · June 23, 2025, 11:10am

Another curl call. Versus torification. Versus onion. Versus proxy settings.

Signing key copy in derivative-maker folder might be simpler. I’ll do that.

e-coin · June 23, 2025, 6:27pm

Ok done.

This is derivative-maker-docker-start now.

SOURCE_DIR="${HOME}/derivative-maker"
BINARY_DIR="${HOME}/derivative-binary"
LOG_DIR="${BINARY_DIR}/logs"
BUILD_LOG="${LOG_DIR}/build.log"

mkdir --parents -- "${BINARY_DIR}" "${LOG_DIR}"

chown --recursive -- "${USER}:${USER}" "${LOG_DIR}"

cd -- "${SOURCE_DIR}"

./help-steps/git-gpg-verify

"$@" 2>&1 | tee -a -- "${BUILD_LOG}"

All that needs to be changed is the location of the key in
buildconfig.d/30_signing_key.conf

derivative_signing_key=""

Please have a look when you have the time.

I made a bunch of back and forth commits, though. Maybe I should clear the history first, sync the fork and then only commit what’s necessary.

Patrick · June 26, 2025, 6:00pm

PR closed? Status?

e-coin · June 26, 2025, 9:45pm

Hey there. Re-forked, forgot to file it again. Current status is tested and working.

Patrick · June 27, 2025, 11:41am

Could you please review the latest docker related source code and integration with git? @arraybolt3

e-coin · June 27, 2025, 5:58pm

Small suggestion regarding this part.

import_key() {
  local derivative_signing_key_fingerprint_item
  for derivative_signing_key_fingerprint_item in "${derivative_signing_key_fingerprint_list[@]}"; do
    if ! gpg --quiet --list-keys -- "${derivative_signing_key_fingerprint_item}" &>/dev/null; then
      missing_key=true
    fi
  done

  if [ ! "${missing_key}" = "true" ]; then
    return 0
  fi

  for derivative_signing_public_key_item in "${derivative_signing_public_key_list[@]}"; do
    gpg --keyid-format long --import --import-options show-only --with-fingerprint -- "${derivative_signing_public_key_item}"
    gpg --import -- "${derivative_signing_public_key_item}"
    gpg --check-sigs -- "${derivative_signing_public_key_item}"
  fi

--check-sigs is passed a key, instead of a fingerprint (will cause error)

gpg --check-sigs -- "${derivative_signing_public_key_item}"

Maybe something like this for example, where fingerprints are associated with a key and only checked individually. (added another key with fingerprints to illustrate)

derivative_signing_public_key_list=("./keys/derivative.asc" "./keys/derivative2.asc")
derivative_signing_key_fingerprint_list=(
"916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA 
6E979B28A6F37C43BE30AFA1CB8D50BB77BB3C48"
"ADC7A2A8D66066A2EEACCDA916B8D99C38EAF5E8 
BE30AFA1CB8D50B6E979B28A6F37C43B77BB3C48 
1CB8D50B6E979B28A6F376F37C43B77BB3C4AGA4"
)


for key_index in "${!derivative_signing_public_key_list[@]}"; do
  
  for derivative_signing_key_fingerprint_item in ${derivative_signing_key_fingerprint_list["$key_index"]}; do
    
    if ! gpg --quiet --list-keys -- "${derivative_signing_key_fingerprint_item}" &>/dev/null; then 
      gpg --keyid-format long --import --import-options show-only --with-fingerprint -- "${derivative_signing_public_key_list["$key_index"]}"
      gpg --import -- "${derivative_signing_public_key_list["$key_index"]}"
      gpg --check-sigs -- "${derivative_signing_key_fingerprint_item}"  
    fi
  
  done

done

As for this part, [ -z "${GIT}" ] could incorporate all corresponding git commands in an if.

  ## TODO: review
  ## - security
  ## - avoid developer loss of progress
  return 0

I’m sure @arraybolt3 has got it covered, though.

arraybolt3 · June 28, 2025, 12:57am

e-coin:

derivative_signing_public_key_list=("./keys/derivative.asc" "./keys/derivative2.asc")
derivative_signing_key_fingerprint_list=(
"916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA 
6E979B28A6F37C43BE30AFA1CB8D50BB77BB3C48"
"ADC7A2A8D66066A2EEACCDA916B8D99C38EAF5E8 
BE30AFA1CB8D50B6E979B28A6F37C43B77BB3C48 
1CB8D50B6E979B28A6F376F37C43B77BB3C4AGA4"
)


for key_index in "${!derivative_signing_public_key_list[@]}"; do
  
  for derivative_signing_key_fingerprint_item in ${derivative_signing_key_fingerprint_list["$key_index"]}; do
    
    if ! gpg --quiet --list-keys -- "${derivative_signing_key_fingerprint_item}" &>/dev/null; then 
      gpg --keyid-format long --import --import-options show-only --with-fingerprint -- "${derivative_signing_public_key_list["$key_index"]}"
      gpg --import -- "${derivative_signing_public_key_list["$key_index"]}"
      gpg --check-sigs -- "${derivative_signing_key_fingerprint_item}"  
    fi
  
  done

done

I’m a little confused about the intention here - is the goal to check all of the fingerprints in the list, or only one of them? If the goal is to check only one of them, you can make this a bit more efficient by adding a break as the last statement in the if block. If you want to check all of them, you need to move the gpg --check-sigs command out of the if block.

I’m looking at this right now - I was thinking of doing something a bit like this, but I fear a developer one day using --git when they didn’t mean to and potentially losing uncommitted changes. (This could easily happen in my setup, where I use an additional wrapper script to call derivative-maker with the right arguments so I don’t have to remember them every time I call the script. I’ve more than once ran a build with the wrong arguments as a result.) So I’m thinking of making it so that if the user is doing anything that will change the git state (pulling, fetching, checking out, etc.), it checks for uncommitted changes using [ -z "$(git status --porcelain=v1 2>/dev/null)" ] first, and bails out if they exist. (Fancy git command shamelessly stolen from https://stackoverflow.com/a/3879077/19474638 :P)

Edit: Actually, help-steps/git_sanity_test can be used to do that job.

e-coin · June 28, 2025, 6:59am

Yes break went missing there, just a quick example.

You don’t need to check all the fingerprints, if gpg --quiet --list-keys returns 1 on a single fingerprint, then you just import that key, and move on to the next fingerprint set & key.

I’ll test this and add it to git-gpg-verify if it’s fine. You have write permissions on my fork anyway.

Seems like that was @Patrick’s intention.
Something like that is used in dm-git-tag-checkout-latest

if [ ! "$(git status --porcelain)" = "" ]; then
   error "Command git status --porcelain failed at the beginning!"
fi

if [ ! "$(git status --porcelain)" = "" ]; then
   error "Command git status --porcelain failed at the end!"
fi

Edit: help-steps/git-sanity-test checks status with regards to dist_build_ignore_uncommitted" = "true" and --allow-uncommitted

git_sanity_test_check_for_uncommitted_changes() {
   if [ -n "$(git status --porcelain)" ]; then
      if [ "$dist_build_ignore_uncommitted" = "true" ]; then
         true "${bold}${cyan}$BASH_SOURCE INFO: Git reports uncommitted changes! But you requested to ignore uncommitted changes, continuing... ${reset}"
         true "${cyan}$BASH_SOURCE INFO: Running \"git status\" for debugging. ${reset}"
         git status
         true "${cyan}$BASH_SOURCE INFO: Running git \"clean -d --force --force --dry-run\" for debugging. ${reset}"
         git clean -d --force --force --dry-run
         true
      else
         error "Uncommitted changes! See above!"

So if --allow-uncommitted true then all that’s needed in git-gpg-verify is if [ -z "$(git status --porcelain)" ] which conditions the execution of these state altering git commands.

I can do that, or if you’d like to that’s fine too.

e-coin · June 28, 2025, 11:44pm

@arraybolt3 here’s an example git-gpg-verify

If uncommitted, only checking out when a different tag is chosen from the current one and stashing first. Something like this could be a reasonable compromise compared to just exiting completely.

  if [ -z "$(git status --porcelain 2>/dev/null)" ]; then
    git pull
    git checkout "${TAG}"
  else
    true "INFO: Uncommitted changes found. Skipping upstream merge."
    if [ "${TAG}" != "$(git describe)" ]; then
      true "INFO: Saving current changes before switch to ${TAG}."
      git stash
      git checkout "${TAG}"
    fi
  fi

Restricting to only fetching tags

  git fetch https://github.com/derivative-maker/derivative-maker --depth=1 -- 'refs/tags/*:refs/tags/*'

arraybolt3 · June 29, 2025, 3:42pm

I don’t like the “secret” git stash. People would have to read this specific part of the source code to know that their changes were being saved in this way, and I don’t think that most people will do that (there are many parts of derivative-maker I still haven’t read). Aborting the build entirely is annoying, but that’s actually a good thing since it forces the user to pay attention to what’s happening. (There are also other things that can mangle uncommitted changes during a build, so in most instances it’s a good idea for the developer to always commit everything before a build. I frequently make “checkpoint” commits that I later throw away in order to prevent lost changes.)

I’ll take a look at adding the fingerprint check code, I have some changes in my local fork of derivative-maker and can integrate those while doing other polish and whatnot.