Docker Container that builds Whonix Images

e-coin · May 16, 2025, 2:15pm

Updated (working) version:

Final fix will be to use an unprivileged container. (working on it right now)

Current issue: Mount fails during cowbuilder build-step

I: mounting /proc filesystem
mount: /var/cache/pbuilder/base.cow_amd64/proc: permission denied.
       dmesg(1) may have more information after failed mount system call.
W: Aborting with an error

Reason: Mounting filesystems from within an unprivileged container is basically impossible by design.

docker run --cap-add SYS_ADMIN is not a viable solution, as the mount error simply switches from permission denied to cannot mount read-only

mount: /var/cache/pbuilder/base.cow_amd64/dev/pts: cannot mount none read-only.
       dmesg(1) may have more information after failed mount system call.
E: setup_mounts failed: E: mount /dev/pts failed at /usr/bin/mmdebstrap line 192.
	main::error("mount /dev/pts failed") called at /usr/bin/mmdebstrap line 1311
	eval {...} called at /usr/bin/mmdebstrap line 1149

I believe I’ve figured out a way to get around this by simply mounting tmpfs /var/cache/pbuilder beforehand.

--tmpfs /var/cache/pbuilder/base.cow_amd64/proc:rw,dev,suid,exec,uid=1000,mode=1770

--mount type=tmpfs,dst=/var/cache/pbuilder,tmpfs-mode=1770

Will report back after more findings and checking with the docker team.

Patrick · May 16, 2025, 6:08pm

e-coin · May 16, 2025, 9:55pm

Hey man, just tested the change.

Looking at the code, $dist_build_unsafe_io and $DOCKER must be true in order to skip.

So I used --unsafe-io true which sets dist_build_unsafe_io="true"

I couldn’t find anything that assigns $DOCKER so I just manually exported it true.

help-steps/variables then sets these opts

if [ "$dist_build_unsafe_io" = "true" ]; then
   aptgetopt_add "Dpkg::Options::=--force-unsafe-io"
   aptgetopt_conf_add "Dpkg::Options:: \"--force-unsafe-io\";"

Which causes the following error:

Chrooting into /var/cache/pbuilder/base.cow_amd64/
env: unrecognized option '--force-unsafe-io'
Try 'env --help' for more information.
E: Sub-process env returned an error code (125)
E: setup failed: E: apt-get -o Dir::Bin::dpkg=env -o DPkg::Options::=--unset=TMPDIR -o DPkg::Options::=dpkg -o DPkg::Chroot-Directory=/var/cache/pbuilder/base.cow_amd64 --yes install -oDebug::pkgProblemResolver=true -oDebug::pkgDepCache::Marker=1 -oDebug::pkgDepCache::AutoInstall=1 -oAPT::Status-Fd=<$fd> -oDpkg::Use-Pty=false apt sudo devscripts debhelper strip-nondeterminism fakeroot apt-transport-tor eatmydata aptitude cowdancer fasttrack-archive-keyring ?narrow(?or(?archive(^bookworm$),?codename(^bookworm$)),?architecture(amd64),?and(?priority(required),?not(?essential))) failed at /usr/bin/mmdebstrap line 192.

I checked it and that is definitely a valid option. Synthax error?
full.log

Patrick · May 16, 2025, 10:15pm

GitHub - tabletseeker/whonix_builder: Docker Container that automatically builds Whonix or Kicksecure images using the official Whonix build script.

As someone who doesn’t use docker much… What’s the use case for using docker here?

Since other people previously did some Whonix docker related work… Let’s not re-invent this over and over again.

Does this need to be a separate repository?

Possible to merge upstream into derivative-maker?

Not a full review… Just some things I quickly noticed…

wget -qO- https://github.com/DNSCrypt/dnscrypt-proxy/releases/download/${DNSCRYPT_VER}/dnscrypt-proxy-linux_x86_64-${DNSCRYPT_VER}.tar.gz | \

issue as per:

major - lacks digital signature verification verification - see Verifying Software Signatures
minor - using wget - see Secure Command Line / Scurl

dnscrypt-proxy

issue:
- dnscrypt-proxy at time of writing is DNSSEC aware but dnscrypt-proxy at time of writing is DNSSEC non-validating
generally, more on that topic: DNS Security - Kicksecure

Don’t set it and no mount issue?

if [ "$dist_build_unsafe_io" = "true" ]; then

I assumed knowledge, that it’s clear that if this isn’t set, if not using --unsafe-io true, then none of the /var/cache/pbuilder mount related code will apply.

It really isn’t set yet. I assumed it to be set from the outside. But now that I think about it, perhaps run inside docker should be auto detected?

e-coin:

E: setup failed: E: apt-get -o Dir::Bin::dpkg=env -o DPkg::Options::=--unset=TMPDIR -o DPkg::Options::=dpkg -o DPkg::Chroot-Directory=/var/cache/pbuilder/base.cow_amd64 --yes install -oDebug::pkgProblemResolver=true -oDebug::pkgDepCache::Marker=1 -oDebug::pkgDepCache::AutoInstall=1 -oAPT::Status-Fd=<$fd> -oDpkg::Use-Pty=false apt sudo devscripts debhelper strip-nondeterminism fakeroot apt-transport-tor eatmydata aptitude cowdancer fasttrack-archive-keyring ?narrow(?or(?archive(^bookworm$),?codename(^bookworm$)),?architecture(amd64),?and(?priority(required),?not(?essential))) failed at /usr/bin/mmdebstrap line 192.

If this is important to you (can be avoided by not using unsafe io), please open a separate forum thread.

e-coin · May 16, 2025, 11:52pm

As someone who doesn’t use docker much… What’s the use case for using docker here?
Since other people previously did some Whonix docker related work… Let’s not re-invent this over and over again.

Really just for ease of use. I don’t use docker that much either, but I like the simplicity.
Maybe some people who wouldn’t have tried building from git, would be more inclined if all they needed to do is docker pull & ./run.sh
Let’s say outreach, maybe? lol

Does this need to be a separate repository?
Possible to merge upstream into derivative-maker?

Sure, I’d love that.
I originally just wanted to fix the broken parts of this.

issue as per:* major - lacks digital signature verification verification - see Verifying Software Signatures* minor - using wget - see Secure Command Line / Scurl

Sure, I can fix that.

dnscrypt-proxy at time of writing is DNSSEC aware but dnscrypt-proxy at time of writing is DNSSEC non-validating

Minor or Major in this case?

It really isn’t set yet. I assumed it to be set from the outside. But now that I think about it, perhaps run inside docker should be auto detected?

Sure, if you’d prefer that, but I think --env 'DOCKER=true' can be expected if using Docker.

Don’t set it and no mount issue?
if [ "$dist_build_unsafe_io" = "true" ]; then
I assumed knowledge, that it’s clear that if this isn’t set, if not using --unsafe-io true, then none of the /var/cache/pbuilder mount related code will apply.

Bear with me, please.

This is your commit in 1300_cow_builder:

   if [ "$dist_build_unsafe_io" = "true" ]; then
      if [ "$DOCKER" = "true" ]; then
         ## https://forums.whonix.org/t/docker-container-that-builds-whonix-images/17494/21
         true "INFO: Skip re-mounting /var/cache/pbuilder as tmpfs because DOCKER=true, ok."
      else
         if $SUDO_TO_ROOT mount | grep -- /var/cache/pbuilder | grep -- tmpfs ; then
            true "INFO: /var/cache/pbuilder already tmpfs."
         else
            $SUDO_TO_ROOT mount -t tmpfs -o size=2G none /var/cache/pbuilder
         fi
      fi
   fi

if $dist_build_unsafe_io is not true you never jump into if [ "$DOCKER" = "true" ] which skips the mount.

Shouldn’t it be this instead, then?

      if [ "$DOCKER" = "true" ]; then
         ## https://forums.whonix.org/t/docker-container-that-builds-whonix-images/17494/21
         true "INFO: Skip re-mounting /var/cache/pbuilder as tmpfs because DOCKER=true, ok."
      else
         if $SUDO_TO_ROOT mount | grep -- /var/cache/pbuilder | grep -- tmpfs ; then
            true "INFO: /var/cache/pbuilder already tmpfs."
         else
            $SUDO_TO_ROOT mount -t tmpfs -o size=2G none /var/cache/pbuilder
         fi
      fi

I understand, that if $dist_build_unsafe_io is not true, the entire code block is simply ignored, but that will cause the mount error again, which is why I’m confused.

If this is important to you (can be avoided by not using unsafe io), please open a separate forum thread.

Ok sure. I’m a bit confused on unsafe io right now. I’m gonna go through it with find and look for everything unsafe related to get a full picture.

Patrick · May 17, 2025, 12:31am

It works like this.

   if [ "$dist_build_unsafe_io" = "true" ]; then
      ## does not apply
   fi

Anything indented more (more white space) below doesn’t apply. [1]

In case the if isn’t true, nothing inside that if clause gets evaluated any further.

ChatGPT and other AIs can explain this code in detail.

But in any case: Trust me on saying:

As it’s implemented now:

not using --unsafe-io true means “no mount issues”.
using --unsafe-io true combined with env DOCKER=true means “no mount issues”
the only way to cause mount issues is to use --unsafe-io true without env DOCKER=true

Difficult. Automated signature verification and downgrade attack detection is kinda difficult.

Major because if DNS security enhancements aren’t implemented at the Kicksecure level, these shouldn’t be implemented at the derivative-maker level, let alone at the docker level.

That would also work but that would ignore $dist_build_unsafe_io. Would require nesting another if around to check that. At which point, that code gets needlessly complex.

The “unsafe io” should actually be safe nowadays. It was just bad naming. It translates to “use more RAM caches more for better speed, don’t ‘aggressively’ flush to disk”.

In worst case, there could be issues with unmounting during build. But that would “only” break the build. Nothing else. In practice, the “unsafe io” could be fixed and even become the default if the quirks can be sorted out.

[1] That indentation style is only convention. Unfortunately, unclean code in theory could have confusing Indentation style and would still be valid. But that’s not he case here.

e-coin · May 17, 2025, 9:22am

Anything indented more (more white space) below doesn’t apply. [1]
In case the if isn’t true, nothing inside that if clause gets evaluated any further.
ChatGPT and other AIs can explain this code in detail.

OK, I’ll take that shot of having an if explained to me.

using --unsafe-io true combined with env DOCKER=true means “no mount issues” `

Correct, which also makes sense.

  if [ "$dist_build_unsafe_io" = "true" ]; then
      if [ "$DOCKER" = "true" ]; then

That just produces the following.

env: unrecognized option '--force-unsafe-io'

not using --unsafe-io true means “no mount issues”.

No, that still triggers the same errror. [ log ]

I: mounting /proc filesystem
mount: /var/cache/pbuilder/base.cow_amd64/proc: permission denied.
       dmesg(1) may have more information after failed mount system call.
W: Aborting with an error
E: pbuilder create failed

This if was also ignored before 99e0ae9, since $dist_build_unsafe_io is not true. Logcially, that must mean the mount command is executed somewhere else.

   if [ "$dist_build_unsafe_io" = "true" ]; then
     
   fi

I’m gonna manually go through that build-step now.

Patrick · May 17, 2025, 12:46pm

This command isn’t run.

No mention of tmpfs inside the log.

Different error. Happening during execution of the real cowbuilder command.

This is a general docker issue for which I don’t have a solution.

e-coin · May 17, 2025, 1:48pm

This command isn’t run.
No mention of tmpfs inside the log.
Different error. Happening during execution of the real cowbuilder comman

Yeah ok, that was a miscommunication then. I thought that was the only one.

mount in an unprivileged container doesn’t work (by default).

I can get around the tmpfs one with docker run --tmpfs /var/cache/pbuilder:rw,dev,suid,exec,uid=1000,mode=1770 but you already fixed that.

I’ll check with some docker people and do more research on this. I’m sure a volume mount can be used somehow to get around it.
Otherwise, --privileged isn’t the end of the world anyways. (I just get obsessed with solving stuff even if there’s no actual need)

e-coin · May 17, 2025, 9:31pm

Btw, I noticed that your latest tag is called adrelanos_b340998e132ea3e11034883afdd07d0823429c79

Not sure if that’s on purpose, or a script messed up 17.4.0.0-developers-only.
But if that’s the new naming scheme. I’ll change the regex.
I’m using this for now:

curl -s https://api.github.com/repos/Whonix/derivative-maker/tags | jq '.[]' |  jq -r '.name | select(test("([0-9.]+-(developers|stable)|adrelanos[a-zA-z0-9._-]+)"))' | head -1

Patrick · May 18, 2025, 5:36am

That’s a bug. Ignore please. I’ll delete that tag.

Patrick · May 18, 2025, 9:08am

This is actually better. The less docker specific if’s we need in derivative-maker, the better.

Appreciated.

Avoiding --privileged would be nice, if feasible.

e-coin · May 18, 2025, 5:21pm

This is actually better. The less docker specific if’s we need in derivative-maker, the better.

Sure. I’m gonna test that properly then. Got through the cowbuilder build step before, though.

Appreciated. Avoiding --privileged would be nice, if feasible.

So, apparently every internal mount can theoretically be supplemented with a docker --volume, but first I have to properly understand how this is done in cowbuilder.

I’m gonna take some time to get familiar with it.

In the meantime I noticed a few things during testing that might be of interest.

Detached loop devices after a successful build(s) can cause kpartx to fail if a new build is started without removing them first.

sudo losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                                                               DIO LOG-SEC
/dev/loop1         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop6         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop4         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop2         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop0         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop7         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop5         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512
/dev/loop3         0      0         0  0 /derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw   0     512

Error:

kpartx -asv /home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw
mount: could not find any device /dev/loop#
can't set up loop

Reason: kpartx cannot assign beyond max dev threshold (7 by default)

Fix: Removing rogue loop devices on host via sudo dmsetup remove_all
(Isolating Whonix lo with grep probably better)

harcoded paths for help-steps in dm-prepare-release expect ~/derivative-maker

#hardcoded paths
source "$HOME/derivative-maker/help-steps/pre"
source "$HOME/derivative-maker/help-steps/colors"
source "$HOME/derivative-maker/help-steps/variables"

Could be frustrating for users who clone into a differently named directory to have the final build-step 5200 fail because No such file or directory:

I’m cloning this separately in whonix_builder, but dirty.

[ -d ~/derivative-maker ] || { mkdir ~/derivative-maker; cd ~/derivative-maker; git init -b master; \
git remote add -f origin ${GIT_URL}; \
git config core.sparseCheckout true; \
cat > .git/info/sparse-checkout << EOF
help-steps/pre
help-steps/colors
help-steps/variables
EOF
git pull origin master; }

e-coin · May 19, 2025, 11:12am

I install the trixie version of dnscrypt-proxy via apt now, and then just grab the binary from that layer.

github.com/tabletseeker/whonix_builder

Dockerfile

master

FROM debian:trixie-slim AS dnscrypt

RUN apt-get update && \
	DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y dnscrypt-proxy

FROM debian:bookworm-slim AS baseimage

ENV USER=user \
HOME=/home/user \
UID=1000 \
GID=1000 \
APT_CACHER_USER=apt-cacher-ng
ARG APT_CACHER_NG_CACHE_DIR \
APT_CACHER_NG_LOG_DIR

RUN apt-get update && apt-get install --no-install-recommends -y apt-transport-https ca-certificates && \
	### enable https sources ###
	sed -i "s|http|https|g" /etc/apt/sources.list.d/debian.sources && \
	apt-get update && apt-get install --no-install-recommends -y git \
	time curl lsb-release fakeroot dpkg-dev fasttrack-archive-keyring \

This file has been truncated. show original

Another way would be to download and extract it directly.

apt-get --download-only -o Dir::Cache="/tmp" -o Dir::Cache::archives="/tmp" install dnscrypt-proxy && \
dpkg --fsys-tarfile /tmp/dnscrypt-proxy*.deb | tar -xO ./usr/sbin/dnscrypt-proxy > /usr/bin/dnscrypt-proxy

As for dnssec, could this be an option? (apart from require_dnssec=true which prefers supported servers)

github.com/thuantran/dnscrypt-asuswrt-installer

DNSSEC should be validated by dnscrypt instead of dnsmasq

opened 06:45PM - 03 Mar 18 UTC

closed 02:13AM - 06 Mar 18 UTC

caribpa

Hi, I've found out that DNSSEC is being validated by dnsmasq (if enabled in the… UI) instead of dnscrypt-proxy. To change this behavior we have to append to dnsmasq.conf the option **proxy-dnssec**, to do that we edit the line 86 of the file **manager** like this: ``` append_on_demand /etc/dnsmasq.conf "no-resolv" "proxy-dnssec" "server=127.0.0.1#65053" ``` Then we have to disable DNSSEC in the UI to prevent it from adding the option _dnssec_ to dnsmasq.conf: ``` LAN (Advanced Settings) → DHCP Server → DNS and WINS Server Setting → Enable DNSSEC support = No ``` And then we reboot the router. I know that this proxy-dnssec option is not well known and does not appear in every manual but it's the way to take advantage of dnscrypt's full potential. You can read about proxy-dnssec [here](https://askubuntu.com/a/764600) and it's recommended in the [Arch Wiki](https://wiki.archlinux.org/index.php/DNSCrypt#dnsmasq).

Patrick · May 19, 2025, 11:33am

All mount/umount are supposedly clean. Meaning, there should be no leftover mounts under any circumstances. (Build success, failure, etc.)

derivative-maker has a sanity test shell functions in derivative-maker/build-steps.d/1100_sanity-tests at master · Kicksecure/derivative-maker · GitHub

check-stray-loop-devices
check-stray-mounts

that should at least notice such situations and abort the build process. It effectively check this:

sudo losetup --all
sudo cat /proc/mounts | grep -i -- "Whonix-Workstation-Xfce_image"

Isn’t this caught?

Cannot do that as this might remove volumes unrelated to derivative-maker.

That’s indeed a problem but hard to fix.
(Failing earlier would be possible.)

Why not simply use canonical location ~/derivative-maker for everything?

I’ll see if I can improve relative path support.

That is not a solution for the missing security feature: DNSSEC should be validated by dnscrypt instead of dnsmasq · Issue #31 · thuantran/dnscrypt-asuswrt-installer · GitHub

I don’t think that DNS security or any other system hardening is a task for derivative-maker (the image builder). A task for Kicksecure or Whonix, yes, but not for the image builder.

e-coin · May 19, 2025, 1:06pm

Edit: I keep replying to myself.

Hmm. Docker specific, apparently.
losetup returns nothing from inside the container, but on the host it does.

Cannot do that as this might remove volumes unrelated to derivative-maker.

I’m guessing something like this on the host could work for Docker, instead of just dmsetup remove_all.

sudo losetup -nl | grep -P "(Whonix|Kicksecure).*.raw" | awk '{print $1}' | while read lo; do sudo dmsetup remove -f "$lo"; done

Or straight up

sudo dmsetup remove -f $(sudo losetup -nl | grep -P "(Whonix|Kicksecure).*.raw")

Why not simply use canonical location ~/derivative-maker for everything?

Probably better anyway with checkout.

I was too lazy to change it, but originally each tag should be cloned into a separate folder with that tag name. So each time the container is restarted you either cd back into that folder if the same tag is chosen or create a new one. Just for looks, basically.

Patrick · May 19, 2025, 1:20pm

Also not that great. Could break future parallelized builds. (Build multiple flavors at once.)

The real solution is preventing this issue from happening. Otherwise it’s an indicator that something is broken anyhow. Could lead to inconsistent images.

e-coin · May 19, 2025, 1:51pm

Could break future parallelized builds

Damn, I didn’t even think of that. I loop the build command, based on how many flavors are passed. Of course, the second or nth one would fail if the lo devices of the previous build are still there. (Never actually checked if derivative-maker accepts multiple flavors at once)

You’re right. Ok, I’ll add that to the list. Gonna find out and report back.

e-coin · May 22, 2025, 2:14am

Hey man, so I’ve done some testing and I think this might be the problem.

Here are two builds, each having been run after a successful build. (To test with stray loop devices).

Build-A: (success)
Build-B: (fail)

sanity_test spots stray loop devices:

+ losetup --all
+ losetup_output='/dev/loop1: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop6: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop4: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop2: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop0: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop7: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop5: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop3: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)'
+ true 'INFO: Stray loop devices detected!

I’m guessing you’re still exiting zero in case there is a legitimate user loop device, but if losetup --all | grep -Eq "(Whonix|Kicksecure).*.raw" then maybe exit 1 during sanity?

Could it be unmount (help-steps/unmount-helper) being skipped? Which is the closest thing to a detachment that happens during the build (as far as I can see)

+ unmount_raw
+ trap exception_handler_unmount-raw ERR INT TERM
+ '[' '' = '' ']'
+ true
+ sync
losetup --all
/dev/loop1: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop2: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop0: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop3: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
+ '[' true = true ']'
+ true 'INFO: kpartx_only=true, skipping unmount /home/user/derivative-binary/Whonix-Gateway-CLI_image'
+ '[' '' = '' ']'
+ local img=/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw
+ wait 302268
+ sleep 2
+ sync

This happens when

   if [ "$kpartx_only" = "true" ]; then
      true "INFO: kpartx_only=$kpartx_only, skipping unmount $CHROOT_FOLDER"
   else
      "$dist_source_help_steps_folder/unmount-helper" "$CHROOT_FOLDER"
   fi

I haven’t yet found what sets kpartx_only, but this certainly looks interesting.

The way I’m solving the stray devices in whonix_builder is by executing the following function before each new build command. I can confirm that this works for now (also with parallel builds), but obviously not a real solution.

lo_check() { LOOP_DEV=$(sudo losetup -nl | grep -E "(Whonix|Kicksecure).*" | awk '{print $1}'); \
[ -z "${LOOP_DEV}" ] || { LOOP_PART=$(echo "${LOOP_DEV}" | sed -e 's,.*/,,'); \
LOOP_DM=$(sudo dmsetup info -c -o name --noheadings | grep "${LOOP_PART}") || true; \
sudo losetup -d ${LOOP_DEV}; [ -z "${LOOP_DM}" ] || sudo dmsetup remove -f ${LOOP_DM}; }; }

Lastly, a small thing I noticed with your commit in 5200_prepare-release

Before:

 dm-prepare-release "$@"

After:

 "$dist_source_help_steps_folder/dm-prepare-release" "$@"

This causes a file not found error:

/./build-steps.d/5200_prepare-release: line 22: /home/user/17.4.0.1-developers-only/help-steps/dm-prepare-release: No such file or directory

dist_source_help_steps_folder if otherwise unset becomes ${source_code_folder_dist}/help-steps" which is $(dirname -- "$MYDIR"), which is basically the clone folder.

There is no derivative-maker/help-steps/dm-prepare-release. (is there supposed to be?)

/usr/bin/dm-prepare-release is the correct location, which is why dm-prepare-release "$@" worked because it’s in the path.

I’m probably missing something here, so already preparing to be corrected.

Patrick · May 22, 2025, 4:23pm

I did Search the Source Code for kpartx_only.

kpartx_only is only used by build-steps.d/4400_zerofree-raw.

As per Whonix ™ Source Code Introduction you could experiment with running that build step only. Example:

./build-steps.d/1100_sanity-tests --target virtualbox --flavor whonix-gateway-cli
./build-steps.d/1200_prepare-build-machine --target virtualbox --flavor whonix-gateway-cli
./build-steps.d/3200_create-raw-image --target virtualbox --flavor whonix-gateway-cli
./build-steps.d/4400_zerofree-raw --target virtualbox --flavor whonix-gateway-cli

First step is kinda optional. Does not need to run over an over again. Can be dropped for each target+flavor+git tag.
Same goes for second step.
Third step only needs to be run once.
Fourth step then is idempotent. Re-running it should not make an effective difference.

After running these steps, I’ve run:

sudo losetup --all

Result:

(No stray loop devices.)

In theory, maybe another build step leaves stray loop devices.

It failed early during the first step (sanity tests). So nothing wrong with that run. That must be stray loop devices from a prior run.

Looking for the first mention of:

help-steps/unmount-raw

Even after:

kpartx -d -s -v /home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw

losetup --all

/dev/loop1: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)
/dev/loop0: [65027]:1230145 (/home/user/derivative-binary/17.4.0.0/Whonix-Gateway-CLI-17.4.0.0.Intel_AMD64.raw)

Problem: For some reason kpartx -d fails to remove the loop devices.

(That early build step isn’t related to kpartx_only.)

Just a bug in the development version. If the CI is failing - see Workflow runs · Whonix/derivative-maker · GitHub - no bug report needed.