Whonix for arm64 / Raspberry Pi (RPi)

Development
,
22

HulaHoop:

I am thinking about working on arm64 support and testing the results on qemu-arm.

Cool.

What files do I need to change or look at when doing this?

Is it as simple as adding the extra arch parameters in this file?

Don’t edit files in .d folders generally. Add your own. See Whonix
source code introduction. You can also use …/buildconfig.d to get away
from any conflicts.

But in this case you don’t even need that. --arch amd64 or similar is
enough.

You’ve added all architectures to the repo source lists so could it be simple as specifying one of them for debootsrap to download?

TBB isn’t available for anything but i386 and amd64 from TPO. Otherwise
maybe not much missing. Try.

23

Not relevant at the moment until I finish testing x86-64. Does the requirements for build machine kernel architecture apply here too? How would the instructions posted in your comment affect what I am trying to do here?

24

I don’t know that. Try.

Possible that you need to be running arm64 kernel. Or even arm64
architecture. Try.

For the build command, will certainly need both, --arch and --kernel.

Try this.

25

The other build seems to be churning along without problems and is pulling packages from stretch. At the moment the build script would not be able to support building images with a kernel archtiecture different from what the build machine is running so its certain this won’t work.

However debootstrap can do this if the build script supports passing the “–foreign” option.

The options available for crossbuilding:

https://wiki.debian.org/EmDebian/CrossDebootstrap#Cross-installing_Debian_using_debootstrap.2Fmultistrap

Example of Debootsrap’s foreign option:

https://wiki.debian.org/EmDebian/CrossDebootstrap#Generating_cross_images_as_root_user

26

HulaHoop:

At the moment the build script would not be able to support building images with a kernel archtiecture different from what the build machine is running so its certain this won’t work.

The build script is a lot less magic than you think. Grep the code for i386, i686 and amd64. There is nothing that makes it platform specific In essence, it combines Debian packages in commands in useful ways. Everything script does could also be done manually. So there is no inherent limitation. If there is a limitation, it would be one of for example [grml-]debootstrap. Not one of the build script.

However debootstrap can do this if the build script supports passing the “–foreign” option.

The options available for crossbuilding:

EmDebian/CrossDebootstrap - Debian Wiki

Example of Debootsrap’s foreign option:

EmDebian/CrossDebootstrap - Debian Wiki

https://github.com/Whonix/Whonix/blob/master/build-steps.d/1300_create-raw-image is of interest here. It runs grml-debootstrap, which internally uses debootstrap. So during 1300_create-raw-image runs, the command invocation line of grml-debootstrap is of interest here.

[grml-deboostrap is a tool that can create Debian raw images.
debootstrap is a tool that can install Debian into folders.]

So in summary, if grml-deboostrap can do, then Whonix’s build script can
also.

grml-deboostrap supports…

--debopt params

Extra parameters passed to the debootstrap command.

You can inject what’s being passed to grml-debootstrap --debopt through the following variable. Example.

whonix_build_debopt+=" --verbose "
27

Great it makes the script extensible.

Besides --foreign the paramter --second-stage-target=DIR is needed but what DIR should DIR take?

28

Can’t you just leave the defaults?

In case you don’t know, you are not supposed to create the full line for
invocation of debootstrap. grml-debootstrap passes most parameters just
fine. Search the build log for debootstrap. You only need to add any
extra arguments for cross platform building.

29

–foreign is required for crossbuilding and --second-stage-target= for debootstrap to be able to successfully build on non native hardware past stage 2 and as non root.

I’m thinking when doing crossbuilds a new folder with the intended arch name should be created. I don’t know if that directory should be in the Whonix source one or created outside.

30

~/whonix_binary folder somewhere.

Please make it work with deboostrap first. Then make it work with
grml-debootstrap. From there it will be simple to make it work with Whonix.

31

Seems that an actual ARM device would be needed for debootstrap’s second stage to succeed:

http://forums.debian.net/viewtopic.php?t=44617

The best option would be to just support a normal build script with --arch on an emulated Debian qemu-arm environment.

EDIT:

After trying that I ran into several problems mainly the extreme slowness of emulation and lack of a video device support.

The only practical way is to generate an image on an arm64 hardware when available with minimal changes to the build script by adding --arch arm64. Crossbuilding won’t be a problem then.

32

This should be left to the builder- getting a Debian jessie based arm64 environment. Bare metal or emulated. The build script can’t do this for the user. Because the emulated arm64 would need a bootable operating system also - a Debian jessie based arm64. Back to root one. There are to my knowledge no downloadable, gpg verifiable, official Debian raw images.

33

There is a different way to do this and it may be worth documenting. What is your opinion on:

https://wiki.debian.org/Arm64Qemu

34

HulaHoop:

There is a different way to do this and it may be worth documenting. What is your opinion on:

Arm64Qemu - Debian Wiki

If it works, why not.

Theoretically… Essentially, you need to do in preparation this step.

apt-get install qemu qemu-user-static binfmt-support debootstrap

(If that works, it can be automated.)

How lucky that grml-debootstrap allows to specify which deboostrap
binary to use. And set the following build variable…

DEBOOTSTRAP=‘qemu-debootstrap’

Plus --arch and plus --kernel. Could work. Try.

35

Successfully tested https://wiki.debian.org/Arm64Qemu

We would feed it the Whonix packages at the last part of step 6

36

HulaHoop:

Successfully tested Arm64Qemu - Debian Wiki

We would feed it the Whonix packages at the last part of step 6

We can’t wretch that all into a single command. At least not without
major refactoring. And it will probably not be needed. Please read all
files from the build-steps.d in lexical order file by file, from top to
bottom, line by line. No need to understand all at once, but then you’ll
see.

As I understand that Debian wiki page, it seems after qemu-debootstrap
did its job, one can just run commands within the chroot normally
without further required changes.

Therefore, given the proper build variables and command line parameters,
as detailed in my last post, the build should just work normally.

37

I found some documentation that actually puts everything together:

https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap

The current script stays the same except it downloads bootstrapping packages with the foreign option then the next step involves qemudebootsrap that can do syscall emualtion with qemu-static or a full system emulation with qemu-system-arm which takes it from there.

The latter is best for high level package compatibility.

38

HulaHoop:

ARM/RootfsFromScratch/QemuDebootstrap - Ubuntu Wiki

Can it create a bootable raw image?

39

https://wiki.debian.org/EmDebian/CrossDebootstrap#QEMU.2Fdebootstrap_approach

sudo apt-get install binfmt-support qemu qemu-user-static debootstrap

sudo mkdir debian_arm64_jessie

sudo debootstrap --foreign --arch arm64 jessie debian_arm64_jessie Index of /debian

sudo cp /usr/bin/qemu-arm-static debian_arm64_jessie/usr/bin

sudo DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true
LC_ALL=C LANGUAGE=C LANG=C chroot debian_arm64_jessie /debootstrap/debootstrap --second-stage

DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true
LC_ALL=C LANGUAGE=C LANG=C chroot debian_arm64_jessie dpkg --configure -a

To confiure extra options:

https://wiki.debian.org/EmDebian/CrossDebootstrap#Configuring_the_new_system_.28target_specific.29

These steps create a rootfs successfully however a kernel needs to be built/downloaded possibly with APT and I have an idea on what to do next but I don’t know how. Next step is to grab the linux arm64 package while chrooted and then fire up full system qemu emulation to verify the results.

Running qemu:

qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic -smp 1 -m 2048 -kernel aarch64-linux-3.15rc2-buildroot.img --append “console=ttyAMA0”

Debian on an emulated ARM machine
Alex's Adventures on the Infobahn – Running Linux in QEMU's aarch64 system emulation mode

40

I think forget about it. Creating a bootable raw image is non-trivial.
That’s what grml-debootstrap is for. Starting from debootstrap is like
reinventing grml-debootstrap. Rather than doing this, contribute the
missing feature to grml-debootstrap or image-bootstrap.

41

Last thing: I found a guide with all the exact steps to do what I was looking for. Please take a look and tell me if these steps can be automated:

https://archive.is/bBlzB (archived link)

grml-debootstrap or package compiling is easy when doing same architecture builds but crossbuilds and crosscompiling are where things get complicated. The qemu-arm compilation step is unecessary in this guide and the packaged version is good enough.