Qubes-whonix Debian package

Qubes-Whonix
#1

I just finished creating the whonix-qubes Debian package that contains all the configurations scripts, etc to allow Whonix to function in Qubes. I am pretty sure we previously agreed they belong with Whonix since they are Whonix specific.

These scripts will allow online updates to fix any bugs or add features that are not directly related to Whonix or Qubes. There are no dependencies on Qubes or Whonix since I plan on installing it as the first package once the template has been bootstrapped.

I tried to follow the Whonix packaging format as most as I could figure out but I also needed to make sure I can build it with Qubes builder which uses a different logic to build. I version-ed it to be same as Whonix Release, so this version is 9.4-1 as I figure I will need a matching version for each build.

It would be nice to get this package in the Whonix repo so I can start testing like adding a feature or fix a bug via an update as well as use the Whonix repo directly as my source when initially creating and building the template. There currently is one issue that would be nice to test on someone like WhonixQubes (I have a way to make those notification work properly; that feature just got missed in the massive merge).

Let me know of any issues. I don’t have much experience with Debian Packages so there may be some room for improvement.

My repo is here [url=https://github.com/nrgaway/whonix-qubes.git]https://github.com/nrgaway/whonix-qubes.git[/url]. I sign all my commits and tags. Here is a link to my public key [url=https://raw.githubusercontent.com/nrgaway/qubes-builder/master/nrgaway-qubes-signing-key.asc]https://raw.githubusercontent.com/nrgaway/qubes-builder/master/nrgaway-qubes-signing-key.asc[/url] for verification or you can grab key from gpg server (0x5A4C6DAD). Fingerprint: E0E3 2283 FDCA C1A5 1007 8F27 1BB9 B1FB 5A4C 6DAD.

#2

Wow. You are a machine! :smiley:

Yes.

Yes. Looking forward to seeing this package get incorporated by Patrick as well.

Yeah, I think I just read this in your NOTES file:

Current issues: - Whonix notifications seems to have reverted back to old style

I’m guessing you’re talking about the “whonixcheck” and “timesync” notifications I mentioned on the Qubes mailing list.

I think I now do remember seeing these notifications be pulled into the upper corner (black and white box) and disappear gracefully and natively by Qubes in one of your prior development builds I had installed.

I had disabled the “whonixcheck” script execution and “timesync” gui, using Patrick’s recommendations to manage this.

Looking for it to handle typical “whonixcheck” and “timesync” notifications without the typical Whonix popup windows getting in the way, and also handle such notifications when the VMs have internet access and when they are cut off from internet access.

Yes. I would be happy to test this fix out for you.

Let me know the best way to proceed with that.

Thanks a ton! :smiley:

#3

Awesome! Looking forward to review. Will do so after I return from c3c1. Shortly after next year.

Small comment on:
https://github.com/nrgaway/whonix-qubes/blob/master/usr/lib/whonix/utility_functions

We’re already setting a global WHONIX variable here (was maybe not the best idea at that time):
https://github.com/Whonix/whonix-base-files/blob/master/usr/lib/whonix/whonix.sh

So it might be better to change the variable name in utility_functions to avoid conflicts.

I don’t think this will be necessary:

 setPermissions root root 0644 /etc/uwt.d/50_uwt_default

Saw this in no Debian package yet. Files in /etc/… are owned by root:root anyhow. /etc/uwt.d/30_uwt_default got permissions 0644 by default. They get the same permissions as in the source folder. So just set permissions as you wish in source folder/git and commit. That should suffice.

For the same reason, https://github.com/nrgaway/whonix-qubes/blob/master/debian/whonix-qubes.preinst should be unnecessary.

enableSystemdUnits

Is that custom code? I think dh_systemd is run automatically during packaging if building on jessie. dh_systemd should abstract all the systemd magic without needing manual custom code in debian maintainer scripts.

The postrm script https://github.com/nrgaway/whonix-qubes/blob/master/debian/whonix-qubes.postrm does nothing yet and should be removed as long as it’s doing nothing. Unless you plan on letting it do something in future. (debhelper auto adds such a script to the deb as needed even if it’s not in the source.)

#4

Great. What is c3c1? A google search did not yield any relevant results.

Small comment on: https://github.com/nrgaway/whonix-qubes/blob/master/usr/lib/whonix/utility_functions

We’re already setting a global WHONIX variable here (was maybe not the best idea at that time):
https://github.com/Whonix/whonix-base-files/blob/master/usr/lib/whonix/whonix.sh

So it might be better to change the variable name in utility_functions to avoid conflicts.

Okay, I will change this. I do not think it should cause any issues with the current code since I do not export the variable and am only using it locally. It would override the global WHONIX=1 variable in the local context, but since I do not need access to that global variable it should be fine. None the less I will change it. Will be a good test as I was trying to find things that could be changed :slight_smile:

I don't think this will be necessary: 
 setPermissions root root 0644 /etc/uwt.d/50_uwt_default

Saw this in no Debian package yet. Files in /etc/… are owned by root:root anyhow. /etc/uwt.d/30_uwt_default got permissions 0644 by default. They get the same permissions as in the source folder. So just set permissions as you wish in source folder/git and commit. That should suffice.

For the same reason, https://github.com/nrgaway/whonix-qubes/blob/master/debian/whonix-qubes.preinst should be unnecessary.

I will remove the relevant code then. That makes it much simpler.

[code]enableSystemdUnits[/code]

Is that custom code? I think dh_systemd is run automatically during packaging if building on jessie. dh_systemd should abstract all the systemd magic without needing manual custom code in debian maintainer scripts.

Yes it is custom code. I wrote this to be able to verify that a service is either enabled and disabled. It does not attempt to enable the service if its already enabled as it first checks the status and will only enable if the status returns disabled. There are cases when you need to disable and re-enable as well.

Since these services are core to the operation I prefer to be over cautious and not rely on the installer script to ensure something is enabled as I did have some issues at one point which may or may not exist now. It could be the issues arose from being installed in a chroot environment, in which much of the systemd functionality is disabled. This Debian installation package scripts will be used both in the initial chroot creation of the template and on live systems for updates so I need to make sure its reliable in both cases.

On a side note, I have also experimented with systemd-nspawn and LXC containers for creating the templates as this gives me more of a live environment to work with with less limitations of chroot. I wrote some simple functions that intercept chroot and based on an environmental variable either use chroot, systemd-nspawn or LXC. This allowed me not to have to modify any source files currently using chroot. I only mention this since it may be of interest to you when building Whonix.

The postrm script https://github.com/nrgaway/whonix-qubes/blob/master/debian/whonix-qubes.postrm does nothing yet and should be removed as long as it's doing nothing. Unless you plan on letting it do something in future. (debhelper auto adds such a script to the deb as needed even if it's not in the source.)

I added the postrm as a template for completeness so its there if I need it. The current version of this package is the exact code implemented at this point in time when creating the Whonix-Qubes templates. My next job will be to actually use this package within my creation of the templates using qubes-builder, then implement any fixes. If it ends up not being used when we need to push our first update, I will remove anything that is not needed.

I may not have much time in near future, so I am not sure when I can complete this task but will try to squeeze it in; maybe train myself to code while I am sleeping :slight_smile:

#5

[quote=“nrgaway, post:4, topic:764”][quote author=Patrick link=topic=841.msg6185#msg6185 date=1419849827]
Awesome! Looking forward to review. Will do so after I return from c3c1. Shortly after next year.[/quote]

Great. What is c3c1? A google search did not yield any relevant results.[/quote]
See first post:

On a side note, I have also experimented with systemd-nspawn and LXC containers for creating the templates as this gives me more of a live environment to work with with less limitations of chroot. I wrote some simple functions that intercept chroot and based on an environmental variable either use chroot, systemd-nspawn or LXC. This allowed me not to have to modify any source files currently using chroot. I only mention this since it may be of interest to you when building Whonix.
Sounds interesting!
I may not have much time in near future, so I am not sure when I can complete this task but will try to squeeze it in; maybe train myself to code while I am sleeping :)
Ha! :)
#6

https://github.com/nrgaway/whonix-qubes/blob/master/etc/uwt.d/50_uwt_default should be renamed.

I tried to come up with some convention.

20_upstream_distro (debian)
30_whonix
31_whonix_alternative_configs
40_derivatives_based_on_whonix
50_and_above_for_users

Maybe rename it to 32_uwt_qubes? Or got any better suggestions?

#7

What about renaming the folder https://github.com/nrgaway/whonix-qubes/tree/master/usr/lib/whonix to [font=courier]/usr/lib/whonix-qubes[/font]?

Would help also with the conditional folder we need for:

#8

[quote=“Patrick, post:7, topic:764”]What about renaming the folder https://github.com/nrgaway/whonix-qubes/tree/master/usr/lib/whonix to [font=courier]/usr/lib/whonix-qubes[/font]?

Would help also with the conditional folder we need for:

I will rename the folder as well as the uwt rule, but not for this version. This current version I want make as close to possible to what is already released since a change like renaming critical directories will require installation removal scripts.

If you want to know if Qubes is installed, I would use the /var/run/qubes conditional and you could then use /usr/lib/whonix-qubes for whonix-qubes confirmation.

#9

Yes. Only talking about next version here. As in Whonix 10 and above. Such big changes should indeed not go into 9.x.

If you want to know if Qubes is installed, I would use the /var/run/qubes conditional
Would not /usr/lib/qubes/ be better for that?
#10

Still reviewing… Still looking at the easy stuff.

[hr]

dist-build-dep::
	@sudo apt-get update
	@grep Build-Depends: debian/control | cut -d: -f 2- |\
	    sed -e 's/ *, */\n/g'| cut -d'(' -f 1 |tr -d " " |\
	    xargs sudo apt-get install -y

This is a great addition. I would like to add this the generic “master” make file:
https://github.com/Whonix/Whonix/blob/master/Makefile

But with a small change. If one were to do this for all packages, it would be a waste always running “sudo apt-get update”.

Could you split this into two commands please? One that automatically runs “sudo apt-get update” and one that only apt-get installs?

Also slightly different names would be good if possible. Since it’s debian specific, “deb-pkg-update-build-dep” and “deb-pkg-build-dep” or so would be good names?

[hr]

Did you modify make-helper.bsh? If not, can you update to https://github.com/nrgaway/whonix-qubes/blob/master/make-helper.bsh please?

[hr]

debian/control… This description would look a bit more consistent and nicer.

Description: Qubes Configuration for Whonix-Gateway and Whonix-Workstation
 This package contains all the scripts and configuration options to be able
 to run Whonix-Gateway and Whonix-Workstation within a Qubes environment.
 .
 Whonix-Gateway should run as a ProxyVM.
 .
 Whonix-Workstation should run as an AppVM.
 .
 Template updates over Tor.
#11

most likely you already know this… For changing the paths, I am using a mass search and replace editor such as jedit. Allows changing everything at once without manually forgetting about something.

[hr]

You probably already know this as well. In debian/whonix-qubes.preinst this should not be required for the same reason as setPermissions is not required. Just noting so it will not be forgotten.

    # --------------------------------------------------------------------------
    # Create required directories
    # --------------------------------------------------------------------------
    mkdir -p /var/lib/whonix/init
    chown -R root:root /usr/lib/whonix
    chmod -R u+rwX,go+rX /usr/lib/whonix

[hr]

Why ship

I think those are likely to conflict with anon-base-files (GitHub - Kicksecure/dist-base-files: base files for distributions - several important miscellaneous files, such as /etc/hostname, /etc/hosts, /var/lib/dbus/machine-id and more), with

Can you remove those from whonix-qubes package or are they required for something? I think at the moment those files are not in use anyhow. Due to anon-base-files use of config-package-dev and .anondist extension, those will take precedence.

#12

In debian/whonix-qubes.triggers you are writing twice.

interest-noawait /etc/network/interfaces.whonix

Dunno if that could lead to issues. Unless it’s auto generated or some other reasons, I’d rather advice to play safe and remove one.

[hr]

I’ve never used triggers, but it’s most interesting. I see a potential issue here.

# Reset back to Whonix defaults
interest-noawait /etc/resolv.conf
interest-noawait /etc/hosts
interest-noawait /etc/hostname

I think you might want to use /etc/resolv.conf.anondist instead. Because the file known to dpkg is
https://github.com/Whonix/anon-gw-dns-conf/blob/master/etc/resolv.conf.anondist
(or https://github.com/Whonix/anon-ws-dns-conf/blob/master/etc/resolv.conf.anondist).
Moving /etc/resolv.conf out of the way and the symlink magic is done by config-package-dev. And I suppose that dpkg uses triggers on files it knows to manage. Probably not on dpkg-diverted files. But I might be wrong about this. I advice to actually test this if you want to make sure this really works.

Similarly, you might wish to use:

  • /etc/hosts.anondist and
  • /etc/hostname.anondist
    for the same reasoning.
#13

This is unnecessary:

Shipping a file

or so could be a better way.

#14

Instead of dynamically patching /usr/lib/anon-shared-helper-scripts/tor_bootstrap_check.bsh, what do you think about implementing some if/then magic into /usr/lib/anon-shared-helper-scripts/tor_bootstrap_check.bsh?

[hr]

Please add the https://github.com/Whonix/Whonix/blob/master/CONTRIBUTING.md file to the source root folder unless there are objections.

[hr]

typos:

  • annondist → anondist
  • accesss → access

[hr]

# Inject custom firewall rules into whonix_firewall

I would be happy if we could add some suitable hook mechanism to whonix_firewall or add some if/then code to whonix_firewall to simplify that code. Sed injection doesn’t look future proof.

[hr]

Generally all the scripts could use some “set -e” or “trap -E error_handler ERR”. You don’t necessarily have to break execution on ERR, but at least a textual notification for unexpected non-zero exit codes would be useful.

#15

The whonix-qubes packages uses a few [font=courier]Recommends:[/font]:

This is fine. Other packages by the Whonix project do this as well. But… You you need to know it has little effect.

Whonix’s build script installs all packages using “apt-get --no-install-recommends”. Reasons for that are a bit weird and difficult, but explained here:

So care to you get the [font=courier]Recommends:[/font] installed by default and plan to install the whonix-qubes package as part of [font=arial]build-steps.d/1700_install-packages[/font] or some anon-meta-package, then recommends would not be installed by default. We’d have to think of something. If you install it as part of the Qubes build process somehow without using --no-install-recommends then there will be no problem.

It’s just something to be aware of.

#16

[quote=“Patrick, post:10, topic:764”]Still reviewing… Still looking at the easy stuff.

[hr]

dist-build-dep::
	@sudo apt-get update
	@grep Build-Depends: debian/control | cut -d: -f 2- |\
	    sed -e 's/ *, */\n/g'| cut -d'(' -f 1 |tr -d " " |\
	    xargs sudo apt-get install -y

This is a great addition. I would like to add this the generic “master” make file:
https://github.com/Whonix/Whonix/blob/master/Makefile

But with a small change. If one were to do this for all packages, it would be a waste always running “sudo apt-get update”.

Could you split this into two commands please? One that automatically runs “sudo apt-get update” and one that only apt-get installs?

Also slightly different names would be good if possible. Since it’s debian specific, “deb-pkg-update-build-dep” and “deb-pkg-build-dep” or so would be good names?[/quote]

Yes I can do both

Did you modify make-helper.bsh? If not, can you update to https://github.com/nrgaway/whonix-qubes/blob/master/make-helper.bsh please?

I have been unable to get the newer make-helper.bsh play nice with Qubes builder. I am looking into it more now. I actually had to use the older script and remove epoch from changelog to get whonix-setup-wizard to build with qubes-builder (5 minutes ago I got it building at least). I am limited to what Qubes currently supports. Also Qubes does not support quilt format at the time, but I was able to get it to build anyway.

I will figure it out but I need to make sure I can build with both systems.

debian/control... This description would look a bit more consistent and nicer.

Description: Qubes Configuration for Whonix-Gateway and Whonix-Workstation This package contains all the scripts and configuration options to be able to run Whonix-Gateway and Whonix-Workstation within a Qubes environment. . Whonix-Gateway should run as a ProxyVM. . Whonix-Workstation should run as an AppVM. . Template updates over Tor.

Hehe. Yes it would. I lack in being descriptive :slight_smile: Will change that too!

#17

Can you change the location from GitHub - nrgaway/qubes-whonix: Configures Whonix to run as a ProxyVM / AppVM on Qubes OS to GitHub - nrgaway/qubes-whonix: Configures Whonix to run as a ProxyVM / AppVM on Qubes OS please?

Because the package is called whonix-qubes as well and the name of the folder is used in scripts as package name. Would really simplify things. That might have been the issue with make-helper.bsh also.

#18

Thanks for all the infos. I am too tired to go though it all today but I will get to it in the next few days.

I implemented many of your suggestions already including moving away from the /usr/lib/whonix directory. Qubes also has there own so I just picked a new one, based on how my systems unit files were named; /usr/lib/qubes-whonix. I changed the package name to match.

That Makefile scripts that installs dependencies won’t work when the depends span multiple lines. I am kind of surprised how lacking good tools are for Debian. I would have expected some easy to use command like Redhat has. Anyway, today I threw together a short python script that should handle multiple lines. I designed it so it will be able to parse pretty much anything from the control file. I also need something to read changelog for proper version detection since the EPOC was messing up Qubes builder. Hopefully that is sorted for now.

[url=https://github.com/nrgaway/qubes-whonix]https://github.com/nrgaway/qubes-whonix[/url].

#19

There is dpkg-parsechangelog. See also:

I think once based on jessie, it will be improved.

It’s used by make-helper.bsh also.

make-helper.bsh can also be sourced, it will do nothing then, but when you run make_get_variables it will set all the version numbers to variables it has parsed from debian/changelog.

Debian must have existing code to parse debian/control, no? Otherwise how is stuff like “apt-get build-dep” working?

#20

How to install / parse build dependencies from debian/control?:

I’d ask some more Debian devs for existing tools. Probably we’re not the first ones ever on the internet that need this, no?