Installation from Repository / proverbial "sudo apt-get install whonix"

[phabricator-migrator]

Information

ID: 461
PHID: PHID-TASK-be7o7fngguw6e36cwh6t
Author: Patrick
Status at Migration Time: resolved
Priority at Migration Time: Normal

Description

Whonix Installation from Whonix APT Repository

discussion:
'sudo apt-get install whonix-(gateway|workstation)' feature
https://groups.google.com/forum/#!msg/qubes-devel/3-BC87cbWVk/Xby-YeqUBwAJ

TODO:

• Requires making Whonix build and work without several chroot scripts. Reimplementing them in postinst or otherwise.
• Handle whonixcheck Whonix News without build version file somehow or have somehow a build version file created.

Comments

---

Patrick

2016-01-04 23:59:38 UTC

converted chroot-script 40_add_torprojects_key to postinst script:
converted chroot-script 40_add_torprojects_key to postinst script · Kicksecure/anon-shared-build-apt-sources-tpo@495fc4c · GitHub

---

Patrick

2016-01-05 00:34:47 UTC

log build version

So the chroot-script
https://github.com/Whonix/anon-shared-build-log-build-version/blob/master/usr/lib/anon-dist/chroot-scripts-post.d/70_log_build_version
can be deprecated.

log build version · Kicksecure/dist-base-files@a9752eb · GitHub

removed anon-shared-build-log-build-version

Because that is now implemented in anon-base-files postinst.

removed anon-shared-build-log-build-version · Whonix/anon-meta-packages@a8d9e66 · GitHub

---

HulaHoop

2016-01-05 22:37:40 UTC

Its a great leap towards upstreaming and overlaps with some work needed for a hypothetical Whonix-Host metapackage.

This could solve the porting nightmare to new architectures. All a user would need to do on an ARM machine is just download the vanilla Debian images, install apt-transport-tor, add the Whonix repo and BAM! They are converted it into the Whonix images.

---

Patrick

2016-04-05 17:10:45 UTC

removed anon-shared-build-upgrade-torsocks since no longer required
    
(version from Debian is recent enough)

thereby chroot-script deprecated
    
https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/cc79fa15580bcb43f01e14d8f306b38eb8e0073e

---

Patrick

2016-04-06 15:50:36 UTC

removed anon-shared-build-inst-tb because it has been merged into tb-updater

https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/5b9189fc6397d24a20cd4d06abc74d63fc9f306c

---

Patrick

2016-04-06 19:56:52 UTC

converted chroot scripts into postinst script

https://phabricator.whonix.org/T461

converted chroot-script 40_add_torprojects_key to postinst script · Kicksecure/anon-shared-build-apt-sources-tpo@495fc4c · GitHub

---

Patrick

2016-04-06 20:17:34 UTC

Created a new package:
https://github.com/Whonix/apt-during-apt

---

More background:

• T472#8603
• T472#8606

---

download packages tor and tor-geoipdb from The The Project's apt repository

using apt-during-apt

thereby abolished previous chroot scripts for doing that

download packages tor and tor-geoipdb from The The Project's apt repo… · Whonix/anon-gw-anonymizer-config@8db878b · GitHub

---

Patrick

2016-04-06 20:20:37 UTC

deprecated anon-gw-build-upgrade-tor package since now integrated as postinst

in package anon-gw-anonymizer-config

https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/b40ad931432498b5865d341d10c102fdd487cfda

---

Patrick

2016-04-07 00:06:06 UTC

cleanup, got rid of obsolete /usr/lib/anon-dist/chroot-scripts-pre.d/20_sanity_checks

https://phabricator.whonix.org/T461

https://github.com/Whonix/anon-shared-build-sanity-checks/commit/55d1c95b06d38245e6505191cd3f03f440dbbd8b

---

Patrick

2016-04-07 00:32:09 UTC

fixed logging version of the package as build version

in case anon_dist_build_version is not set

https://phabricator.whonix.org/T461

fixed logging version of the package as build version · Kicksecure/dist-base-files@a2bc954 · GitHub

removed anon-shared-build-log-build-version because it was merged into anon-base-files

https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/a82bbc43599767d40b3151246b74d159276b4979

---

Patrick

2016-04-07 01:08:02 UTC

There are only 4 chroot scripts left.

find . -type f -not -iwholename '*.git*' | grep chroot-script

• anon-shared-build-sanity-checks/usr/lib/anon-dist/chroot-scripts-post.d/20_sanity_checks at master · adrelanos/anon-shared-build-sanity-checks · GitHub
• whonix-initializer/usr/lib/anon-dist/chroot-scripts-post.d/80_cleanup at master · adrelanos/whonix-initializer · GitHub
• anon-shared-build-ban-nonfree/usr/lib/anon-dist/chroot-scripts-post.d/75_vrms at master · adrelanos/anon-shared-build-ban-nonfree · GitHub
• anon-shared-build-remember-sources/usr/lib/anon-dist/chroot-scripts-post.d/75_sources at master · adrelanos/anon-shared-build-remember-sources · GitHub

I am wondering how/where to most elegantly re-implement them.

---

Patrick

2016-04-07 18:58:04 UTC

check for nonfree packages during

https://phabricator.whonix.org/T461

https://github.com/Whonix/whonixcheck/commit/41eb9e1eba1b1e45c39ff20b3730bf184d6432fe

removed anon-shared-build-ban-nonfree because it was merged into whonixcheck

https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/471b9aa5cbe0a28d6b5631dff3ba1b08420f3895

---

Patrick

2016-04-07 19:11:43 UTC

merged chroot scripts from anon-shared-build-sanity-checks and anon-shared-build-remember-sources

https://phabricator.whonix.org/T461

merged chroot scripts from anon-shared-build-sanity-checks and anon-s… · Kicksecure/initializer-dist@151eec1 · GitHub

added dependencies debsums and damngpl for new chroot scripts that were merged

https://phabricator.whonix.org/T461

added dependencies debsums and damngpl for new chroot scripts that we… · Kicksecure/initializer-dist@d087c44 · GitHub

removed anon-shared-build-remember-sources and anon-shared-build-sanity-checks

because they were merged into whonix-initializer

https://phabricator.whonix.org/T461

https://github.com/Whonix/Whonix/commit/e74de8c44c29c4888b05586546f069b14c99fb6a

---

Patrick

2016-04-07 19:30:28 UTC

• Chroot scripts were reduced to a minimum.
• They are no longer relevant for installation of Whonix from repository using apt-get install whonix-gatewayetc.
• Non-Qubes-Whonix: chroot scripts are still relevant for building Non-Qubes-Whonix images using Whonix’s build script. To my knowledge, there is no sane way to run the cleanup chroot script from within a Debian package maintainer script.
• Qubes-Whonix: chroot-scripts are mostly irrelevant. Mostly. When the whonix-gateway or whonix-workstation package gets installed into a Qubes Debian template - with the purpose of morphing it into Whonix - perhaps while running on top of the Qubes installer DVD - then the cleanup of the template would be up to the script doing that. It could do so by running /usr/lib/anon-dist/chroot-scripts-post.d/80_cleanup or otherwise.

---

marmarek

2016-04-07 21:12:08 UTC

Great! This will allow major reduction of duplicated data on installation ISO (Debian base packages, Whonix gw/ws common packages). And also result in smaller Whonix templates (no build depends installed there).
As for the cleanup - that’s fine - it can be done using salt management stack.

One minor thing: when installing Whonix from firstboot, it should be done without network access, because we can’t assume user have set it at that stage. But that isn’t a problem - we can bundle copy of appropriate deb packages in installation image. Then creating Whonix templates would look like this:

1. Clone debian-8 template to whonix-gw/whonix-ws
2. Start whonix-ws/whonix-gw
3. qvm-block --ro -A whonix-ws dom0:/usr/share/qubes-whonix/repository.img
4. qvm-run whonix-ws 'mkdir /mnt/whonix-repo && mount /dev/xvdi /mnt/whonix-repo
5. Add /mnt/whonix-repo to /etc/apt/sources.list.d/whonix-local.list (or sth like this)
6. Install appropriate packages (whonix-gateway/whonix-workstation)
7. Remove temporary repository, execute cleanup script, shutdown VM

That local repository could be easily created using apt-get --download-only install whonix-gateway whonix-workstation.
Instead of mounting repository as a block device, we could ship AppVM/ProxyVM with HTTP server configured, but IMO that would be overkill.
Any better idea? Do you see any drawbacks of such approach?

Question: do we need Debian minimal template for that? Or standard template also can be morphed into Whonix template?

---

Patrick

2016-04-07 23:26:12 UTC

I yet have to actually try apt-get install whonix-... and see how it goes. Will work on this next.

! In T461#8640, @marmarek wrote:
One minor thing: when installing Whonix from firstboot, it should be done without network access, because we can’t assume user have set it at that stage. But that isn’t a problem - we can bundle copy of appropriate deb packages in installation image.

One complication. Tor Browser. If tb-updater no longer runs in a chrooted environment with network access, it cannot download Tor Browser from torproject.org. Any idea how to solve that?

We concluded earlier already we don’t want to have a debian package hosting Tor Browser tarballs (tb-binary). (Mentioned in T417.) We might reconsider if not avoidable.

One way or another, Tor Browser tarballs would have to end up on Qubes installer DVD, then be copied into the to-be-morphed-to-Whonix Qubes Debian template. From there, tb-updater could take over with installation. (reusing that existing code)

That local repository could be easily created using apt-get --download-only install whonix-gateway whonix-workstation.

Sure, I don’t think that would resolve and download the dependencies also?

Instead of mounting repository as a block device, we could ship AppVM/ProxyVM with HTTP server configured, but IMO that would be overkill.

Yes, that sounds overkill. At least if not needed for various other purposes also.

Any better idea? Do you see any drawbacks of such approach?

Your idea sounds alright. No objections, just the details mentioned in this comment.

Question: do we need Debian minimal template for that? Or standard template also can be morphed into Whonix template?

At the moment the packages expect nothing other than Debian, something as minimal as a debootstrapped system. [everything else could be seen as a package bug]

However, I don’t think we need the Debian minimal template. For two reasons.

• It’s less tested.
• It has had issues in past.

Maybe these are no valid reasons. (Issues in past were due to missing installed Qubes default packages. And after it was morphed into Whonix, it would be more widely tested.)

Debian standard template should work,

• as soon as ported to gnome to avoid duplicate applications (konsole plus gnome-terminal installed at the same time), i.e. Whonix 14.
• as long as it does not install any unwanted packages by default.

---

marmarek

2016-04-08 00:12:10 UTC

We concluded earlier already we don’t want to have a debian package hosting Tor Browser tarballs (tb-binary). (Mentioned in T417.) We might reconsider if not avoidable.

Indeed that may be a complication. In worst case we can include downloaded tarball in the same image as debs and have some script which will hand it over to tb-updater (copy to /var/cache/tb-updater so tb-updater will use that?). But that soulds like a hack, not a clean solution.

Sure, I don’t think that would resolve and download the dependencies also?

apt-get --download-only install should download everything what is needed. apt-get download downloads only a single package, not dependencies.

• It’s less tested.
• It has had issues in past.

Hmm, maybe we should add it to the set of templates I’m running automated tests on? We don’t have much tests for VM stuff, but at least something.

as long as it does not install any unwanted packages by default.

There is chrony + ntpdate. And there is also /usr/bin/wnpp-check, but it belongs to devscripts (that may be only my template, not installed by default).

I see two possible solutions:

• handled that using custom step in Whonix setup script (just before step 6)
• have some (optional?) package, which will “conflict” with all the unwanted packages (“conflict” = the type of dependency which removes the other package - is it “replaces”?)

---

Patrick

2016-04-08 00:18:55 UTC

current blockers:

• 1. Tor Browser tarball / installation (described above)
• 2. download of tor, tor-geoipdb, deb.torproject.org-keyring not possible (T472). If we have no networking during that stage… Which is on one hand a very good thing, so we do not depend on it.

If anyone has any ideas how to solve that without increasing maintenance effort…

---

marmarek

2016-04-08 00:39:31 UTC

2. download of tor, tor-geoipdb, deb.torproject.org-keyring not possible (T472). If we have no networking during that stage… Which is on one hand a very good thing, so we do not depend on it.

Just for installation it shouldn’t be a problem - we can simply add deb.torproject.org repo for the time of apt-get --download-only install .... But it doesn’t help in anyway how to keep track on tor package later.

Hmm, maybe for T472 do not download the package separately, but simply keep some package with strict version dependency like tor (= x.y.z)? That should work in both cases - collecting packages for offline installation and later tracking its version. Does deb.torproject.org keep old version of packages in repository?

---

Patrick

2016-04-08 15:26:45 UTC

! In T461#8647, @marmarek wrote:
2) download of tor, tor-geoipdb, deb.torproject.org-keyring not possible (T472). If we have no networking during that stage… Which is on one hand a very good thing, so we do not depend on it.

Just for installation it shouldn’t be a problem - we can simply add deb.torproject.org repo for the time of apt-get --download-only install .... But it doesn’t help in anyway how to keep track on tor package later.

anon-shared-build-apt-sources-tpo could do that. It ships /etc/apt/souces.list.d/torproject.list. Then newer versions on installed systems would come from deb.torproject.org.

Hmm, maybe for T472 do not download the package separately, but simply keep some package with strict version dependency like tor (= x.y.z)? That should work in both cases - collecting packages for offline installation and later tracking its version.

Does deb.torproject.org keep old version of packages in repository?

No.

---

HulaHoop

2016-04-08 18:37:33 UTC

systemd-nspawn talked about as a drop in replacement for chroot in build environments:

systemd for Administrators, Part VI
https://lindenberg.io/blog/post/debian-containers-with-systemd-nspawn/

Could it be an answer for the chroot problem in non-Qubes builds?

---

Patrick

2016-04-14 23:39:26 UTC

added install-from-local-repository script

https://phabricator.whonix.org/T461

added install-from-local-repository script · Kicksecure/developer-meta-files@976b713 · GitHub

---

Patrick

2016-04-15 01:16:28 UTC

more robust handling of /etc/hostname

https://phabricator.whonix.org/T461

more robust handling of /etc/hostname · Kicksecure/dist-base-files@887094d · GitHub

---

Patrick

2016-04-19 22:29:02 UTC

This is done. Further work being tracked in T498.