Speedup Whonix 11 build time

[phabricator-migrator]

Information

ID: 295
PHID: PHID-TASK-3swcvgblkgrn5iwj5c6a
Author: nrgaway
Status at Migration Time: resolved
Priority at Migration Time: Normal

Description

We are considering using ----force-unsafe-io and eatmydata options to speed up the performance of installing the Debian templates and was wondering what your thoughts would be on implementing same for Whonix11 build, either as standard or as a build option?

The following is Marek’s comment, and your will see he states he is receiving a huge performance boost; What used to take 40 minutes to install has been cut to 10 minutes.

dpkg is quite slow in installing packages, most likely because it calls fsync() at each file. In Qubes IO have noticeable latency, especially when building template (effectively two loop device layers, or even three when building in DispVM).
Those patches disables most of fsync calls, which greatly improve performance (prepare-chroot-debian takes 10min compared to over 40 earlier). debootstrap itself still still calls fsync, because I haven’t found a way to pass dpkg options there and Fedora doesn’t have eatmydata package.

Of course, there is nothing for free - here we pay with data integrity - if the system crashes during chroot/template preparation, most likely the only option would be to remove it and generate again.

You can view, comment on, or merge this pull request online at:

https://github.com/marmarek/qubes-builder-debian/pull/10

Commit Summary

Add --force-unsafe-io to speedup installation
Further performance optimization - use ‘eatmydata’ for IO-intensive calls

File Changes

M prepare-chroot-debian (9)
M prepare-chroot-qubuntu (6)
M template_debian/01_install_core.sh (2)
M template_debian/distribution.sh (8)
M template_debian/vars.sh (2)

Patch Links:

[[ https://github.com/marmarek/qubes-builder-debian/pull/10.patch | https://github.com/marmarek/qubes-builder-debian/pull/10.patch ]]
[[ https://github.com/marmarek/qubes-builder-debian/pull/10.diff | https://github.com/marmarek/qubes-builder-debian/pull/10.diff ]]

Comments

---

Patrick

2015-05-10 18:01:57 UTC

Great idea! Great to have as a build option. Great for debug/personal/developers-only builds. I wouldn’t want to release testers-only or stable releases build with that option. Looking forward to it!

---

Patrick

2015-05-11 11:05:06 UTC

implemented $apt_misc_opts - https://phabricator.whonix.org/T295:
https://github.com/Whonix/Whonix/commit/cc635d254a9a77b5c0bdd33eedfe68009a77ce93

- implemented build parameter '--unsafe-io true', that speeds up builds, that uses '-o Dpkg::Options::=--force-unsafe-io', eatmydata and ignores 'sync'. - Thanks to @nrgaway for the suggestion!  - https://phabricator.whonix.org/T295
- implemented $apt_misc_opts - https://phabricator.whonix.org/T295

https://github.com/Whonix/Whonix/commit/5fc20ac69b6e994f6813ec7328c4a0a97f58d05a

---

Patrick

2015-05-11 11:43:44 UTC

don't use both, eatmydata command and LD_PRELOAD eatmydata, not required and causes conflicts with symlinks (dpkg-diversions, uwt):
https://github.com/Whonix/Whonix/commit/8f5c58f2765f7ef039e6c72e33afdaa7ce66830d

---

Patrick

2015-05-11 12:47:22 UTC

That didn’t work. ld preloading works a bit different on jessie. Fixed.

fix, refactoring, cleanup, output:
https://github.com/Whonix/Whonix/commit/3c172558a8afa348fccb9b64a8e02f58b1b05ab2

Currently testing if it speeds up the build.

---

Patrick

2015-05-11 13:34:01 UTC

--unsafe-io true has been implemented. As of 11.0.0.1.2-developers-only.

Benchmark results…

11.0.0.0.8-developers-only
sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target virtualbox
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:35:03'

11.0.0.1.2-developers-only
sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target virtualbox --unsafe-io true
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:31:09'

Not too big an improvement. Could be because I am using an SSD anyhow.

---

Patrick

2015-05-11 13:59:36 UTC

Doesn’t work for build-steps.d/1200_create-debian-packages yet. Working on that.

---

Patrick

2015-05-11 14:49:10 UTC

Fixed that.

fixed '--unsafe-io true' in build-steps.d/1200_create-debian-packages; unsafe-io wheezy compatibility; refactoring; output:
https://github.com/Whonix/Whonix/commit/3c172558a8afa348fccb9b64a8e02f58b1b05ab2

However, the time that is saved, if any, is below measuring inaccuracy.

sudo ./build-steps.d/1200_create-debian-packages --build --target root --allow-uncommitted true --allow-untagged true --internalrun --unsafe-io false
+ true 'INFO: End of: ./build-steps.d/1200_create-debian-packages | exit_code: 0 | error(s) detected: 0 | benchmark: 00:13:16'

sudo ./build-steps.d/1200_create-debian-packages --build --target root --allow-uncommitted true --allow-untagged true --internalrun --unsafe-io true
+ true 'INFO: End of: ./build-steps.d/1200_create-debian-packages | exit_code: 0 | error(s) detected: 0 | benchmark: 00:14:11'

Feel free to experiment with --unsafe-io true and tell me if that helps.

---

Patrick

2015-05-11 15:30:21 UTC

As of 11.0.0.1.3-developers-only…

sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target virtualbox --unsafe-io true
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:28:13'

---

Patrick

2015-05-11 16:52:19 UTC

As of 11.0.0.1.3-developers-only… This time for comparison, just creating a raw image. (Skips conversion to a vdi image as well as ova export.)

Why do I mention raw image benchmarks? Because that’s very similar to the Qubes build process. Only differences… Whonix: using grml-debootstrap for creation of the base image; not adding Qubes specific packages. Qubes: using qubes-builder-debian. My point is, you should be able to reach very similar build speeds.

sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target raw --unsafe-io true
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:22:08'

export make_use_lintian="true"
sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target raw --unsafe-io true
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:22:23'

export make_use_lintian="true"
sudo -E ./whonix_build --flavor whonix-gateway -- --report minimal --build --target raw --unsafe-io true --sanity-tests false
+ true 'INFO: End of: ./whonix_build | exit_code: 0 | error(s) detected: 0 | benchmark: 00:21:40'

That’s the ~20 minutes you were looking for?

---

nrgaway

2015-05-15 21:56:59 UTC

I be testing it soon. Looks like only new options are --unsafe-io true and --target qubes

---

Patrick

2015-05-16 01:15:04 UTC

Yes.

(--target qubes has time (low priority at this moment). Explanation and ticket for the actual switch to --target qubes: T307)

---

nrgaway

2015-05-16 12:57:54 UTC

Marek says he sees improvement. I may too; its hard to tell at this point; need a few more dozen builds to get a better idea.

Thanks for adding the option!