Make / make help / makefile / package build process changes

Small changes in packaging.

The scripts.

./build

And.

./clean

have been deleted and replaced. Those are now.

make deb-pkg

And.

make clean

This change have been made to comply with debian upstream guide (UpstreamGuide - Debian Wiki). They wish to have a “make dist” target, which was implemented.

Also other cool stuff has been added. See.

make help

Such as.

make deb-pkg-install

Which creates and installs the deb.

The wiki has been updated accordingly:

Overall it should now be more easy and standard complaint.

The installation of the packages is much easier! :slight_smile:

However, a dependency is missing in https://www.whonix.org/wiki/Dev/Build_Documentation/apparmor-profile-torbrowser: dh-apparmor (‘sudo apt-get install dh-apparmor’).

Glad you like it!

Added. :slight_smile:

Some more improvements. See.

make help

(Or https://github.com/Whonix/anon-base-files/blob/master/make-helper.bsh#L243)

Highlights:

There is now a combined command for building and installing the package to make debugging a package even simpler.

make deb-pkg-install

And you can now cleanup the source folder (remove debhelper temp files) as well as the parent folder where the deb is created using a single command.

make deb-cleanup

(Technical: Git has been removed as a build dependency, because Debian build servers work with upstream tarballs, and upstream tarballs do not include git folders.)

The package build should now be quite robust as well as generic. To test compatibility, with Debian build servers, I created my first ppa for testing purposes:
https://launchpad.net/~adrelanos/+archive/ubuntu/anon-apt-sources-list

While developing the apparmor-profiles, I found myself using deb-pkg-install and deb-cleanup a lot. To streamline debugging some more an optional additional make comment has bene added. (Example: Implemented make deb-icup: Combination of make deb-pkg, make deb-pkg-… · Kicksecure/anon-apt-sources-list@bbb0e61 · GitHub)

Implemented

make deb-icup

Combination of make deb-pkg, make deb-pkg-install and make deb-pkg-cleanup.

FYI

The make files of individual packages, for example,

I though it might be useful to have a “master location” and propose:

Unfortunately, and a bit ironically, [font=courier]whonix-developer-meta-files[/font] cannot use that makefile, because [font=arial]whonix-developer-meta-files[/font] as an exception is not (yet?) packaged as a normal Debian package. Historically grown that way. A fun exercise to refactor it some day. So if you want to propose changes to [font=courier]Makefile[/font] or [font=courier]make-helper.bsh[/font], please patch those files in the [font=courier]whonix-developer-meta-files[/font] repository. When useful changes have been made, we can copy that file over to all the packages. (I propose I’ll do that.) (Automation script available: https://github.com/Whonix/whonix-developer-meta-files/blob/master/debug-steps/packaging-helper-script)

[1] If we wanted [font=courier]individual features[/font], maybe make-helper.bsh could get some mechanism to [font=courier]source[/font] files from a yet to be created [font=impact]make-helper.d[/font] folder, that overrides functions or implements new features. Or not use the generic [font=courier]makefile[/font] for selected packages. But we can discuss this when we actually have need for this.

[hr]

If you’re wondering how to check any packaging code files do not include any malicious code? You don’t have to review the diff of all of them.

There is:
https://github.com/Whonix/whonix-developer-meta-files/blob/master/debug-steps/packaging-helper-script

Usability of that script is not great yet. Just a dev tool.

There is a function diff_common_packaging_files. You can comment it in.

#diff_common_packaging_files
->
diff_common_packaging_files

Then run:
debug-steps/packaging-helper-script

It’ll loop through all the packages and compare if the makefile(-helper) of any package matches the makefile(-helper) […].

I would like to implement a few more features.

Please check if the parameter names and descriptions make sense.

make tag-sign
   Sign git tag of latest debian/changelog version.
make tag-push
   Push git tag of latest debian/changelog version.
make tag-verify
   Verify git tag of latest debian/changelog version.
make tag-verify-checkout
   Verify and checkout git tag of latest debian/changelog version.
make tag-check
   Check if current git head needs to be tagged.

“make tag-sign” - should be easy to implement. (git tag -s “${make_pkg_version}-${make_pkg_revision}” -m . + sanity tests)

[hr]

“make tag-verify” - should be easy to implement. (git tag -v “${make_pkg_version}-${make_pkg_revision}” + sanity tests)

[hr]

“make tag-verify-checkout” - should be easy to implement. (git tag -v “${make_pkg_version}-${make_pkg_revision}” + git checkout “${make_pkg_version}-${make_pkg_revision}” + sanity tests)

[hr]

“make tag-push” - Not sure how to implement this. I would prefer something like “make tag-push origin adre”, where “origin” and “adre” are git remotes. (“git push adre gittag”). “origin” and “adre” a variables. Different devs have different git remote names. Unfortunately, make files don’t have a pretty way to pass command line parameters:

And “make tag-push REPO=origin” looks a bit ugly. Any ideas?

[hr]

“make tag-check” - The problem I want to solve here…

When there is a good time for tagging a git tag and/or bumping [debian/changelog] version numbers is non-obvious to me. Let’s say for example current git head is a8faad584571620ba42d26eb95069aac96e54379, current debian/changelog version is 0.7-1, tagged as 0.7-1. Now, when development continues and I am adding new commits, should I when the next commit is added on top of 0.7-1 already bump the [debian/changelog] version number? And when I think that version is ready, finalize it by git tagging it?

I’d like to prevent confusion and non-consistent results here. Someone who fetches the git repo’s master, using git head that might be a few commits ahead of 0.7-1, who runs “make deb-pkg” or even “make deb-icup” might think 0.7.1 is being installed, while (git describe output:) 0.7-1-2-geb9ac5f is being used.

“make tag-check” would run “git describe” and check if it’s output matched the debian/control version number. Exit 0 if it matches, error and exit 1 otherwise. Before a mass git tagging of all packages or mass git tag pushing, I could run a mass “make tag-check” to see if I still need to finalize/git tag releases. What do you think?

[hr]

Input appreciated.

[quote=“Patrick, post:7, topic:360”]I would like to implement a few more features.

Please check if the parameter names and descriptions make sense.

make tag-sign
   Sign git tag of latest debian/changelog version.
make tag-push
   Push git tag of latest debian/changelog version.
make tag-verify
   Verify git tag of latest debian/changelog version.
make tag-verify-checkout
   Verify and checkout git tag of latest debian/changelog version.
make tag-check
   Check if current git head needs to be tagged.

“make tag-sign” - should be easy to implement. (git tag -s “${make_pkg_version}-${make_pkg_revision}” -m . + sanity tests)

[hr]

“make tag-verify” - should be easy to implement. (git tag -v “${make_pkg_version}-${make_pkg_revision}” + sanity tests)

[hr]

“make tag-verify-checkout” - should be easy to implement. (git tag -v “${make_pkg_version}-${make_pkg_revision}” + git checkout “${make_pkg_version}-${make_pkg_revision}” + sanity tests)

[hr]

“make tag-push” - Not sure how to implement this. I would prefer something like “make tag-push origin adre”, where “origin” and “adre” are git remotes. (“git push adre gittag”). “origin” and “adre” a variables. Different devs have different git remote names. Unfortunately, make files don’t have a pretty way to pass command line parameters:

And “make tag-push REPO=origin” looks a bit ugly. Any ideas?[/quote]

Use a configuration file that the Makefile imports

Makefile

WHONIXCONF ?= whonix.conf

#Include config file
-include $(WHONIXCONF)

Then you can include any variables you will need in the config file. The config is really a Makefile format.

whonix.conf

REPO = origin
BUILD_TYPE = gateway
64BIT_LINUX = 1
CURRENT_SOURCES = 1
ENABLE_WHONIX_APT_REPOSITORY = 1
WHONIX_APT_REPOSITORY_DISTRIBUTION = wheezy
INSTALL_TO_ROOT = 1
SKIP_VERIFIABLE = 0
MINIMAL_REPORT = 0
SKIP_SANITY_TESTS = 0

Now you can over-ride the default config file by setting an environment variable or calling make like this:

WHONIXCONF=other.conf make <option>
"make tag-check" - The problem I want to solve here...

When there is a good time for tagging a git tag and/or bumping [debian/changelog] version numbers is non-obvious to me. Let’s say for example current git head is a8faad584571620ba42d26eb95069aac96e54379, current debian/changelog version is 0.7-1, tagged as 0.7-1. Now, when development continues and I am adding new commits, should I when the next commit is added on top of 0.7-1 already bump the [debian/changelog] version number? And when I think that version is ready, finalize it by git tagging it?

I’d like to prevent confusion and non-consistent results here. Someone who fetches the git repo’s master, using git head that might be a few commits ahead of 0.7-1, who runs “make deb-pkg” or even “make deb-icup” might think 0.7.1 is being installed, while (git describe output:) 0.7-1-2-geb9ac5f is being used.

“make tag-check” would run “git describe” and check if it’s output matched the debian/control version number. Exit 0 if it matches, error and exit 1 otherwise. Before a mass git tagging of all packages or mass git tag pushing, I could run a mass “make tag-check” to see if I still need to finalize/git tag releases. What do you think?

Input appreciated.


Personally, I think everything in master should be tagged.

So to start the tags is 0.7-1, you keep working adding commits and tag those as xx_1234abdfdsfsdd, where ‘xx’ is your initials and ‘1234abdfdsfsdd’ is the current HEAD. Now when you are ready to up the version, you then tag it as 0.7-2.

This makes it clear

We’re talking past each other here.

I am only talking about the make files of individual packages. Now some for the build script https://github.com/Whonix/Whonix.

Writing a config file with a git tag for a individual package to be able to use “make tag-push” rather than “git push origin tag ; git push adre tag” would not increase convenience.

For an eventual make file for the build script https://github.com/Whonix/Whonix please let’s use a separate thread.

Not sure I understand? You’re suggesting each and every commit must be tagged? Please open a separate thread or use Whonix Forum if appropriate.

Nevermind the “make tag-push”, I think just using a git alias would do:

Implemented the rest:

(Not yet mass updated the makefile of the packages.)

Please note, if there any other features missing in the make file for the packages (not Whonix/Whonix build script).