Information
ID: 278
PHID: PHID-TASK-x5gw7rfkg76j2vhjw4ro
Author: nrgaway
Status at Migration Time: resolved
Priority at Migration Time: High
Description
I am getting errors like these:
The following packages have unmet dependencies:
network-manager : Depends: libpam-systemd but it is not going to be installed
Depends: policykit-1 but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
My build script looks like this:
This script is called first so I can set some stuff up as sudo user before calling the actual build script:
# Pre Fixups
sudo mkdir -p /boot/grub2
sudo touch /boot/grub2/grub.cfg
sudo mkdir -p /boot/grub
sudo touch /boot/grub/grub.cfg
sudo mkdir --parents --mode=g+rw "/tmp/uwt"
# Whonix seems to re-install sysvinit even though there is a hold
# on the package. Things seem to work anyway. BUT hopfully the
# hold on grub* don't get removed
sudo apt-mark hold sysvinit
sudo apt-mark hold grub-pc grub-pc-bin grub-common grub2-common
# Whonix expects haveged to be started
sudo /etc/init.d/haveged start
sudo ~/whonix_build_post $@
Then this is called (whonix_build_post
)
#!/bin/bash -e
# vim: set ts=4 sw=4 sts=4 et :
export WHONIX_BUILD_UNATTENDED_PKG_INSTALL="1"
# Whonix 11 Hacks (stretch does not exist)
export whonix_build_apt_newer_release_codename="jessie"
# Make sure we clear Qubes overrides of these vars
#export GENMKFILE_INCLUDE_FILE_MAIN=
#export GENMKFILE_BOOTSTRAP=
# Prevents Whonix makefile use of shared memory 'sem_open: Permission denied'
echo tmpfs /dev/shm tmpfs defaults 0 0 >> /etc/fstab
mount /dev/shm
# --freshness current \
pushd /home/user/Whonix
/home/user/Whonix/whonix_build \
--flavor $1 \
-- \
--build \
--arch amd64 \
--kernel linux-image-amd64 \
--headers linux-headers-amd64 \
--target root \
--report minimal \
--verifiable minimal \
--allow-uncommitted true \
--sanity-tests false || { exit 1; }
popd
Comments
Patrick
2015-04-27 00:59:30 UTC
Minor stuff:
Probably Qubes specific? Because I just succeeded package installation step with Whonix-Gateway-11.0.0.0.1-5-gc94a68696170a5febe680a92452eadb33743c485
. A real tag is coming soon. Just wanted to try with early help fixing this.
If you like my help, you can try providing a full build log, because by the current information provided I don’t know what could be causing this.
nrgaway
2015-04-27 01:08:02 UTC
Is there a way to build with current sources
? That may solve the issue.
Qubes components are built with current-sources
and may also require current-sources for Whonix
although its too early to tell. When we are at a point that I can build with both frozen
(which do not exist for jessie) and current-sources
, I will be able to determine that.
I also do realize there is a debate over which method is better, frozen
or current
which seems to tend to go either way based on opinion. In relation to Qubes, I tend to lean in favour of current sources
, but provide an option to build with frozen sources
since as stated Qubes is currently built with current sources
for all distributions and packages and Marek keeps up with security patches. Also the use case and threat model is different from a VirtualBox environment.
I do not wish to enter such a debate within this task, but am open to discussing it within a thread on the forums if anyone has any views on this especially in relation to QubesOS where considerations can be taken into the already present security model of Qubes since I think reasonings from that discussion could differ from previous discussions on the matter.
Currently, I would be most please to just get past the build install errors and test next with current sources.
Questions:
What is the purpose of whonix_build_apt_newer_release_codename
. By default it was set to stretch
which gave build errors so I set it to jessie
What is required to use the current sources
.
Is the definition of current sources
that is uses the standard main
Debian repos?
nrgaway
2015-04-27 01:13:34 UTC
! In T278#3998, @Patrick wrote:
Minor stuff:
export WHONIX_BUILD_UNATTENDED_PKG_INSTALL="1"
is the default anyhow, no need to set it?
Great, will remove it
Probably Qubes specific? Because I just succeeded package installation step with Whonix-Gateway-11.0.0.0.1-5-gc94a68696170a5febe680a92452eadb33743c485
. A real tag is coming soon. Just wanted to try with early help fixing this.
Good, I will also try it. I am not sure what the issue may be at the moment. I am building a base Debian system using debootstrap
then install a few packages to prep Whonix, then use a chroot
environment to build Whonix
.
If you like my help, you can try providing a full build log, because by the current information provided I don’t know what could be causing this.
Thank you. Where would this build log be found, or should I redirect screen output to file?
Where / how should I send it?
Patrick
2015-04-27 02:06:15 UTC
Was a bug that is now fixed (https://github.com/Whonix/Whonix/commit/34b042ac565056a5da67e2140a5d1a50f31d9b98 ) in latest master. [And soon a new tag.] Related to build-steps.d/1100_prepare-build-machine
. The purpose was to be able to install packages from a newer codename. On wheezy
we needed to install a few packages from jessie
. That’s what whonix_build_apt_newer_release_codename
was good for. What will it be good for? I hope it can be avoided, but I could imagine at some point while we’re jessie
based, we may need to install packages from stretch
. I am not sure about it and I was thinking about removing that build script feature.
Frozen vs Current Sources, it should all be possible already. I made a detailed answer in the forums already. See:
Whonix Forum
Please add if there are any follow up questions.
For a pros and cons of Frozen vs Current Sources, see https://www.whonix.org/wiki/Dev/Build_Documentation/9_full#CurrentSources_Builds_.28Optional.29 and click expand.
The discussion thread of Frozen vs Current Sources could be this one:
Whonix Forum
(It’s not about VirtualBox vs non-VirtualBox at all. Same pros / cons for any image.)
No. Current Sources is the regular repository. Frozen Sources is a specific snapshot from snapshot.debian.org .
main
, contrib
, non-free
are called components (as per Debian apt sources.list terminology). Has nothing to do with Current vs Frozen Sources.
Just set your Konsole
(if you are using that) to unlimited scrollback (gui, profile settings) and then use the gui to save the whole log. Or similar. You can try pasting it in here surrounded by the usual code tags. Or in the forums using code tags. Or on some free paste service. Another option is to use regular redirections to store the log into a file. >> ~/log 2>&1
Then tail -f ~/log
in another terminal. Doesn’t really matter where / how. The log just somehow needs to materialize somewhere.
nrgaway
2015-04-27 09:24:41 UTC
I have been about to get Whonix 11
to successfully build using the 11.0.0.0.1-developers-only
tag (since submodules
seemed broken on master
branch), although the build process kept asking for input starting at line 450 of 1700_install-packages
and all the way until it exited. I just kept entering c
manually since by that stage everything had already been installed:
# 1700_install-packages:450 --> "$WHONIX_SOURCE_HELP_STEPS_FOLDER/remove-local-temp-apt-repo"
INFO: Setting... export UWT_DEV_PASSTHROUGH="1"
INFO: Variable anon_dist_build_version was already set to: 11.0.0.0.1
/home/user/Whonix/help-steps/pre: line 20: error_: command not found
...
+ true 'INFO: Currently running script: /home/user/Whonix/help-steps/unprevent-daemons-from-starting '
+ true 'INFO: Currently running script: /home/user/Whonix/help-steps/unchroot-raw '
+ true 'INFO: Skipping script, because ANON_BUILD_INSTALL_TO_ROOT=1: /home/user/Whonix/help-steps/unmount-raw'
+ true 'INFO: Currently running script: ././build-steps.d/2300_run-chroot-scripts-post-d '
In order to prevent the original build dependencies as described in task details, I had to make sure Whonix was installed directly after debootstrap. Normally I would have installed the base Debian system as distributed before running Whonix build. The build process seems intolerant to unknown packages. This seems acceptable since I was able to build Whonix, and update with Qubes required packages afterwards.
Since I ran whonix_build
next after debootstrap
I did not have to uninstall apt-listchanges
. I mention this since you may want to consider modifying your apt-get
parameters to some of the ones I use that prevent the need to uninstall apt-listchanges
:
DEBIAN_FRONTEND="noninteractive" DEBIAN_PRIORITY="critical" DEBCONF_NOWARNINGS="yes" \
chroot apt-get ${APT_GET_OPTIONS} install ${files[@]}
After building I was able to also successfully run the gateway
and workstation
.
Gateway
issues included the following:
torsocks errors:
[Apr 27 08:05:16] WARNING torsocks[31405]: [syscall] Unsupported syscall number 202. Denying the call (in tsocks_syscall() at syscall.c:165)
gpg issues?:
## error_cause: error_handler called with error_text: /usr/bin/whonixcheck script bug Please report this bug!
BASH_COMMAND: gpg_bash_lib_output_gpg_import_output="$( gpg --no-options --ignore-time-conflict
--homedir "$gpg_bash_lib_input_temp_folder" --import "$gpg_bash_lib_internal_key" 2>&1 )" exit_code: 1
----- gpg_bash_lib_output_diagnostic_message: gpg_bash_lib_internal_gpg_verify_status_fd_file:
/tmp/tmp.7NYjOmRiLu/news_verify_dir/news_gpg/gpg_bash_lib_internal_gpg_verify_status_fd_file
gpg_bash_lib_internal_gpg_verify_output_file: /tmp/tmp.7NYjOmRiLu/news_verify_dir/news_gpg
gpg_bash_lib_internal_gpg_verify_output_file gpg_bash_lib_output_gpg_import_output: mktemp: failed to
create file via template '/tmp/uwt/tmp.XXXXXXXXXX': Permission denied mktemp failed with exit code 0!
gpg_bash_lib_output_gpg_verify_output: gpg_bash_lib_output_gpg_verify_status_fd_output:
Workstation
issues included the following:
Same gpg?
issue
Tor Browser downloaded successfully, but also received some gpg error
Both Whonix10 and 11 had some strange display issues in regards to dialog boxes that pop up. Some of the dialog boxes are partially transparent and seem not to be receiving proper focus until clicked on and some time three quartes of the dialog is transparent when the other quarter is normal, and the smaller dialogs seems to have the bottom two corners transparent as well until focus is gained.
I have done some research and such a bug has been reported 6 weeks ago to kde as other applications are also having same issue. I am going to also going to build a gnome workstation and see if that makes any difference, since the bug was said not to effect gnome. It could also just be my windows manager; guess that’s something @WhonixQubes can also look for when testing (which can be done when the required qubes component is re-released with fix for wheezy)
It time for me to get some sleep, but now that this builds, I can start testing with it so later today I will first finish off the issues regarding the replace-ip
regression and a few others and then package up control-port-filter-python
.
WhonixQubes
2015-04-27 11:11:24 UTC
WhonixQubes
2015-04-27 11:13:12 UTC
Patrick
2015-04-27 14:48:28 UTC
! In T278#4002, @nrgaway wrote:
I have been about to get Whonix 11
to successfully build using the 11.0.0.0.1-developers-only
tag (since submodules
seemed broken on master
branch)
Yes. That should now be fixed. (I was mass bumping debian/control Standards-Version to fix a lintian warning and did not push all the packages by that time. Will also need to mass bump package versions after fixing T281.)
In order to prevent the original build dependencies as described in task details, I had to make sure Whonix was installed directly after debootstrap.
Interesting. Well, that’s how Whonix raw images are usually build outside of Qubes build process. That’s why something else might not work. Not tested before. Got a related question about that:
Whonix Forum
Normally I would have installed the base Debian system as distributed before running Whonix build. The build process seems intolerant to unknown packages.
Intolerant, well. Implicitly intolerant. That’s a bug. There is no explicit counter measure or so. I’d be interested to fix it and/or to take patches. A full build log would be required here.
This seems acceptable since I was able to build Whonix, and update with Qubes required packages afterwards.
Alright.
Since I ran whonix_build
next after debootstrap
I did not have to uninstall apt-listchanges
. I mention this since you may want to consider modifying your apt-get
parameters to some of the ones I use that prevent the need to uninstall apt-listchanges
:
You bet I am interested in that. I dislike glitches, baggage such as “apt-listchanges should not be installed”. That piles up. I want to do that soon for Whonix 11. Created T282 for it where I have a question.
After building I was able to also successfully run the gateway
and workstation
.
Gateway
issues included the following:
torsocks errors:
[Apr 27 08:05:16] WARNING torsocks[31405]: [syscall] Unsupported syscall number 202. Denying the call (in tsocks_syscall() at syscall.c:165)
Needs a separate ticket and instructions on how to reproduce. Possibly a log.
gpg issues?:
## error_cause: error_handler called with error_text: /usr/bin/whonixcheck script bug Please report this bug!
BASH_COMMAND: gpg_bash_lib_output_gpg_import_output="$( gpg --no-options --ignore-time-conflict
--homedir "$gpg_bash_lib_input_temp_folder" --import "$gpg_bash_lib_internal_key" 2>&1 )" exit_code: 1
----- gpg_bash_lib_output_diagnostic_message: gpg_bash_lib_internal_gpg_verify_status_fd_file:
/tmp/tmp.7NYjOmRiLu/news_verify_dir/news_gpg/gpg_bash_lib_internal_gpg_verify_status_fd_file
gpg_bash_lib_internal_gpg_verify_output_file: /tmp/tmp.7NYjOmRiLu/news_verify_dir/news_gpg
gpg_bash_lib_internal_gpg_verify_output_file gpg_bash_lib_output_gpg_import_output: mktemp: failed to
create file via template '/tmp/uwt/tmp.XXXXXXXXXX': Permission denied mktemp failed with exit code 0!
gpg_bash_lib_output_gpg_verify_output: gpg_bash_lib_output_gpg_verify_status_fd_output:
Needs a separate ticket. Please attach the output of the following two commands.
whonixcheck --function download_whonix_news
bash -x whonixcheck --function download_whonix_news
Workstation
issues included the following:
Same gpg?
issue
Tor Browser downloaded successfully, but also received some gpg error
Not sure if it’s the same gpg
issue. Possibly be a bug in #gpg-bash-lib .
Needs a separate ticket. Please attach the output of the following two commands.
update-torbrowser --input gui
bash -x update-torbrowser --input gui
Patrick
2015-04-27 22:21:34 UTC
nrgaway
2015-04-27 22:30:08 UTC
Patrick
2015-04-30 14:59:43 UTC
nrgaway
2015-04-30 22:21:08 UTC
! In T278#4002, @nrgaway wrote:
I have been about to get Whonix 11
to successfully build using the 11.0.0.0.1-developers-only
tag (since submodules
seemed broken on master
branch), although the build process kept asking for input starting at line 450 of 1700_install-packages
and all the way until it exited. I just kept entering c
manually since by that stage everything had already been installed:
# 1700_install-packages:450 --> "$WHONIX_SOURCE_HELP_STEPS_FOLDER/remove-local-temp-apt-repo"
INFO: Setting... export UWT_DEV_PASSTHROUGH="1"
INFO: Variable anon_dist_build_version was already set to: 11.0.0.0.1
/home/user/Whonix/help-steps/pre: line 20: error_: command not found
...
+ true 'INFO: Currently running script: /home/user/Whonix/help-steps/unprevent-daemons-from-starting '
+ true 'INFO: Currently running script: /home/user/Whonix/help-steps/unchroot-raw '
+ true 'INFO: Skipping script, because ANON_BUILD_INSTALL_TO_ROOT=1: /home/user/Whonix/help-steps/unmount-raw'
+ true 'INFO: Currently running script: ././build-steps.d/2300_run-chroot-scripts-post-d '
I just completed Whonix 10 and have not had much time yet to debug your replay above. I am running a build right now and still have the build error where everything starts producing errors as initially described above using 11.0.0.0.1-developers-only
and 11.0.0.0.3-developers-only
tags.
I figure I should at least get a successful build before I start to debug other issue since there could be the possibility something was not installed correctly to begin with due to the build errors.
Where do you want the logs sent to as they are large.
Patrick
2015-05-01 01:03:15 UTC
nrgaway
2015-05-01 08:42:49 UTC
I was able to identify all the issue that prevented complete and successful build.
help-steps/variables:
lsb_release --short -i
is returning ‘Whonix’, not ‘Debain’ which causes error
TEMP HACK:
export whonix_build_on_operating_system="debian"
The following are all grub related. I did not install grub for Whonix 11 and pulled out the code that fakes installation of grub since its not needed.
build-steps.d/2300_run-chroot-scripts-post-d
run-parts --verbose --exit-on-error "/usr/lib/anon-dist/chroot-scripts-post.d/"
/usr/lib/anon-dist/chroot-scripts-post.d//30_backup_grub_cfg
cp /boot/grub/grub.cfg /var/lib/anon-dist/grub-backup/grub.cfg.chroot-post1
/usr/lib/anon-dist/chroot-scripts-post.d//85_update_grub
/usr/lib/anon-dist/chroot-scripts-post.d//90_fix_grub
cp /boot/grub/grub.cfg /var/lib/anon-dist/grub-backup/grub.cfg.chroot-post4
In regards to the grub
errors, what are our options?
export ENV var to preven grub
related items from installing?
configuration files to skip
those 3 chroot-post
scripts?
Add conditional tests before executing any grub
related command (see if file or command exists first)
As to lsb_release I have no idea why it stops working in 1700* script.
Once these issues are resolved, I can attempt another build
Patrick
2015-05-01 13:00:47 UTC
Good catch!
Only answering the grub issue in this comment. About lsb_release, I need to think some more and do another comment.
side note , skippable chroot-scripts:
chroot-scripts are skippable already. Example code:
https://github.com/Whonix/anon-shared-build-fix-grub/blob/c80a6af12f9559ba119827f8fc4215dc780b7131/usr/lib/anon-dist/chroot-scripts-post.d/85_update_grub#L33
But we won’t chroot-script skipping here. Just wanted to let you know. Perhaps the interface is not the nicest, since you cannot match those by *_grub*
and changing file names in later versions could break it.
skippable package installation :
On Whonix 10
, all packages installed in build-steps.d/1700_install-packages
by pkg-install-maybe
(as opposed of being installed as part of a dependency) are skippable through the whonix_build_script_skip_package_install
environment variable. Example:
whonix_build_script_skip_package_install+=" anon-shared-build-fix-grub "
export whonix_build_script_skip_package_install
Should prevent installation of that package.
side note , export
not in same line:
I prefer not using export in the same line but in it’s own. Otherwise failing commands still return zero rather than non-zero.
export testvar="$(false)" ; echo $?
0
side note , environment variable vs build configuration files:
fixing it in mainline Whonix
Is there some “if Qubes” already available at build-steps.d/1700_install-packages
time? Then, for Whonix 11
, we could apply the fix in mainline Whonix. I.e. “if Qubes, don’t install the anon-shared-build-fix-grub package”.
Patrick
2015-05-01 13:29:25 UTC
nrgaway
2015-05-01 21:43:27 UTC
! In T278#4230, @Patrick wrote:
skippable package installation :
On Whonix 10
, all packages installed in build-steps.d/1700_install-packages
by pkg-install-maybe
(as opposed of being installed as part of a dependency) are skippable through the whonix_build_script_skip_package_install
environment variable. Example:
whonix_build_script_skip_package_install+=" anon-shared-build-fix-grub "
export whonix_build_script_skip_package_install
Should prevent installation of that package.
Okay, so I should add the export for above and remove my hack for the lsb_release
issue.
fixing it in mainline Whonix
Is there some “if Qubes” already available at build-steps.d/1700_install-packages
time? Then, for Whonix 11
, we could apply the fix in mainline Whonix. I.e. “if Qubes, don’t install the anon-shared-build-fix-grub package”.
I will create a new task when I can confirm a successful build that will contain all the work-a-rounds to be able to install Whonix.
Then at some point it may also be of benefit to also remove as much of the special workarounds I needed to do for Qubes on boot. I needed a special startup
file in the past due to whonixcheck
starting too soon, which may be able to be solved with systemd
unit files.
nrgaway
2015-05-02 02:36:41 UTC
@Patrick
Is the 11.0.0.0.7-developers-only
tag supposed to be available? While most submodules are cloning properly, I get this error:
Submodule path 'packages/xchat-improved-privacy': checked out 'afc52a514e0b86bd4fe61344e5e44023d6885890'
Unable to checkout '435741702cb49cb38f410b7597078df7d6659a6c' in submodule path 'packages/whonix-developer-meta-files
Patrick
2015-05-02 04:39:18 UTC
! In T278#4255, @nrgaway wrote:
@Patrick
Is the 11.0.0.0.7-developers-only
tag supposed to be available? While most submodules are cloning properly, I get this error:
Submodule path 'packages/xchat-improved-privacy': checked out 'afc52a514e0b86bd4fe61344e5e44023d6885890'
Unable to checkout '435741702cb49cb38f410b7597078df7d6659a6c' in submodule path 'packages/whonix-developer-meta-files
Fixed.
! In T278#4245, @nrgaway wrote:
! In T278#4230, @Patrick wrote:
skippable package installation :
On Whonix 10
, all packages installed in build-steps.d/1700_install-packages
by pkg-install-maybe
(as opposed of being installed as part of a dependency) are skippable through the whonix_build_script_skip_package_install
environment variable. Example:
whonix_build_script_skip_package_install+=" anon-shared-build-fix-grub "
export whonix_build_script_skip_package_install
Should prevent installation of that package.
Okay, so I should add the export for above and remove my hack for the lsb_release
issue.
Not necessarily. That would work but only be a workaround.
For a real fix - that I would enjoy implementing - you perhaps missed my question… Let me restate.
Is there some “if Qubes” already available at build-steps.d/1700_install-packages time? Then, for Whonix 11, we could apply the fix in mainline Whonix. I.e. “if Qubes, don’t install the anon-shared-build-fix-grub package”.
fixing it in mainline Whonix
Is there some “if Qubes” already available at build-steps.d/1700_install-packages
time? Then, for Whonix 11
, we could apply the fix in mainline Whonix. I.e. “if Qubes, don’t install the anon-shared-build-fix-grub package”.
I will create a new task when I can confirm a successful build that will contain all the work-a-rounds to be able to install Whonix.
Okay, great.
Then at some point it may also be of benefit to also remove as much of the special workarounds I needed to do for Qubes on boot. I needed a special startup
file in the past due to whonixcheck
starting too soon, which may be able to be solved with systemd
unit files.
Yes. I am most definitively interested to remove as much workarounds as possible.
nrgaway
2015-05-16 04:47:30 UTC