I tried hardcoding the tag 16.0.5.3 to be built, and ran in to the same errors. I ultimately went ahead and built it with the latest commit because this is the way it should function in the future. Attached is a log of the build errors
Log details
############################################################
ERROR in ././build-steps.d/1200_create-debian-packages detected!
dist_build_version: f69a4c663d6dfa1105c145770b2ef2aea8704214
dist_build_error_counter: 1
benchmark: 00:00:00
last_failed_exit_code: 254
trap_signal_type_previous: unset
trap_signal_type_last : ERR
process_backtrace_result:
1: : init
2: : sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
3: : sshd: ansible [priv]
4: : sshd: ansible@pts/4
5: : /bin/sh -c /usr/bin/python3 /home/ansible/.ansible/tmp/ansible-tmp-1660320029.2783656-1983-3611384331708/AnsiballZ_command.py && sleep 0
6: : /usr/bin/python3 /home/ansible/.ansible/tmp/ansible-tmp-1660320029.2783656-1983-3611384331708/AnsiballZ_command.py
7: : /bin/sh -c /home/ansible/derivative-maker/derivative-maker --flavor whonix-gateway-xfce --target virtualbox --build >> /home/ansible/build.log 2>&1
8: : /bin/bash /home/ansible/derivative-maker/derivative-maker --flavor whonix-gateway-xfce --target virtualbox --build
9: : /bin/bash ././build-steps.d/1200_create-debian-packages
function_trace_result:
main (line number: 406)
build_run_function (line number: 35)
main (line number: 402)
build_run_function (line number: 35)
create-debian-packages (line number: 392)
build_run_function (line number: 35)
create_derivative_distribution_debian_packages (line number: 354)
errorhandlergeneral (line number: 380)
errorhandlerprocessshared (line number: 209)
last_failed_bash_command: "$source_code_folder_dist/packages/kicksecure/genmkfile/usr/bin/genmkfile" reprepro-remove
############################################################
'
++ unset error_reason
++ '[' ERR = INT ']'
++ '[' ERR = TERM ']'
++ '[' ERR = ERR ']'
++ '[' '!' 0 = 0 ']'
++ true 'INFO: dist_build_auto_retry set to 0 (--retry-max). No auto retry.'
++ unset dist_build_auto_retry_counter
++ true
++ ignore_error=false
++ answer=
++ '[' ERR = ERR ']'
++ '[' '' = true ']'
++ '[' -t 0 ']'
++ true 'INFO: stdin connected to terminal, using interactive error handler.'
++ true 'ERROR in ././build-steps.d/1200_create-debian-packages detected!
Please have a look above (the block within ###...).
- Please enter c and press enter to ignore the error and continue building. (Recommended against!)
- Please press r and enter to retry.
- Please press s and enter to open an chroot interactive shell.
- Please press a and enter to abort.'
++ read -p 'Answer? ' answer
Answer?
I would upload the full log file for posterity, but it seems as though I am unable to upload a txt file on this discourse instance. @Patrick any chance you could help with this?
Next Steps
Once the builds successfully work, we need to put them somewhereā¦probably as a github actions artifact. That said, they may be too large and this might require some sort of s3 bucket or remote place to push them.
Set up the build to load the OVA in to virtual box on the remote server, so @Patrick can VNC in to the server and test the running VMs after build
Longer term goals
Run WATS testing suite automatically on the new VMs
For any build success/failure, would it be possible to post that as a comment to github, gitlab or somewhere? Perhaps post the log? That would be immensely helpful.
Maybe the log is too big. Could be posted here too when using code tags. https://www.kicksecure.com/wiki/Forum_Best_Practices#Code_Tags
Otherwise some paste page.
But no log of broken build required. Current git master is broken indeed. Will be fixed soonish. Atm thereās been a lot of wiki work so it got delayed. Why the latest stable tag is broken I donāt know but instead of investigating this, Iāll prioritize fixing getting a newer tag that is functional out.
Currently the build error handler being interactive (asking for what to do on stdin) is an issue?
Since itās on a server and non-non-interactive the good news is the build script has a non-interactive feature. By setting environment variable dist_build_non_interactive=true the question can be avoided and the build would just error out.
In any case, whether the build succeeded (exit code 0) or failed (non-zero exit code), just post a new ticket on github (or gitlab or somewhere) with the result? Perhaps the title would say ābuild failedā or ābuild succeededā and something else useful such as the nearest tag (describe --always --abbrev=10000000000000).
Size could be an issue indeed.
The builds created on a remote server would not be redistributed to users. These are only created locally for better security.
A CI server however is very useful since then build failures would be promptly reported (and therefore fixed much, much sooner).
Auto testing the images using WATS sounds also very interesting.
I will fix the dist_build_non_interactive variable, that will make it where github actions will notify a failed build, and you can see it quickly.
Normally, do you test things manually? If I went ahead and loaded the successful builds in to virtualbox on the VPS, could you VNC and test? Would that be useful to you?
Also, when it comes time for you to test this functionality let me know and I will give you the necessary environment variables for the derivative-maker repo settings
Also a succeeded build notification somewhere would be good. Could be a comment to the same ticket (hardcoded ticket number). Otherwise Iād be wondering if just the server is broken, script is broken versus the build really having succeeded.
@Patrick I am going to do a few small clean up things over the next couple of days, but otherwise the work I am able to do on this project is blocked by the breaking build
once the builds succeed, my next steps are:
autoload the OVAās on the runner
set up the server where you can VNC in and test out the builds
push the OVAs to a storage bucket where you can download tagged images with the commit SHA in the names, so you can download them and work locally if desired
After that is complete, I will merge the commit. We can work together to get it setup for derivative-maker repo, then we can figure out how to make the WATS test run via the pipeline as well.
This unfortunately broke during a major refactoring. But anyhow.
Git tag 16.0.6.5-developers-only is hopefully fixed. Build is already beyond the issue which you experienced. Will post again when the build successfully completed.
Small request, since we are automating things to run without user interaction, can set it where dist_build_non_interactive avoids this logic:
INFO: Script running as as non-root, ok.
INFO: Running 'sudo --non-interactive --validate' to test if sudo password entry prompt is needed...
sudo: a password is required
INFO: Going to run 'sudo --validate' to prompt for password...
INFO: Please enter sudo password.
I have it set up where the ansible user on the VPS doesnt need to enter any sudo password to run the necessary commands, but --validate asks for a password regardless
At time of writing, Qubes uses passwordless sudo by default. sudo --validate does not show a password prompt for me. So I am surprised that doesnāt work yet.
Not sure yet how this could be avoided. Certainly a command line parameter or environment variable could be added as a last resort but before going to such lengths would be good if thereās a cleaner solution.
