This is a big and thorough PR. I am happy with this work 
1 Like
Looks like building from a tag got closer (no --remote-derivative-packages on tag builds)
1 Like
I see the new error. I forgot I need to add 127.0.0.1 host localhost in /etc/hosts to satisfy the scripts. Rebuilding on a new branch. Hopefully this should work
1 Like
--remote-derivative-packages build for commit is still failing. Testing a demo tag build now though to see if that one works 
/home/ansible/derivative-maker/packages/kicksecure/genmkfile/usr/share/genmkfile/make-helper-one.bsh: INFO: debdist
/home/ansible/derivative-maker/packages/kicksecure/genmkfile/usr/share/genmkfile/make-helper-one.bsh: INFO: debdsc
dpkg-source: info: using source format '3.0 (custom)'
dpkg-source: info: building developer-meta-files in developer-meta-files_22.1-1.dsc
/home/ansible/derivative-maker/packages/kicksecure/genmkfile/usr/share/genmkfile/make-helper-one.bsh: INFO: deb-pkg-build
E: cannot canonicalize filename /var/cache/pbuilder/base.cow_amd64, does not exist
++ errorhandlergeneral ERR
++ last_failed_exit_code=1
++ last_failed_bash_command='"$source_code_folder_dist/packages/kicksecure/genmkfile/usr/bin/genmkfile" deb-pkg'
++ output_cmd_set
++ '[' -o xtrace ']'
++ output_cmd=true
++ true 'INFO: Middle of function errorhandlergeneral of ././build-steps.d/1140_local-dependencies.'
++ errorhandlerprocessshared ERR
++ last_script=././build-steps.d/1140_local-dependencies
++ trap_signal_type_previous=
1 Like
@Patrick this PR fixes the most recent issue and is ready to merge
Currently, building full VMs from tags without remote derivative packages is working, and builds are passing
But the issue mentioned above still persists for remote derivative paackage builds from commit
1 Like
Merged. Thank you!
The build worked locally for me:
I didn’t know there’s such a requirement.
Will investigate.
A look at these logs from your developer tag shows the error 
No idea why, but adding that line to /etc/hosts/ fixes things 
https://github.com/Whonix/derivative-maker/actions/runs/3558225207
1 Like
To clarify, the build only is breaking when I set --remote-derivative-packages true
Full build is fine
1 Like
Ok, I understand the issue now.
- true ‘INFO: build_remote_derivative_pkgs is set to true, skipping cowbuilder setup.’
But the 1140_local-dependencies step uses cowbuilder by default. Will fix ASP.
1 Like
Yay!
@Patrick I am getting very close. WATS runner script is executing, but I am running in to some weird failures with apt packages not downloading on a fresh machine. Do the User Agreements have to be completed before the machine will connect to the internet? If so, I am going to have to get orca working.
Would be nice if there was a way to build the OS without the user agreements or a flag to auto-accept them
Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
Err:3 tor+https://deb.debian.org/debian bullseye-updates InRelease
Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
Err:4 tor+https://deb.debian.org/debian-security bullseye-security InRelease
Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
Err:5 tor+https://deb.debian.org/debian bullseye-backports InRelease
Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
Reading package lists...
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye/InRelease Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye-updates/InRelease Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
E: Failed to fetch tor+https://deb.debian.org/debian-security/dists/bullseye-security/InRelease Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye-backports/InRelease Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
E: Failed to fetch tor+https://fasttrack.debian.net/debian/dists/bullseye-fasttrack/InRelease Read error - read (104: Connection reset by peer) Reading the greet back from SOCKS proxy socks5h://127.0.0.1:9050 failed [IP: 127.0.0.1 9050]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package git-all
E: Unable to locate package python3-behave
E: Unable to locate package python3-pip
E: Unable to locate package python3-pyatspi
E: Unable to locate package python3-dogtail
1 Like
The exact reply is “no”. Internet connectivity isn’t restricted depending on the reply to whonixsetup disclaimer wizard. But what’s required is enabling Tor connectivity. (Setting DisableNetwork 0.) Users usually do that using ACW. (Anon Connection Wizard - Whonix)
In the far future it would be good to have a test for whonixsetup and ACW but for now it will be easiest to complete whonixsetup / ACW using command line tools. [1]
orca?
Why orca?
[1] But indeed on that wiki page something is documented which isn’t documented elsewhere yet. That is:
easo
which automates whonixsetup and ACW acceptance. However, easo runs dsudo easyorca-root and that runs sudo setup-dist-noninteractive 1.
So all is hopefully nicely compartmentalized and there’s already support for headless usage of Whonix.
So in summary, either should work:
A)
easo
B)
dsudo setup-dist-noninteractive 1
Not sure which one is best? I guess easo isn’t bad so we get a free automated test for that feature however with a small overhead for orca. Or dsudo setup-dist-noninteractive 1 which should do the same but without orca.
Please let me know if there’s any issues with the automation / headless way to use Whonix. I’ll fix ASP.
Another thing coming to mind is, Tor might need a while until it’s really functional. And since packages are downloaded over Tor, apt-get update might fail. To avoid this, sleeping 3 to 6 minutes might be required for this to work reliable.
I am also happy to install the dependencies of wats by default so this step won’t be needed.
git-all → git
sufficient?
Installing git by default doesn’t seem great but I’d also be happy and eager to package wats and install it by default. Any issues with that such as how it’s updated? If so, I might have a solution for that in mind too.
I changed to git-all after git didnt work, but am changing it back to the git package now that I realize it doesn’t have to do with package name.
I think not packaging dependencies on Whonix is best. A normal user does not need dogtail or behave testing packages by default. Also I think it is important to verify that the apt functionality is in tact, and downloading a package is a good way to see that.
Regarding orca vs dsudo, I will try the dsudo route first to get it functional. I think having running WATS is more important than getting the additional orca stuff for now.
Lastly, I think it is good not to package WaTS on the Whonix VMs. This has been a bit more labor intensive, but IMO a better long term solution. WATS connects to the internet and runs Tor browser tests, so grabbing the latest master branch of WATS via Tor is a small thing.
Thank you for your patience
1 Like
Ooops. I committed directly because I didn’t know I had write access to your repository. And it’s untested. (Syntax was checked, was OK.)
echo changeme | sudo -S apt-get update -q
sudo apt-get install git python3-behave python3-pip python3-pyatspi python3-dogtail -yq --no-install-recommends
This seems to be likely that this would fail. Depending on sudo timeout.
One time the password is piped to sudo. The other time not. This can lead to a fail. But I have an idea to fix it. Going to commit.
NBD
Also, regarding this
I already put that functionality in the branch
1 Like
(dsudo)
I am not sure that is the cleanest way to solve it. A command dsudo sudo-passwordless could be invented to disable sudo password prompt the the CI environment.
1 Like
I tested it manually and it passed
1 Like
Another one just coming to mind: