[HOME] [DOWNLOAD] [DOCS] [BLOG] [SUPPORT] [TIPS] [ISSUES] [Priority Support]

Long Wiki Edits Thread


#41

After much pain and suffering, I have fixed/merged both of these. :slight_smile:

Basically, all the stuff was merged into the ‘Why use Qubes over other Virtualizers’ entry (plus a shit load more bullet points), and I just cut the Rutkowska template back to one paragraph re: summary outcome of her research.

So, it all looks nice and thorough now.

Do you mean just note a step that the user should recreate sys-whonix and anon-whonix AppVMs after applying the new kernelopts with AppArmor initialized?

If so, easy enough.

BTW - that is not my recollection when I did it on my system i.e. I thought that it worked okay with pre-existing AppVMs, since the tests resulting in ouput ‘0’ (for existance of AppArmor framework) came up trumps from memory. So, I do actually wonder we have this right…


#42

There is no reason to recreate. Just set the kernel options.

Just now tested. Changing kernel options for whonix-ws does not modify kernel options of anon-whonix. (Checked with qvm-prefs -l anon-whonix kernelopts and in VM with sudo aa-status.)


#43

Template:Tor_Browser_Remove_Proxy_Setting

  • Removed all the text around Tor Button ‘Open Network’ feature, since it doesn’t exist in its redesign (Edit: correction; I’ve edited that text and put it back - see below)
  • For the start-tor-browser Method, I assumed you meant that “export TOR_TRANSPROXY=1” is entered as a line of text within the start-tor-browser file (with save and exit), but the text wasn’t clear in the original entry.

Template:Qubes_AppArmor

I must admit I’m very confused now. :slight_smile:

Do you mind cutting and pasting from that entry exactly what is wrong and tell me what needs to be deleted/added re: this process.

As I understood it, we:

  1. Did all that stuff in dom0 (as currently exists in the entry) to show AppArmor is part of new kernel parameters for whonix-ws and whonix-gw (TemplateVMs)

  2. Magically, newly created TemplatedBasedVMs (AppVMs) based on whonix-ws and whonix-gw (ie anon-whonix and sys-whonix) would inherit the same kernelopts, since according to Qubes:

“Newly created TemplateBasedVMs inherit the kernelopts setting of their TemplateVM.”

Thus my reasoning was a) since we already specified whonix-ws (TemplateVM) and whonix-gw (TemplateVM) in the dom0 steps b) sudo aa-status should output “0” for both (freshly created) instances of appVMs.

But you’re saying that doesn’t work?

So, if AppVMs don’t inherit this value when created after dom0 steps, and you’re saying we don’t recreate AppVMs based on modified TemplateVMs, we should then set kernelopts directly to the AppVMs like so (?):

qvm-prefs -s anon-whonix kernelopts “nopat apparmor=1 security=apparmor”

&

qvm-prefs -s sys-whonix kernelopts “nopat apparmor=1 security=apparmor”

Is that right? If not, then I’m completely stumped and I have no idea how to proceed to fix this entry :wink:


#44

Correction: moved all the Tor Button stuff back in (edited) because I realized that Whonix removed that setting, not The Tor Project. :blush: As follows:

Ignore Tor Button’s Open Network Settings

Whonix has disabled the “Open Network Settings” menu option in Tor Button. See the footnote if you want to know why.

Footnote ->

Networking settings can be changed inside Tor, when using the regular Tor Browser Bundle from The Tor Project (without Whonix), via the “Open Network Settings” menu option. It has the same effect as editing Tor’s config file torrc. In Whonix, the environment variable export TOR_NO_DISPLAY_NETWORK_SETTINGS=1 has been [https://github.com/Whonix/anon-ws-disable-stacked-tor/blob/master/usr/lib/anon-ws-disable-stacked-tor/torbrowser.sh set] to disable the “TorButton” -> “Open Network Settings…” menu item. It is not useful and confusing to have in the workstation, because:

  • In Whonix, there is only limited access to Tor’s control port (see [[Dev/CPFP]] for more information); and
  • For security reasons, Tor must be manually configured in /etc/tor/torrc on the Whonix-Gateway, and not from the Whonix-Workstation (see also [[Features#VPN_.2F_Tunnel_support|VPN/Tunnel support]] for more information).

#45

sys-whonix and anon-whonix might be created by Qubes installer. By that time, the kernelopts do not include AppArmor settings yet. So “these newly created AppVMs” will not have AppArmor kernelopts enabled. Does that make sense?


#46

1) Yes. I was a bit slow on the uptake there. Thanks for signing all that stuff off.

I was confused because I didn’t remember creating new AppVMs in my instance (when I’ve had AppArmor running for ages), but then I remembered recreating the anon-ws etc later on after I did those steps. ha ha

I’ll fix that up soon.

2) I updated the http://kkkkkkkkkk63ava6.onion/wiki/Grsecurity#How-To:_Qubes-Whonix entry to reflect the latest grsec kernel which is available now -> 4.9.20

Coldhak instructions are still the same as before (I checked).

But this begs the question, how does one update their grsec kernel in Qubes Debian-8 TemplateVM, without going through all those steps again? i.e. check out the latest kernel version, build it and ditch the older version?

Or just easier to create a whole new TemplateVM and do the long process again? I’m under no illusion that it is that easy in Qubes, but I’ve not seen this referenced anywhere that I remember.

Any ideas? We should probably have an entry in there (eventually) and I’m happy to guinea pig it, with a few pointers.

I’d be interested if you’ve had any luck using a Debian-8 grsec kernel AppVM for networking and firewall too.

I’ve not tried it, but the forums suggest that only an additional module or 2 is required (during build?) to have it compatible with networking.

3) If anyone has tried that MirageOS unikernel FW, that would be interesting too, and one day (6 months later) I might get around to trying it and documenting it here somewhere.

But right now, I think most of 2017 is gonna be taken up with this long wiki, based on the current rate of progress (which is slow).

But, I think it is coming into shape bit by bit.


#47

I significantly changed/updated the Tox template, which was quite (3 years) out of date. Most “planned” features are now actually implemented, depending on the client in use.

http://kkkkkkkkkk63ava6.onion/wiki/Template:Tox

Also, it is much easier to install now, with no key verification etc steps required I gather, since we are downloading it straight from an opensuse.org repository.

I just focused on installing the main qTox graphical client in the steps.

If the user wants uTox (lightweight dependencies) or other supported Linux clients, I left it for them to follow up the client descriptions and install steps via the links in the entry (it’s pretty simple).

But do check that the steps are safe as I have edited it. I suppose we can auto-trust repos straight off the opensuse website…

Cheers


#48

grsecurity upgrades… Untested… Does this help?

  1. Clone your Debian-8 TemplateVM [skip]
  1. Increase the maximum storage size of the Debian-8 TemplateVM [skip]
  1. Edit your sources.list [skip]
  1. Install dom0 dependencies. [skip]

In dom0, run:

sudo qubes-dom0-update grub2-xen [skip]

  1. Install Debian dependencies. [skip]

In the Debian TemplateVM, run:

Building the grsec Coldkernel

  1. Clone and verify the coldkernel build scripts.

wget “https://coldhak.ca/coldhak/keys/coldhak.asc” -O coldhak.asc [skip unless signing key changed]
gpg --import coldhak.asc [skip unless signing key changed]
git clone https://github.com/coldhakca/coldkernel [don’t clone]
cd coldkernel
[new command] git fetch
git verify-tag coldkernel-0.9a-4.9.20 [repeat with new version]
git checkout tags/coldkernel-0.9a-4.9.20 [repeat with new version]

make qubes-guest [repeat]

https://coldhak.ca/assets/img/blog/coldkernel_pt1/build.png
Installing the grsec Coldkernel

Post-build, in the Debian Template VM run:

wget https://grsecurity.net/paxctld/paxctld_1.2.1-1_amd64.{deb,deb.sig} [repeat if there was an update]
gpg --homedir=.gnupg --verify paxctld_1.2.1-1_amd64.{deb.sig,deb} [repeat if there was an update]
sudo dpkg -i paxctld_1.2.1-1_amd64.deb [repeat if there was an update]
sudo make install-deb [repeat]
sudo cp paxctld.conf /etc/paxctld.conf [repeat]
sudo paxctld -d [repeat?]
sudo systemctl enable paxctld [can be skipped]
sudo mkdir /boot/grub [can be skipped]
sudo update-grub2 [repeat]
sudo shutdown -h now [repeat]

Post-install TemplateVM Configuration

  1. Change the Debian-8 TemplateVM kernel. [skip]
  1. Check the Debian-8 TemplateVM is functional. [repeat]
  1. Set default grsec special groups. [skip unless there is some new group]
  1. Install paxtest and check that Grsecurity is running.

In the Debian TemplateVM, run:

sudo apt-get install paxtest [skip]
paxtest blackhat [repeat]


#49

Thanks.

Works up until this point:

sudo make install-deb

Before getting errors like:

Error! Bad return status for module build on kernel: 4.9.20-coldkernel-grsec-1 (x86_64)
Consult /var/lib/dkms/u2mfn/3.2.3/build/make.log for more information.

The reason is that the u2mfn version is too out of date in (stable repos) Qubes. Thus, if people want to use a 4.9 kernel, this will first need to upgrade to the testing repos in Qubes (needed version updloaded on 23 March).

https://github.com/coldhakca/coldkernel/issues/55

So, I’ll upgrade to testing and see if it completes correctly and then fix up the grsec entry for 4.9 kernels & how to push an update to an existing grsec template…


#50

OK - I got the 4.9.20 grsec template built and booting (general logs look okay), but then the VM light goes from green to yellow. :unamused:

It doesn’t allow me to do post-install tinkering (setting grsec special groups) or open any applications e.g. terminal.

The only error I can see is from Qubes VM Manager:

ErrorHandler: BadAccess (attempt to access private resource denied)
Major opcode: 130 (MIT-SHM)
Minor opcode: 1 (X_ShmAttach)
ResourceID: 0x219
Failed serial number: 49
Current serial number: 50

So, I probably stuffed something up, but this is major progress :slight_smile:

Really, we need at this point in the guide -> http://kkkkkkkkkk63ava6.onion/wiki/Grsecurity#Debian_TemplateVM a step 6 saying:

Additional Debian Dependencies

Note: If attempting to build a Linux 4.9 kernel, you require qubes-kernel-vm-support 3.2.4+deb8u1 from the Qubes Debian Testing repository.

First, temporarily enable the Debian-TemplateVM testing repository following the steps here: https://www.qubes-os.org/doc/software-update-vm/

Then update your package lists:

sudo apt-get update

And install the latest kernel support:

sudo apt-get install qubes-kernel-vm-support

Afterwards, set your default repository back to the stable Qubes version.

We should probably also note that if people get this error message when running sudo update grub-2, it can be safely ignored ->

When running sudo update-grub2, you can safely ignore this error message:

grub2-probe: error: cannot find a GRUB drive for /dev/mapper/dmroot. Check your device.map

See: https://github.com/QubesOS/qubes-doc/blob/master/configuration/managing-vm-kernel.md

But, I don’t know if I can be bothered with the 4.9 kernel stuffing around right now… it’s probably just easier to note that 4.8 versions work with the instructions and wait for the 4.9 series to stabilize before building (unless you have any ideas on the above error).

I’m sure someone over on the Qubes forums could probably answer this problem, but I’m not registering on filthy Google forums to find out. Life’s too short.


#51

re: tox. What is the source of that link http://download.opensuse.org/repositories/home:/antonbatenev:/tox/Debian_8.0? Looks a bit weird. A home folder? Who owns the account? Really the developer of tox? Could you add a reference please?


#52

All your contributions are appreciated! I wasn’t aware so many issues were still left. It’s awesome to see it being perfectionated.

You might like to get a backup since you also put a lot time into this.

Could you please have a look at https://www.whonix.org/wiki/WhonixBackupScript and see if it works for you so we have a few people actually doing backups “of Whonix”? Does initial backup work and if keeping it up to date as well?


https://www.whonix.org/wiki/Template:Tunnel_Support
[[Tunnels/Introduction|Using Tunnels with Whonix]]

vs new

[[Tunnels/Introduction|Using Tunnels in Whonix]]

It’s not just “in”. Since tunnels can be configured elsewhere, such as on the host in Non-Qubes-Whonix or in other VMs in Qubes-Whonix. Do you know a better adverb than in if withis wrong?


https://www.whonix.org/wiki/Grsecurity

Do we have any proof for this fingerprint 1073 B61B 69CB 0444 33B6 4F7B 0B1F 9321 DE32 FEBB? New fingerprint?


Improved https://www.whonix.org/wiki/Template:Uwt_wrappers_deactivate_all_permanently a bit. Please check.


#53

No problem!

Re: Template:Tunnel_Support -> OK. Just leave it as “Using Tunnels with Whonix” then, since it otherwise changes the meaning.

Okay, I better list my follow up tasks from this editing spree so I don’t miss anything:

TO DO:

1) Check Tox repository source (who is it? developer? etc). BTW, that repo comes straight from the Tox website (download section) for “Debian”.

2) Finish finer edits of Qubes:Apparmor Template

3) grsecurity, add:

a) Patrick’s grsec kernel update instructions (since I’ve tested and they work)
b) Note dependencies for 4.9 kernel. Note that 4.8 kernel works with current instructions
c) Check proof of coldhak fingerprint (although that one I added and saw on my system matches the one from the coldhak blog)

4) Check Qubes forums for grsec blocking / failing message, since this probably relates to overly strict settings in Pax etc

5) Test WhonixBOT wiki-backup works

6) Check Template:Uwt_wrapper_deactivate_all_permanently edits

BTW are you worried that Whonix wiki [and backup?] could be lost from the servers simultaneously - from an accident or malicious actors?. How big is it i.e. need a very large VM set aside?


#54

1) Tox Template -> Fixed.

Added this footnote:

This repository is directly referenced on the Tox Download webpage, see: https://software.opensuse.org/download.html?project=home%3Aantonbatenev%3Atox&package=qtox Anton Batenev is a Tox developer.

FYI, here he is on Github, you can see long term work on core Tox libraries etc -> https://github.com/abbat?tab=overview&from=2016-12-01&to=2016-12-31

2) Qubes: AppArmor Template -> Fixed.

Added explicit steps re: deleting old AppVMs and creating new ones (used the templates for this) and edited the text to make it all very clear.

Moving on…


#55

3) Grsecurity#How-To:_Qubes-Whonix -> Fixed.

  • Added note about dependencies for 4.9 Kernel
  • Noted one can ignore grub error messages if they appear at sudo update-grub2 step
  • Added entry for upgrading kernel builds
  • Noted the signing key output should be checked against the one found on the coldhak blog (see below for query)

Query: Should we be doing another step using one of the keys at the link below instead for ‘verification’ checks?

[https://github.com/coldhakca/coldkernel/tree/master/keys]

The folder only lists torvalds.asc (for Linux kernels), spender.asc (for grsec patches) & I have no idea who greg.asc is and whether that matches the “coldhak signing key” or not. How do we check at the terminal?

The Tor Project instructions just refer to a website and checking the key matches for TBB, so I just put in the wiki the matching .png (coldhak website) in the interim. Is that safe enough or not?

4) Re: Sending Qubes-user forum a message about the 4.9 grsec kernel failure ->

I was going to send an email to the Qubes-Users forums about the 4.9 kernel not working with grsec (that error far above), but I see SIGAINT has closed down. So, my old throw-away email can’t be used to send this query to:

qubes-users@googlegroups.com

I’ve found another provider for throw-away email, but they don’t allow posting for 24-28 hours when you register by Tor. So, we can wait a day or 2 to post, of if you’re keen, you might drop this into Qubes-User forums so we might get some advice earlier and I can fix up the wiki grsecurity entry if necessary(?). Anyway, this is what can be posted either way:

Greetings,

I am currently using Qubes 3.2 and have had success to date with running the 4.8 grsec kernel series (coldkernel) with Debian-8 AppVMs following the steps / advice outlined on the coldhak blog and github account.

I have recently tried to apply the 4.9.20 upgraded kernel to the Debian-8 TemplateVM and hit some problems.

I have followed the advice to install the latest qubes-kernel-vm-support package from the Qubes testing repository (for the Debian-8 TemplateVM) and avoided the error messages around “Bad return status for module build.” [https://github.com/coldhakca/coldkernel/issues/55] [https://github.com/QubesOS/qubes-issues/issues/2691]

The upgraded kernel successfully builds and the TemplateVM boots. However, the TemplateVM state light soon shifts from green to yellow. The qrexec.log and console.log look okay (no obvious error messages), but the guid.log shows a new cryptic error message I’ve never seen before:

ErrorHandler: BadAccess (attempt to access private resource denied)
Major opcode: 130 (MIT-SHM)
Minor opcode: 1 (X_ShmAttach)
ResourceID: 0x219
Failed serial number: 49
Current serial number: 50

Any attempts to run applications fail e.g. terminal. So, grsec groups can’t be set, paxtest can’t be run, and obviously it’s not functional, so there is no point creating a new AppVM based on it.

Can anyone who has the 4.9 grsec kernel up and running provide any advice on how to resolve this problem?

Regards

6) Template:Uwt_wrapper_deactivate_all_permanently -> Fixed

I think you meant that these steps will deactive all wrappers AND remove stream isolation (it wasn’t clear to me). If that’s not the case, I’ll re-edit it. But this is what I came up (just changed the first paragraph):

The following instructions permanently deactivate all uwt wrappers and remove stream isolation for [[Stream_Isolation#By_uwt_wrapper|uwt wrapped applications]] system-wide. Consequently, all uwt wrapped applications revert to the default system networking configuration.

Moving on to the last point (5) re: Whonix Bot testing…


#56

re Qubes AppArmor:

AppVMs based on Whonix templates - anon-whonix and sys-whonix

-> Please fix to whonix-gw / whonix-ws.

You need to delete the old AppVMs and recreate anon-whonix and sys-whonix for the changes to take effect.

There is no need to delete them. Just set the kernel settings.

Imagine there is a text file where Qubes stores these settings in dom0. One text file per VM. When a new TemplateBasedVM is created, it inherits defaults from the TemplateVM. Changing TemplateVM kernel settings will only change the settings file for the TemplateVM and not touch TemplateBasedVMs. When Qubes dom0 starts a VM, it looks into the text file on which kernel settings to pass during boot.

(It’s actually not a text, but xml file in /var/lib/qubes, I think.)


re tox:

Could you please add gpg fingerprint verification instructions?

Adding the repository key is not optional. Without it there would be an apt-get gpg verification warning that really should not be ignored.

Could you please use gpg --with-fingerprint as used on for example https://www.whonix.org/wiki/YaCy?


re todo: Feel free to do that here or to open tickets at phabricator.whonix.org. Whatever works best for you.


It’s probably paranoid but the more people have backups the less likely it can be purged from history.

2 GB should suffice for now.


We root trust in coldkernel to not ship us malicious keys there. That’s sufficient.

Verification of the keys in that folder is less user documentation, but more the task of a source code auditor, who would verify that among the rest of coldkernel source code. Just verifying these keys by other people and not auditing the rest of the source code would not buy anything.


#57

grsecurity message posted to qubes-users. Hopefully I picked a good subject.

https://groups.google.com/forum/#!topic/qubes-users/2X8wi5XebJc


#58

LOL @ Qubes Template. :grin:

  1. I feel like I’m back in Calculus class with this Qubes template!

I’m gonna test this again with fresh templates with default kernelopts, just for my interest first. Then edit for the last time, and if that isn’t right, I might have to defer that editing to the expert (you)

  1. Re: ‘stable’ version of qTox, I’ll check if there are keys about and instructions (I think there may not be, unless for nightly versions). If that’s the case, should we be referring people to nightly versions of qtox?

Thanks for posting that message to Qubes forums. If it doesn’t get a reply, I might try a clean templateVM to install the 4.9.20 grsec kernel in, instead of the testing ‘upgrade’ template I used. I could have stuffed up something, or missed a step possibly to explain why I don’t have permissions to open anything. Very likely with the large number of commands entered at the terminal.


#59

This is my detective work with qTox.

Anton Batenev is the main man for signing off Linux OBS repositories (whatever OBS is), see here (“abbat” is his callsign on github):

Linux OBS repositories, managed by abbat:

3EB5 027B 3CD8 D7CA AC30 EB6B F2AA 0B1E 5EF8 303B

This key is also confirmed by this github issue (where the key had expired):

My fault, key really expired. You have to update key:

$ wget -O - http://download.opensuse.org/repositories/home:antonbatenev:tox/Debian_Stretch/Release.key
| sudo apt-key add -

or

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0xF2AA0B1E5EF8303B

Nowhere can I find reference to and standalone packages/binaries and .asc signature files, so we can run a gpg --verify step.

So sure we can:

  • Download Anton Batenev’s key which is used for signing the Linux qtox builds

gpg --keyserver keyserver.ubuntu.com --recv-keys “3EB5 027B 3CD8 D7CA AC30 EB6B F2AA 0B1E 5EF8 303B”

  • Check the fingerprint

gpg --fingerprint “3EB5 027B 3CD8 D7CA AC30 EB6B F2AA 0B1E 5EF8 303B”

With the output showing:

pub 2048R/5EF8303B 2014-09-04 [expires: 2019-01-20]
Key fingerprint = 3EB5 027B 3CD8 D7CA AC30 EB6B F2AA 0B1E 5EF8 303B
uid home:antonbatenev OBS Project home:antonbatenev@build.opensuse.org

But I’m not sure how this helps us.

Note: In the current wiki entry, I have taken the download and adding key steps straight off the Tox website. Nightlies for Linux no longer appear available. Ironically Windows has standalone files and sigs, but not Linux.

So, I think we’re just expected to trust Anton’s folder on the opensuse website since he is trusted with all the Linux packaged files. Of course, if somebody interferes with the download over https, I guess we’re screwed.

In other words, perhaps I should just note/change the entry to reflect that the user should trust the repository key step, since it’s the best that can be done in the situation (?).

Test install of qTox

I tested installing qTox, and one has to add the repository key to apt FIRST, or you get a PUB KEY not found error, when running apt-get update.

Oh yeh, all the commands have to be run in root (su) too. My mistake.

I tested in an AppVM and qTox is all starting properly etc. I haven’t used it for comms though, just seeing it booted up okay.

Suggested template edit

I think the correct order for it to work now is (for correct install):

su

wget -nv http://download.opensuse.org/repositories/home:antonbatenev:tox/Debian_8.0/Release.key -O Release.key

apt-key add - < Release.key

echo ‘deb http://download.opensuse.org/repositories/home:/antonbatenev:/tox/Debian_8.0/ /’ > /etc/apt/sources.list.d/qtox.list

apt-get update

apt-get install qtox

exit

If you’re happy with that, I’ll change it accordingly.


#60

I should have done my homework first. That crash in the grsec Debian template appears to be related to known Qubes issues e.g.

Possibly:

[https://github.com/QubesOS/qubes-issues/issues/2617]

This seems to be a bug in our code. Newer Xsevers have a separate thread for input processing. So when we access the window object to get the memory pages with the image data we have a race condition with the main thread if the client changes the Pixmap.

QubesOS/qubes-gui-agent-linux@5ea68d2 should fix this.

And most definitely this:

[https://github.com/QubesOS/qubes-issues/issues/2455]

Qubes-GUID crashed in one AppVM as soon as I started monodevelop the first time. Cannot reproduce this problem either. Error in guid log was:

ErrorHandler: BadAccess (attempt to access private resource denied)
Major opcode: 130 (MIT-SHM)
Minor opcode: 1 (X_ShmAttach)
ResourceID: 0x2000054
Failed serial number: 3670
Current serial number: 3671

So, our advice to daring people using the latest 3 day old grsec kernels from Coldhak should be:

  • Apply the Qubes testing repository for the Debian TemplateVM for cutting edge release kernels

Or similar. This would have solved the module crashing issue during build, plus (probably) this Qubes-GUI error in the first place.

So, I’ll try that and get back to you on the success of the 4.9.20 coldkernel… god it’s a never ending story :wink:

Edit: And 2 hours later from a clean templateVM grsec rebuild with full Qubes testing repo, I now get the:

  • “Cannot execute qrexec-daemon!” (Qubes) error; and also the
  • “Loading initial ramdisk …
    alloc magic is broken at 0x18e21260: 18c481a0” (Coldhak) error in combination.

You already noted the latter one here 9 months ago:

https://github.com/coldhakca/coldkernel/issues/35

Needless to say, between Qubes coding errors somewhere in the graphical system and the very, very alpha state of the latest grsec template, users should not waste their time until a 0.9b version comes out, and Qubes testing updates make it to the stable repos.

That is, stick to the 4.8 grsec series, which I’ve used for months with no problem whatsoever, including combined with Firejail.