Long Wiki Edit Thread

@pano

Colors may be useful? Opinions?

Color relations?

  • Qubes: blue - since the Q logo of Qubes is blue
  • console: black (maybe “more black then black”? can we do bold or something other than block to illustrate console)
  • VirtualBox: ?

Maybe colors for box colors?

Or maybe symbols for Qubes, console and graphical would help?

Some templates that could use such a revision:

2 Likes

That’s outstanding work 0brand!

I wonder whether the method for setting disposableVMs for sys-net and sys-firewall belongs on the Qubes website, since it is Qubes-specific? @adw We could then link to it and add it as an obvious hardening step.

If we don’t hear anything, I think we just add all this text, clearly mark it as experimental / testers-only, and find an appropriate home for it (Advanced Security Guide?).

Also keen for someone to chime in that it is the canonical method and safe - although it looks logical to me. Also, does the anon-ws VM and Tor Browser work fine in a disposableVM in this configuration i.e. disp-sys-net, disp-sys-firewall, disp-anon-whonix? If so, you’re probably the first person to pull it off.

My main worry is that anonymity might be affected somehow given previous (still open) Qubes issues with dvm-templates.

For bonus points or fun (since you like testing), you could attempt customizing the disp-anon-whonix with the alpha sandboxed Tor Browser (that requires Whonix 14) and see it works okay in this combo. That would be another first in the community probably. :slight_smile:

Good idea Patrick. It’s just a case of getting around to it.

If anyone wants to edit those templates, I’ll unprotect them.

PS Community is humming along nicely. I see in different threads: I2P development, VBox .ova shrinking, dispVMs in Qubes, encrypted email etc, new contributors solving issues on phabricator etc. Good stuff.

2 Likes

torjunkie:

I wonder whether the method for setting disposableVMs for sys-net and sys-firewall belongs on the Qubes website, since it is Qubes-specific?

Yes.

fix Qubes documentation edit cumbersomeness · Issue #3629 · QubesOS/qubes-issues · GitHub applies? :slight_smile:

If we don’t hear anything, I think we just add all this text, clearly mark it as experimental / testers-only, and find an appropriate home for it (Advanced Security Guide?).

Yes.

Also this shouldn’t be manual instructions for long. Is there a Qubes
ticket yet to make that the default?

PS Community is humming along nicely. I see in different threads: I2P development, VBox .ova shrinking, dispVMs in Qubes, encrypted email etc, new contributors solving issues on phabricator etc. Good stuff.

:))

Once again, great minds think alike! I was thinking this would be something that should be on the Qubes Docs. I didn’t know how to ask though.

That sounds great!

If one of the qubes devs could take a look at this would be very helpful. If we don’t here back, maybe ask on Qubes mailing list?

Yes! I’ve been using this setup since RC3 or RC4 (can’t remember which) The only problem I have is the disp-anon-whonix will not start after updating my system when I first boot every once in a great while. Requires system reboot for disp-anon-whonix to launch.

Example

Debian and Whonix Templates

sudo apt-get update && sudo apt-get dist-upgrade (all template VMs)

sudo dom0-qubes-updates

qvm-run disp-anon-whonix 'firejail torbrowser'

Seems like the Dispvm starts but no apps can be run?

Very possible

No problem!

BTW disp-sys-usb functioning. Should be finished writing the steps later on today :slight_smile:

1 Like

0brand:

If one of the qubes devs could take a look at this would be very helpful. If we don’t here back, maybe ask on Qubes mailing list?

I wouldn’t recommend wait here. Looks to me they have an even bigger
flood of incoming inquiries than Whonix.

The proper process as far as I understand would be to create a
qubes-issues ticket “document …”. Offering to provide a pull request.
Maybe wait for answers. Only maybe if it looks required. And then to
provide a pull request against qubes-doc on github.

qvm-run disp-anon-whonix 'firejail torbrowser'

For debugging… Does this help?

qvm-run disp-anon-whonix konsole

Does it work that far?

firejail torbrowser

Might give more clues.

Sorry, I replied to torjunkies post without reading through your post first. I just finished reading through fix Qubes documentation edit cumbersomeness · Issue #3629 · QubesOS/qubes-issues · GitHub

@torjunkie I can create the Qubes ticket if you like? I have a lot on my plate so it may take a few day to get this completed.

I know this command works

qvm-run disp-anon-whonix konsole

I can’t remember if I tried running running any apps from it when I was having problems. I will try runnig firejail torbrowser from the konsole next time it happens. Thanks!

2 Likes

We always welcome quality documentation contributions! The procedure for contributing Qubes documentation is explained here:

Basically: Submit a pull request to qubes-doc. We’ll review it and may ask for changes, if necessary. Once it passes review, it’ll be merged and appear on the live website.

I’m afraid you’ll probably never receive a response from the Qubes devs here or on our mailing lists about this. It’s not intentional or malicious; they’re simply so swamped that they probably won’t reply or even see it. Moreover, all of their time spent on reviewing documentation submissions is spent on the ones submitted via the proper channels (which are explained above), so your best bet for getting developer eyes on the document is by submitting it that way.

Yes, Patrick understands. :slightly_smiling_face:

For a documentation contribution, it is not necessary to create an issue in qubes-issues. Instead, you can just submit a pull request as explained on the page linked above.

3 Likes

Awesome! There is still some fine tuning needed before the instruction will be ready. Should not take long to complete. Once the changes are reviewed, I will submit a pull request to qubes-doc

Completely understood. I can well imagine the workload you/they have. I have problems myself just keeping up with Whonix wiki edits in a timely manner. :slight_smile:

Thanks for the quick reply.

And as always, Thanks for all the hard work!

2 Likes

I think it can help, but if you want something really good, then allow the reader to see only the sections he need.

If for example there was a dropdown or two, at the top of the page, instead of all the “If you are using..” boxes throughout the page, using the wiki would be much easier.

1 Like

pano:

I think it can help, but if you want something really good, then allow the reader to see only the sections he need.

If for example there was a dropdown or two, at the top of the page, instead of all the “If you are using..” boxes throughout the page, using the wiki would be much easier.

That would be very good, but we don’t have this skill in Whonix team at
the moment. So not realistic. (We stuck without webmaster, stuck with
mediawiki, let alone such sophisticated web development.)

Instructions for Disposable USB qube are complete. These instructions assume the the service-dvm in the previous instructions would be used as the disp-sys-usb template i.e. template_for_dispvms true

My next step is to do some fine tuning on both sets of DispVM instructions. Get feedback for any changes. Then I will submit a pull request to qubes-docs (if that is OK with everyone)

Also, I know much of the sys-usb info I mentioned is already in the Qubes docs. I wanted to add it here in case anyone wanted to test the instructions. Having all info in the same place makes it easier.

Create a Disposable sys-usb VM

Qubes 4.0 users now have the option of creating a sys-usb Disposable VM. This will provide significant benefits over the default sys-usb VM. Considering that USB devises can pose a serious security risk, the use of a disposable USB qube would mitigate many of the risks associated with attaching a USB. Not only is dom0 afforded the same protections the default USB qube provides from a malicious USB. The Disposable USB qubes’ non-persistent filesystem prevents malware from gaining a persistent foothold in the VMs /home folder.

Users must first create the dvm template which the disposable USB VM is based on (steps 1-3 “Create sys-net and sys-firewall Disposable VMs” chapter)

1. In dom0, create the disposable USB VM

qvm-create -P <pool_name> --template service-dvm --class DispVM --label red disp-sys-usb

2. In dom0, set the virtualization mode to hvm

qvm-prefs disp-sys-usb virt_mode hvm

3. In dom0, set disp-sys-usb NetVM to none

qvm-prefs usb-disp netvm ""

4. In dom0, list all available PCI devices

qvm-pci

5. In dom0, attach the USB controller to the USB VM

Note: the bakend:BDF address will look similar to this dom0:00_1a.0

qvm-pci attach --persistent disp-sys-usb <backined>:<bdf>

6. Hide USB controllers from dom0

Warning: If a USB AEM is being used, all USB controllers should not be hidden from dom0. This would prevent the USB AEM devices from functioning.

Note: If users created a usb qube during qubes install or by using salt, the following steps can be ignored.

If users have created a USB qube manually, there will be a brief period of time during the boot process when dom0 will be exposed to your USB controllers and any attached devices. This is a security risk, since even a brief exposure to a malicious USB device could result in dom0 being compromised. It is recommended users hide (blacklist) all USB controllers from dom0.

GRUB2

In dom0 terminal, open GRUB2 configuration file in a text editor

sudo nano /etc/default/grub

Locate the line that begins with GRUB_CMDLINE_LINUX and append the following text

rd.qubes.hide_all_usb

save and exit

In dom0 , update GRUB2 for the new changes to take affect

grub2-mkconfig -o /boot/grub2/grub.cfg

In dom0, reboot Qubes for the new boot parameters to take affect

sudo reboot

EFI

In dom0, open EFI boot configuration file in a text editor

sudo nano /boot/efi/EFI/qubes/xen.cfg

Locate the line that begins with “kernel=” and append the following text

Note: more than one “kernel=” line may be present

rd.qubes.hide_all_usb

save and exit

Troublshooting

If the USB qube fails to start the likely reason is one of the controllers does not support reset. Quite often the offending contoller is a USB 3.0 device. The following errors would suggest this is the issue.

In dom0, xl dmesg output

(XEN) [VT-D] It’s disallowed to assign 0000:00:1a.0 with shared RMRR at dbe9a000 for Dom19.
(XEN) XEN_DOMCTL_assign_device: assign 0000:00:1a.0 to dom19 failed (-1)

In dom0, when qvm-start disp-sys-usb is run

internal error: Unable to reset PCI device […] no FLR, PM reset or bus reset available

Users have serveral solutions available to restore VM functionality

  • remove the offending controller from the disp-sys-usb VM and try to restart the VM.

  • disable the USB 3.0 in the BIOS.

  • try to force USB 2.0 modes for the USB ports

    In dom0, force USB modes for all USB ports

    lspci -nn | grep USB | cut -d '[' -f3 | cut -d ']' -f1 | xargs -I@ setpci -H1 -d @ d0.l=0

  • set the pci_strictreset option

    In dom0, set the no-strict-reset option

    qvm-pci attach --persistent --option no-strict-reset=true disp-sys-usb <backend>:<bdf>

1 Like

It looks like dispvm-sys-net, dispvm-sys-firewall, and disp-sys-usb have been pushed to the qubes docs. Not by me.

I looked it over and with the method described you are not able to use custom firewall rules set with static DispvVMs . However it is possible If users follow the instructions in this thread. The rule set must be added to the dvm template which the disp-sys-firewall is based on. Obviously firewall rules cannot be added for Whonix due to stream isolation.

This method is a little complicated but it is possible.

1. Create dvm template disp-sys-firewall is based on
2. Create disp-sys-firewall
3. Add custom rule set to dvm template. Note: Use the disp-sys-firewall IP address when creating rule set

1 Like

I haven’t seen one. It would be a nice option at install time - Create disposable sys-net, sys-firewall, and sys-usb VMs.

Or some fancy script that runs all those commands in order so the user doesn’t screw up.

He stole your thunder :slight_smile:

Great work on all of this BTW!

At least it’s done. You could suggest your improvements on github? Easy enough to create an anonymous account there.

Marek is assigned to review it ATM.

Excellent!

Thanks for chiming in Andrew. Appreciated.

I like yours better. If the docs don’t have your suggested improvements and passes Marek’s approval, you are welcome to add all your text to a page here somewhere, as it is more user-friendly and educational which is better for low-intermediate users IMO.

Qubers are more about getting the job done with few beginner tips, which doesn’t do it for everyone :wink:

1 Like

To summarise, I think there is where it’s at.

TO DO

  • Startpage bypass of Tor blocks.
  • 0brand’s dispVM 101 for low-intermediate users. Perhaps a whole new page, with a section in Advanced Security Guide pointing to it (?)
  • Footnote Qubes-Whonix VM protection script as per Patrick suggestion
  • Remaining email fixes (numerous)

Up-for-grabs

  • Proxy steps as per entropy’s info in previous post.
  • One Time Pads
  • Prettying up boxes in documentation perhaps (color)?
  • Template request
  • Debian developer request re that system-d setting that is a risk to anonymity (in another forum post)
1 Like

I have an pseudonymous github account. Don’t use it much though.

I completed the changes needed for qubes-docs and added a few thing that may be of interest the users (i think) :slight_smile:

It would be helpful if i could get a little feedback before the pull request is submitted. So I don’t duplicate all the content already on this tread. Just the introduction and the first 2 sections are posted.

I wanted to kill 2 birds with one stone with this submission.

  • find out if this way of creating DispVMs is sound
  • find out if adding custom scripts, software in the dvm is OK (I use them). This would be great to know for future wiki docs?

Create Custom sys-net, sys-firewall, and sys-usb DispVMs

Users have the option of creating cusomized DispVMs for the sys-net, sys-firewall and USB VMs. These VMs behave much like the default VMs created during Qubes installation with the exception of a non-persistent filesystem. Another similarity shared with the default service VMs is the option to use custom fireall rulesets as will as Qubes VPN scripts. This can be accomplished in spite of the fact that a fresh VM is created each time a DispVM is launched by adding rules or cusum scripts to the dvm template. Users also have the option to set the DispVMs to autostart at system boot in addition to attaching pci devices with the --persisent option. Since both of the aformentioned configuration options are required only at the initial DispVM creation. Using DispVMs in this manner is ideally suited for untrusted VMs which reqire persistent pci devices such as USB VMs and NetVMs.

Note: if the dvm is customized with a VPN or firewall ruleset. A seperate dvm must be created for use by each DispVM. Otherwise, if no custimizations are made to the dvm. Users need only create a singular dvm which can be used as template for all DispVMs

Create the dvm from which the DispVM will be based on

1. In dom0, create the dvm template

qvm-create -P <pool_name> --template <template_name> --class AppVM --label gray service-dvm

2. (Optional) In dvm, add user customizations

Firewall rulestets and Qubes VPN scripts can be added like any other VM   

3. In dom0, set service-dvm as template for disposable VMs

qvm-prefs service-dvm template_for_dispvms true

Create the sys-net DispVM

1. In dom0, create sys-net DispVM based on service-dvm

qvm-create -P <pool_name> --template service-dvm --class DispVM --label red disp-sys-net

2. In dom0, set disp-sys-net virtualizaion mode to hvm

qvm-prefs service-dvm virt_mode hvm

3. In dom0, set disp-sys-net to provide network for other VMs

qvm-prefs disp-sys-net provides_network true

4. In dom0, set disp-sys-net NetVM to none

qvm-prefs net-disp netvm ""

5. In dom0, list all available PCI devices to determine the correct backend:BDF address(es) to assign to disp-sys-net

qvm-pci

6. In dom0, attach the network PCI device(s) to disp-sys-net instructions on finding and assigning pci devices can be found here (link to documention)

qvm-pci attach --persistent net-disp <backend>:<bdf>

7. (recommended) In dom0, set disp-sys-net to start automatically when Qubes boots

qvm-prefs disp-sys-net autostart true

8. (Optional) In dom0, set disp-sys-net as the dom0 time source

qubes-prefs clockvm disp-sys-net

If after reading this you think it should be reverted back to the original content/format please let me know. Maybe bring up the custom scripting in the dvm another time?

2 Likes

Sorry, time poor at the minute. I’ll have a look shortly.

Anyhow, adw’s recent doc effort re: upgrading Fedora 26 to 27 templates makes me realize that we can start to address the whiner comments i.e. “too much text”, “too many warnings” etc.

Why don’t we for fun mirror Qube’s approach, and have a short section also, something like so:

(applies if you have unique content differing from the Qubes entry)

Quick Guide

Create the dvm which the DispVM will be based on.

[dom0]: qvm-create -P <pool_name> --template service-dvm --class DispVM --label red disp-sys-net
[dom0]: qvm-prefs service-dvm virt_mode hvm
[dom0]: qvm-prefs disp-sys-net provides_network true

Create the sys-net DispVM.

[dom0]: qvm-create -P <pool_name> --template service-dvm --class DispVM --label red disp-sys-net
[dom0]: qvm-prefs service-dvm virt_mode hvm
[dom0]: qvm-prefs disp-sys-net provides_network true
[dom0]: qvm-prefs net-disp netvm “”
[dom0]: qvm-pci footnote reason
[dom0]: qvm-pci attach --persistent net-disp : footnote reason
[dom0]: qvm-prefs disp-sys-net autostart true
[dom0]: qubes-prefs clockvm disp-sys-net footnote optional

etc…

Now put this in a pretty box, and everyone will be happy :wink:

2 Likes

I like that idea! Bare bones: install software-foo / configure software-foo

whiners re: there are two categories of whiners

  • i know what I’m doing so these warning or “notice boxes” don’t apply to me. ( there the ones who should be reading them)

  • i don’t understand how important this info really is (they think the magical properties of Whonix/Tor will keep them anonymous)

That information if there for a reason. If those whiners lived in a country where even thinking of free speech could get you thrown in prison they would be grateful for those warnings/text.

On another note, I went ahead and submitted that qubes-docs DispVM pull request. The instructions may be a little more verbose than they’re used to seeing. I have my fingers crossed.

2 Likes

Great stuff 0brand. If they accept it, we link to your stuff. If rejected, we create a page here as previously discussed.

1 Like

→ Fixed

→ Fixed

See below.

@Patrick

GitHub - tasket/Qubes-VM-hardening: Fend off malware at Qubes VM startup

vm-boot-protect-root - Protects /home/user as above, automatic /rw executable deactivation, whitelisting, checksumming, deployment. Works with appVMs, netVMs, etc. that are template-based.

CAUTION: The -root option by default removes prior copies of /rw/config, /rw/usrlocal and /rw/bind-dirs. This can delete data!

Is the -root option safe for Whonix? This is why I didn’t recommend it in hardening list - not sure.

1 Like

Minor nits and rewording. Suggestion for introduction:

Create Custom sys-net, sys-firewall, and sys-usb DispVMs

Users have the option of creating customized DispVMs for the sys-net, sys-firewall and sys-usb VMs. In this configuration, a fresh VM instance is created each time a DispVM is launched. Functionality is near-identical to the default VMs created following a new Qubes’ installation, except the user benefits from a non-persistent filesystem.

Functionality is not limited, users can:

  • Set custom firewall rulesets and run Qubes VPN scripts. [ref]By making necessary changes to the dvm template.[/ref]
  • Set DispVMs to autostart at system boot.
  • Attach PCI devices with the --persistent option. [ref]Since both of the aforementioned configuration options are only required for the initial DispVM creation.[/ref]

Using DispVMs in this manner is ideal for untrusted qubes which require persistent PCI devices, such as USB VMs and NetVMs.

Note: Users who want customized VPN or firewall rulesets must create a seperate dvm for use by each DispVM. If dvm customization is not needed, then a single dvm is used as a template for all DispVMs.

(put note: stuff in an info box)

2 Likes