This wasn’t meant as a disrespect.[/quote]
Let’s forget this. I do apologize.
We don't know how many people use KVM, so we shouldn't assume they are few. Linux users tend to be proficient in using computers in general and so you wouldn't have seen complaints from that usergroup in general. Also those who are happy campers don't tend to write to say that they are, its only when shit hits the fan ;)Right. Seems I was irrational yesterday. Maybe it was the temperatures.
[quote=“HulaHoop, post:120, topic:166”]Alright so I read the manual for virsh more carefully this time and here is what it’s about, it tells a user how to control and manage a vm running under virsh, but doesn’t really have much to do in the way of creating a vm. The creation step is hinted at to involve dumpxml.
I did some digging around and I think I found the answer this time. Check the second answer with 9 upvotes to see what I mean:
virt-clone is supposed to deal with duplication and also do it in a way that makes it work for other machines host machines, bypassing uuid and folder path problems:
Theoretically if this works all a user needs to do is to import 3 xml files, 2 vm and 1 virtual network, through commandline and be done.
Manual I found:
Patrick please understand that I am not asking you to do anything here, just need to know if the tools I mention here is a step in the right direction and satisfy the verifiable builds property of Whonix.[/quote]
Sounds good.
Scratch space for commands here - More to be added later:
Work in progress: will remove this tag when done
Using and applying the example on stackexchange but this time for the virtual network:
Virsh Command Reference – to look up any commands I might need.
Here are the configuration files and the commands to import them
Work in progress: will remove this tag when done
IMPORT COMMANDS:
#To import virtual isolated network and to set it to autostart
virsh net-define /tmp/whonix.xml
virsh net-autostart /tmp/whonix.xml
#To import vms
virsh define /tmp/whonix_gateway.xml
virsh define /tmp/whonix_workstation.xmlCONFIG FILES:
–Removed version number 8.2 from vm name setting.
whonix.xml
<network connections='2'>
<name>whonix</name>
<bridge name='virbr1' stp='on' delay='0'/>
<domain name='whonix'/>
<ip address='10.0.0.1' netmask='255.255.255.0'>
</ip>
</network>
name as: whonix_gateway.xml
[code]
whonix_gateway
524288
524288
1
hvm
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
/usr/bin/qemu-kvm
/dev/random
[/code]
name as: whonix_workstation.xml
[code]
whonix_workstation_
1048576
1048576
1
hvm
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
/usr/bin/qemu-kvm
/dev/random
[/code]
When you make the images,would it be better to name the qcow images to something more generic? The version number could be something only visible in the configuration files instead.
I suggest that xml files be packed in their own separate tar archive unless there is a way to define different extraction destinations for files in one archive .
Note: There is a nifty feature in kvm that attaches a paravirtual /dev/random to guests, allowing them to have much higher entropy because it uses the host’s pool. That’s vital for generating secure keys.
maybe consider using 32 bit architecture for such little vm ram?
I set the cpu to ‘hypervisor default’ in order to avoid a fallback dependency on host’s cpu model. There is no other way.
The stackexchange instructions prove helpful only when preparing the original vm template. The contents of these xml files would need to committed to the repo as they are the kvm/libvirt equivalent of the vbox build script you linked to.
What happens with these config files, is what needs scripting. But I would need to know more about the default build process because that might need changes.
Do the current scripts need to be changed to generate fixed generic vm image names, for example whonix_gateway instead of whonix_gateway_versionX?
Does the path of the format conversion output need to be set to /var/lib/libvirt/images or maybe everything could be extracted to /tmp instead?
Should the import steps listed above be automated ideally to improve usability? So we tell people download the tarballs and put them in a specific direcotry and just run this script?
If relative paths could be accomplished with this, it would make life much more easier. but I am not sure if this is possible - or if it needs virsh to acknowledge relative paths…
I don’t know anymore were this was discussed, but originally there were complaints about not using version numbers in file names so version could not be read from file name.
But if it helps… It’s in help-steps/variables.
[ -n "$binary_image_qcow2" ] || binary_image_qcow2="$WHONIX_BINARY/$VMNAME-$whonix_build_closest_git_tag.qcow2"Link:
https://github.com/Whonix/Whonix/blob/master/help-steps/variables#L278
We can change this to.
[ -n "$binary_image_qcow2" ] || binary_image_qcow2="$WHONIX_BINARY/$VMNAME.qcow2"But if it helps. Well. If you wish, send a pull request, or I do it if you wish.
I suggest that xml files be packed in their own separate tar archive unless there is a way to define different extraction destinations for files in one archive .Sounds good!
Note: There is a nifty feature in kvm that attaches a paravirtual /dev/random to guests, allowing them to have much higher entropy because it uses the host's pool. That's vital for generating secure keys.Great. Surely add it.
I don't know anymore were this was discussed, but originally there were complaints about not using version numbers in file names so version could not be read from file name.But if it helps… It’s in help-steps/variables.
I experimented to see if --relative-path option works as per the commands in the now obsolete opensolaris kvm version: virsh(1m) [opensolaris man page]
And unfortunately this no longer works. I can now confirm that relative paths are not recognized in virsh/libvirt and so I will need to proceed accordingly.
So for the qcow images we will have to rely on generic names no way around that. Please change the build string to not include a version number.
error: command 'define' doesn't support option --relative-path
dumpxml domain [--relative-path path]Output the configuration of the given domain in XML format. Captured in a file, this data can be used as the argument to a subsequent create subcommand. [b] By default, all paths in the XML will be absolute. Adding the --relative-path option will make all disk paths relative to path.[/b]</blockquote>
We could make a folder “libvirt” (because it’s more libvirt than kvm specific, also qemu can be easily used) [or other name] in the source root folder in https://github.com/Whonix/Whonix if you wish. Feel free to send a pull request, so you get all the authorship.
It would also be very simple to make a separate debian package. https://github.com/Whonix/whonix-libvirt-vms or so. Files could be installed to /usr/share/whonix-libvirt-vms/… Creating such a package is very simple and I could do that quickly. We just need to think about what is most simple for users. Someday we could even offer a whonix-libvirt-installer debian package that automates creating Whonix kvm VMs and downloading the qcow2.
What happens with these config files, is what needs scripting. But I would need to know more about the default build process because that might need changes.Yeah. This is up to us. One thing we could do during the build process is taking the ./libvirt folder and creating a whonix-libvirt-9.1.xz and offer that one mirror.whonix.org.
1. Do the current scripts need to be changed to generate fixed generic vm image names, for example whonix_gateway instead of whonix_gateway_versionX?Just needs to be changed in one place. Answered above. (Easy stuff.)
2. Does the path of the format conversion output need to be set to /var/lib/libvirt/images or maybe everything could be extracted to /tmp instead?
3. Should the import steps listed above be automated ideally to improve usability? So we tell people download the tarballs and put them in a specific directory and just run this script?
If relative paths could be accomplished with this, it would make life much more easier. but I am not sure if this is possible - or if it needs virsh to acknowledge relative paths...
This all depends on how we want to implement it. Let’s start with a instructions proposals. Just quick thoughts. Feel free to suggest better ones. We pick the one that is most simple (download) and maybe one that is most secure (from source).
Proposal 1:
download all to same folder
somehow bother with getting signing key and doing gpg verification
download gw qcow2
download ws qcow2
download xml tarball
extract xml tarball
run installer script (from xml tarball)
start virt-manager
enjoy whonix
Proposal 2:
no qcow2 downloads
build from source using ./whonix_build_both --kvm
somehow bother with getting signing key and doing gpg verification
qcow2’s are moved to /var/lib/libvirt/images/… at the end of the build process
a new kvm specific build-step in build-steps.d folder automates setting up the VM settings
start virt-manager
enjoy whonix
Proposal 3:
download the kvm installer script
somehow bother with getting signing key and doing gpg verification
make the kvm installer script executable
run the kvm installer script
[the kvm installer script downloads the qcow2’s and xml’s, imports the xml’s]
start virt-manager
enjoy whonix
So many possibilities.
We could make a folder "libvirt" (because it's more libvirt than kvm specific, also qemu can be easily used) [or other name] in the source root folder in https://github.com/Whonix/Whonix if you wish. Feel free to send a pull request, so you get all the authorship.
Sure, calling ‘libvirt’ does make sense. I’ll create a github account and add that.
It would also be very simple to make a separate debian package. https://github.com/Whonix/whonix-libvirt-vms or so. Files could be installed to /usr/share/whonix-libvirt-vms/... Creating such a package is very simple and I could do that quickly. We just need to think about what is most simple for users. Someday we could even offer a whonix-libvirt-installer debian package that automates creating Whonix kvm VMs and downloading the qcow2.
Definitely agree. I think more than one of those proposals are ideal for different user models
Proposal 1: download all to same folder somehow bother with getting signing key and doing gpg verification download gw qcow2 download ws qcow2 download xml tarball extract xml tarball run installer script (from xml tarball) start virt-manager enjoy whonix
Ideal for those who acquire whonix anonymously over Tor as no direct connections made to whonix website. I do that so I think its important as an option personally.
Proposal 2: no qcow2 downloads build from source using ./whonix_build_both --kvm somehow bother with getting signing key and doing gpg verification qcow2's are moved to /var/lib/libvirt/images/... at the end of the build process a new kvm specific build-step in build-steps.d folder automates setting up the VM settings start virt-manager enjoy whonix
For those who build from source its a very clever approach that leads to the desired result.
Proposal 3: download the kvm installer script somehow bother with getting signing key and doing gpg verification make the kvm installer script executable run the kvm installer script [the kvm installer script downloads the qcow2's and xml's, imports the xml's] start virt-manager enjoy whonix
Excellent for newcomers and beginners and what we should aim for.
IMO all three proposals are very useful and have specific valid use cases. Great ideas Patrick 
Just a thought. For those who might complain about being confused, about the version, maybe having whonixcheck announce the version should be enough
Proposal 4:
download gateway.libvirt.xz (contains gateway.qcow2 + libvirt network xml + libvirt gateway xml + installer script)
somehow bother with getting signing key and doing gpg verification
extract all files to the same folder
sudo ./gw-installer (moves files to /var/lib/libvirt/images/…, imports xml)
download workstation.libvirt.xz (contains workstation.qcow2 + libvirt workstation xml + installer script)
extract all files to the same folder
sudo ./ws-installer (moves files to /var/lib/libvirt/images/…, imports xml)
start virt-manager
enjoy whonix
Forget about Proposal 3 for now. That’s a separate feature “Secure Downloader to download Whonix”:
I think proposal 4 is better than proposal 1. Less file downloads.
[And proposal 3 could be seen as an extension to whatever we choose here that we might do at a later point.]
Proposal 4 is my favorite for now. [And for those who build from source (proposal 2), they would just run the installer after building and automation to automatically do this after build should also not be hard.]
[quote=“Patrick, post:136, topic:166”]I think proposal 4 is better than proposal 1. Less file downloads.
[And proposal 3 could be seen as an extension to whatever we choose here that we might do at a later point.]
Proposal 4 is my favorite for now. [And for those who build from source (proposal 2), they would just run the installer after building and automation to automatically do this after build should also not be hard.][/quote]
Yeah I agree, this make the most sense.
A few ideas for script dialogues:
Will the setup script automatically overwrite the older version of the images and xml of the same name? Will it mention that it is about to overwrite an already present setup?
I added the xml in the repo btw.
On Github I am trying to create a subfolder named kvm under libvirt and move the files I put up there into that subdirectory, but I don’t know how to easily. Is the the only way to delete and recreate them gain under this new folder?
Reason for this folder tree arrangement, is that it allows us to differentiate between the the different libvirt supported options should they be extended to support something more ie. qemu.
It is important that this be done through libvirt becuase of its enforcement of protection measures thru sVirt like Mandatory Access Controls and seccomp.
Also other things I was thinking we could change with your permisssion is to remove the kvmclock warning its still enabled but bears no ill effects on anonymity as I discovered since time is synced only at guest startup.
[quote=“HulaHoop, post:137, topic:166”]A few ideas for script dialogues:
Will the setup script automatically overwrite the older version of the images and xml of the same name?[/quote]
Images: doing a test if the file already exists and not overwriting it without asking is simple to script.
XML: We can probably run something like “virsh domstate Whonix-Gateway ; echo $?”. If it exists 0, VM already exist: ask if it should overwrite. If it exists 1, then it does not exist, then just import.
Will it mention that it is about to overwrite an already present setup?It probably should?
On Github I am trying to create a subfolder named kvm under libvirt and move the files I put up there into that subdirectory, but I don't know how to easily. Is the the only way to delete and recreate them gain under this new folder?git can do anything. It can record any changes in a folder. github web interface is limited.
Reason for this folder tree arrangement, is that it allows us to differentiate between the the different libvirt supported options should they be extended to support something more ie. qemu.Will this be really necessary? How much changes in kvm vs qemu? Is it just ? In that case it might be better to share the same file so improvements in one are shared in the others and just to patch it in the installer script depending on user choice.
It is important that this be done through libvirt becuase of its enforcement of protection measures thru sVirt like Mandatory Access Controls and seccomp.That what is done through libvirt?
Also other things I was thinking we could change with your permisssion is to remove the kvmclock warning its still enabled but bears no ill effects on anonymity as I discovered since time is synced only at guest startup.What does KVMClock do? "kvmclock or KVM pvclock lets guests read the host’s wall clock time." This is bad. (source: http://rwmj.wordpress.com/2010/10/15/kvm-pvclock/)
We already figured out how to disable kvmclock ?
Why not just apply it?
Does it work with
so it autodetects it?
Maybe we can also leave
<emulator></emulator>empty or omit it?
(Trying to somehow make it more generic.)
See also: