Replacing Whonix meta packages with SaltStack, Chef or Puppet?

Just get rid of the meta packages and use a micro-version of SaltStack, Chef or Puppet to manage the installation and removal of individual packages.

I could set up a Salt minion in a day or so that would be able to manage many scenarios. These management systems can install using the systems package manager or directly from git, pip, ftp, etc. They can also take care of any configurations that may be required.

It can be hard to wrap you head around what they actually can do for you and there is a steep learning curve if you don’t have anything to start with, but if someone like me provided a pretty much complete setup, you would then be able to very easily figure stuff out.

For instance, not that I am suggesting you do this at all, but to give an example to the configuration power they offer, Salt, Chef or Puppet could completely replace the existing Whonix installation where everything is handled by the management software including building VM’s, and the ability to deploy on multiple platforms as it has ability to map package names.

Anyway, just let me know if you want me to though a test system together for you to take a look at as its no bother for me to do so since I will be creating something for myself anyway. Just have to know what you want to see.

Maybe have a glace at Salts reference page, as that may give you an idea of what it can do [url=http://docs.saltstack.com/en/latest/ref/index.html]http://docs.saltstack.com/en/latest/ref/index.html[/url] and most importantly its built in modules [url=http://docs.saltstack.com/en/latest/ref/modules/all/index.html]http://docs.saltstack.com/en/latest/ref/modules/all/index.html[/url]

For instance, not that I am suggesting you do this at all, but to give an example to the configuration power they offer, Salt, Chef or Puppet could completely replace the existing Whonix installation where everything is handled by the management software including building VM's, and the ability to deploy on multiple platforms as it has ability to map package names.
So there could be build targets virtualbox, kvm and also physical isolation?

[hr]

Sounds like a huge change.

Are there any other distributions doing something like that?

I am concerned, that we’re going to do something very non-standard. But that could be FUD.

Does it cower image builds from source as well as upgrading from a remote (apt) repository as well as locally upgrading from source code?

What changes users would notice in their Whonix work flow? Which commands they’d run in order to upgrade? Which commands they’d run when deselected a single package form a “meta package” replacement thingy?

Anyway, just let me know if you want me to though a test system together for you to take a look at as its no bother for me to do so since I will be creating something for myself anyway. Just have to know what you want to see.
Sure. I am interested to look at it. Let's see.

So there could be build targets virtualbox, kvm and also physical isolation?[/quote]

Yes. Virtual Machines can be created and pushed to the cloud, not that you need that functionality. Instead of using bootstrap, they usually pull from an official image and boot strap that since its quick and can be configured however you want. Services can be disabled, uninstalled, installed based on set conditions.

Sounds like a huge change.

Are there any other distributions doing something like that?

I am concerned, that we’re going to do something very non-standard. But that could be FUD.

I don’t know of any distributions doing anything like this. Most larger corporations, governments, etc use this though to be able to manage and provision computers desktops, servers, etc all centrally. I like it cause I can bootstrap a VM or bare metal machine and very quickly it is configured exactly how I want it including personal settings, network connection, etc.

I would not suggest diving into the deep end. You can consider this as another installation option; one that would be able to install on a users own bare metal or virtual machine. Initially all it needs to do is act like the meta packages by downloading an actual Debian package all the required Whonix packages and dependencies for what ever options, flavours the user chooses. All required configurations will also be able to take place at that time.

Then upgrades would be normal; just use apt-get or the GUI packagekit.

Does it cower image builds from source as well as upgrading from a remote (apt) repository as well as locally upgrading from source code?

Yes, it can be configured to do pretty much anything. They are driven by state files, which are read by various modules to do certain tasks. Tasks can be one shot or scheduled. The state files are normally written in yaml and are simple to understand once you learn how to use them. They can also be scritped to add additional functionality. I will provide a few sample state files at the end.

What changes users would notice in their Whonix work flow? Which commands they'd run in order to upgrade? Which commands they'd run when deselected a single package form a "meta package" replacement thingy?

I would imagine building something into Whonixsetup. You could have an advanced panel that could have some check mark which could be checked or unchecked to select desired operation. Then salt would set the system up to the desired state. At that point everything would operate as normal; apt-get will do its thing and the user only ever needs to fire up Whonixsetup again if they want to change system options offered.

[HR]

Ok, here are two sample state files. The first one is a simple state to make sure a Nginx server is installed, enabled and runnig and that a www directory is created with specific permissions. salt can re-run the states every XX minutes to configure system is in an expected state or just wait for manual operation

# Make sure nginx is installed and up
nginx:
  pkg:
    - installed
  service.running:
    - require:
      - pkg: nginx
      - file: www_root

# Make sure the default web directory exists and permissions are set correctly
www_root:
  file.directory:
    - name: /var/www
    - makedirs: True
    - user: www-data
    - group: www-data

The next example is more complicated, but makes set up KVM including brige networking and some kernel tweaks, makes sure it also installs all depends and related packages

kvm-packages:
  pkg.installed:
    - pkgs:
      - qemu-kvm
      - libvirt-bin
      - qemu-system
      - bridge-utils
      - virt-manager
      - virt-viewer
      - virt-goodies
      - virt-top
      - virtinst
      - python-libvirt
      - ubuntu-vm-builder
      - spice-client-gtk
      - spice-client
      - vlan
      - guestfish
      - guestfsd
      - guestmount
      - lvm2
      - nfs-common
      - munin-libvirt-plugins
      - munin-node
      - multipath-tools

/etc/libvirt/qemu.conf:
  file.managed:
    - user: root
    - group: root
    - mode: 444
    - source: salt://kvm/qemu.conf
    - require:
      - pkg: kvm-packages

kvmhostkey:
  ssh_auth.present:
    - user: root
    - source: salt://users/keys/master.id_rsa.pub

/root/.ssh/id_rsa:
  file.managed:
    - user: root
    - group: root
    - mode: 600
    - user: root
    - source: salt://users/keys/master.id_rsa

libvirt-bin:
  service.running:
    - require:
      - pkg: kvm-packages
      - file: /etc/libvirt/qemu.conf
      - ssh_auth: kvmhostkey
    - watch:
      - file: /etc/libvirt/qemu.conf

vm.swappiness:
  sysctl.present:
    - value: 0

vm.zone_reclaim_mode:
  sysctl.present:
    - value: 0

net.bridge.bridge-nf-call-arptables:
  sysctl.present:
    - value: 0

/hugepages:
  mount.mounted:
    - device: hugetlbfs
    - fstype: hugetlbfs
    - mkmnt: True

grub-settings:
  file.append:
    - name: /etc/default/grub
    - text: 'GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0,9600n8 console=tty0 text nosplash nomodeset nohz=off transparent_hugepage=always"'
  file.append:
    - name: /etc/default/grub
    - text: 'GRUB_CMDLINE_LINUX="console=ttyS0,9600n8 console=tty0 text nosplash nomodeset nohz=off transparent_hugepage=always"'

update-grub:
  cmd.run:
    - name: update-grub
    - require:
      - file.append: grub-settings

Well, here is one last example, showing an applications that gets it updates via git and not a package manager. The package downloaded from git will be installed and compiled if needed. Salt checks for updates if that options is selected, not that you need this feature.

salt-dependencies:
  pkg.installed:
    - names:
      - git
      - python
      - python-dev
      - python-m2crypto
    - require:
      - pkg: pip-dependencies
      - python-jinja2    # APT: 2.7.2-2

salt-pip-dependencies:
  pip.installed:
    - names:
      - pyzmq             # PIP: 14.0.1
      - PyYAML            # PIP: 0.8.4
      - pycrypto          # PIP: 2.6.1
      - msgpack-python    # PIP: 0.3.0
      - jinja2            # 2.7.2
      - psutil            # not-installed 
    - require:
      - pkg: salt-dependencies

# Install development version from git
salt:
  pip.installed:
    - name: salt
    - editable: "git+https://github.com/saltstack/salt.git@2014.7#egg=salt"
    - no_deps: True # We satisfy deps already since we cant build m2crypto on debian/ubuntu
    - install_options: "--prefix=/usr --force-installation-into-system-dir"
    - upgrade: True
    - require:
      - pkg: salt-dependencies
      - pip: salt-pip-dependencies
      - pkg: git

Like I said, its very configurable and nicely human readable. Once you see you Whonix state files you will catch on very quickly since you you know the internals so well. I bet if you review these state files you will already have an understand of what implementing.

And even after I put this together, I really don’t mind if you never want to implement any of it since I was planning on using this fo myself anyway

TAILS do Puppet modules : Tails - Git repositories

EDIT:
I don’t know much about the topic to advocate for it or which implementation should be used.

Alright. Phew. I thought this was going to be a proposal for a full solution of Whonix Forum / full replacement of anon-meta-packages.