swap - swap file - Whonix-Gateway freezing during apt-get dist-upgrade - encrypted swap-file-creator

Patrick · October 20, 2019, 7:13am

Whonix-Gateway default RAM is currently set to 768 MB.
(Whonix-Workstation default RAM is currently set to 2048 MB.)

During upgrade Whonix-Gateway freezes. Not totally. But it looks frozen. The seconds on the clock stay at the same number for several minutes. This happens during module building (VirtualBox guest additions) as well as kernel header package upgrade. During that time, Tor running on Whonix-Gateway is not accessible from Whonix-Workstation.

Hence, I plan to re-vitalize swap-file-creator. I couldn’t find out why it was previously removed from installation by default. Once swap-file-creator is running, this issue is gone. (A small fix for buster was required.)

This is about Non-Qubes-Whonix only since Qubes-Whonix has dynamic RAM assignment (and higher system requirements).

Could you please review / improve this line? @HulaHoop

cryptsetup --batch-mode --hash sha512 -c aes-xts-plain64 --key-size 512 luksFormat

github.com

Kicksecure/swap-file-creator/blob/master/usr/share/swap-file-creator/common#L97


      
          fi
          
          
[ -n "$SLEEP_SECONDS_ON_ERROR" ] || SLEEP_SECONDS_ON_ERROR="5"
          [ -n "$DESC" ] || DESC="Encrypted Swap File Creator"
          [ -n "$NAME" ] || NAME="swap-file-creator"
          [ -n "$SCRIPTNAME" ] || SCRIPTNAME="/etc/init.d/$NAME"
          [ -n "$LOG" ] || LOG="/var/log/$NAME.log"
          [ -n "$SWAPFILE" ] || SWAPFILE="/var/swapfile"
          [ -n "$VERBOSE" ] || VERBOSE="yes"
          [ -n "$UUID" ] || UUID="0615ba72-85b0-4183-8d54-300bb0d2e491"
          [ -n "$DD_IF" ] || DD_IF="/dev/zero"
          [ -n "$DD_BS" ] || DD_BS="1M"
          [ -n "$DD_COUNT" ] || DD_COUNT="512"
          [ -n "$PV_SIZE" ] || PV_SIZE="${DD_COUNT}M"
          [ -n "$MAPPER" ] || MAPPER="swapfile"
          [ -n "$MAPPER_FULL" ] || MAPPER_FULL="/dev/mapper/$MAPPER"
          [ -n "$RUN_FOLDER" ] || RUN_FOLDER="/run/$NAME"
          ## https://gitlab.com/cryptsetup/cryptsetup/-/issues/617
          [ -n "$cryptsetup_pre_wrapper" ] || cryptsetup_pre_wrapper="ld-system-preload-disable"
          [ -n "$CRYPTSETUP_FORMAT" ] || CRYPTSETUP_FORMAT="cryptsetup --batch-mode --pbkdf pbkdf2 --hash sha512 -c aes-xts-plain64 --key-size 512 luksFormat"
          [ -n "$SHRED_ON_STOP" ] || SHRED_ON_STOP="no"

HulaHoop · October 21, 2019, 1:40pm

Alright I guess this doesn’t affect live mode since everything is redirected to RAM including the swap file. Question is if this would push current RAM reqs way higher than it is now vs increasing GW RAM without adding a swap file.

Everything else looks good. Since we don’t care about memorizing passphrases here just use a randomly generated character string.

Patrick · October 24, 2019, 11:57am

In effect,

no swap file will be created by default in Whonix-Workstation (since it has 2 GB RAM).
But in Whonix-Gateway would create a swapfile (since default 768 MB RAM). - Unless the user gives Whonix-Gateway 2 GB RAM manually.

Disabled in live mode already (but in theoretic case that was broken, then yes). I guess. Location of swapfile is /var/swapfile.

https://github.com/Whonix/swap-file-creator/blob/master/usr/share/swap-file-creator/start#L95

Why would adding a swap file increase GW RAM requirements? Perhaps I got that wrong.

I don’t like swap / swapfile since use of these can make the system super slow but it seems necessary. Also therefore, related, swappiness kernel setting is set to lowest:

https://github.com/Whonix/swappiness-lowest/blob/master/etc/sysctl.d/swappiness-lowest.conf

Indeed. However, anti-forensics cannot be guaranteed due to the same reasons listed on Encrypted VM Images.

HulaHoop · October 24, 2019, 12:05pm

I was under the impression any newly file created since boot up is redirected to RAM and since it’s being recreated anew every boot it occupies space?

Patrick · October 24, 2019, 12:08pm

This concern is for live mode only?
Yes, I have no idea what would happen there. Creating a (swap) file (really any file) in live mode could be an issue.
But fairly certain, it swap-file-creator will not create a swapfile in live mode.
Therefore no issue?

HulaHoop · October 24, 2019, 12:09pm

Yes

Perfect

Patrick · October 21, 2019, 3:08pm

HulaHoop via Whonix Forum:

Patrick:

Hence, I plan to re-vitalize swap-file-creator. I couldn’t find out why it was previously removed from installation by default. Once swap-file-creator is running, this issue is gone. (A small fix for buster was required.)

Alright I guess this doesn’t affect live mode since everything is redirected to RAM including the swap file.

Disabled in live mode already (but in theoretic case that was broken,
then yes). I guess. Location of swapfile is /var/swapfile.

Question is if this would push current RAM reqs way higher than it is now vs increasing GW RAM without adding a swap file.

Why would adding a swap file increase GW RAM requirements? Perhaps I got
that wrong.

Well, adding 256 MB or 512 MB “real” (virtual) RAM instead of a swap
file could abolish need for a swapfile. Additional 512 MB should e
sufficient for sure since swapfile has that size currently and solved
these issues.

swap-file-creator could abort creating a swap file if more than XXX MB
of RAM is available. Such a feature would be easy to add if that seems
useful.

I don’t like swap / swapfile since use of these can make the system
super slow but it seems necessary. Also therefore, related, swappiness
kernel setting is set to lowest:

https://github.com/Whonix/swappiness-lowest/blob/master/etc/sysctl.d/swappiness-lowest.conf

Since we don’t care about memorizing passphrases here just use a randomly generated character string.

Indeed.

Patrick · December 22, 2023, 10:58am

swap-file-creator can probably be speed up massively by using fallocate instead of dd to create the swap file at boot time.
It might also become more compatible, fast, reliable by making better use of systemd as per:

How big should the swap file be?

Patrick · December 22, 2023, 2:36pm

github.com

Kicksecure/helper-scripts/blob/master/usr/bin/calculate-swap-size

#!/bin/bash

set -e
set -o pipefail
set -o nounset

calculate_swap_size() {
  local ram_in_mb=$1
  local hibernation=$2
  local swap_size=0
  local sqrt_ram_gb
  local sqrt_ram_mb

  echo "Calculating swap size..."
  echo "RAM in MB: $ram_in_mb"
  echo "Hibernation considered: $hibernation"

  # Convert MB to GB for comparison
  local ram_in_gb=$((ram_in_mb / 1024))
  echo "RAM in GB (approximated for calculation): $ram_in_gb"

This file has been truncated. show original

Patrick · December 22, 2023, 3:51pm

github.com/Kicksecure/swap-file-creator

vastly improve swap-file-creator

committed 03:49PM - 22 Dec 23 UTC

adrelanos

+208 -290

* always create a swap file if swap-file-creator is installed * increased minimu…m size of swap file to 6 GB * vastly speed up swap-file-creator through use of `fallocate` and `systemd-cryptsetup` from several or tens of seconds to around 1 second * use `calculate-swap-size` to calculate size of swap file to be created * take into account for size calculation if the system is using hibernation, it is assumed that this is not the case in VMs, therefore a smaller swapfile is sufficient. * do not attempt to create a swap file if there is insufficient disk space * improved output * systemd `After=systemd-random-seed-load.service` for better entropy * added a test to check if a swap file with the expected size has been created * improved error handling * code simplification through integration with systemd / journald for logs and status * removed little/unused configuration options, see /etc/default/swap-file-creator https://forums.whonix.org/t/swap-swap-file-whonix-gateway-freezing-during-apt-get-dist-upgrade-encrypted-swap-file-creator/8317/8

Done.

Patrick · December 25, 2023, 7:15pm

This is now in the testers repository.