constrained system resources program starter wrapper

Patrick · December 28, 2020, 9:32am

systemd provides a powerful tool, systemd-run which can constrain how many system resources an application started by it can use. It can limit (non-exhaustive list) use of CPU, memory and disk. However, this powerful too is too complicated to use on the command line. A wrapper is required to make it easily accessible. Use case: running resource intensive tasks (such as backups, compression) without making the system (almost) unusable by the load generated by that task.

limit-low design goals:

Do one thing and do it well.

The one thing is to be a wrapper that limits system resources for wrapped
applications.

Support being run as:

user

root

in shell or script in pipes

stdout, stderr, stdin interactive shell input/output

graphical (GUI) applications

Being as non-intrusive as possible. For example, not adding extraneous
output to stdout such as
“Running scope as unit: run-r0d607a8f35dc4dea909b830f9d922b99.scope”.

limit-low is a around systemd-run setting systemd-run parameters to run an
application with limited system resources. systemd-run itself is using Linux cgroups.

systemd upstream feature request:

To start graphical (GUI) applications such as Tor Browser it requires installation of package dbus-user-session. The (non-)security impact of this is being discussed here: dbus - user vs system session.

sudo apt update
sudo apt install dbus-user-session

Depending on the outcome of that discussion, dbus-user-session will be installed by default in Kicksecure and Whonix or perhaps systemd-run can be run with different parameters such as --scope not requiring dbus-user-session. --scope as alternative works but systemd tells:

--pty/--pipe is not compatible in timer or --scope mode

Which would break limit-low design goals.

Patrick · December 28, 2020, 12:45pm

To test:

sudo apt update
sudo apt install stress

Does not work as user yet.

limit-low stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s

Only works as root.

sudo limit-low stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s

Difference can be easily seen for example in ksysguard.

Patrick · December 28, 2020, 2:18pm

Perhaps it would be better to implement this with plain cgroups, i.e. cgexec? Avoiding systemd-run. Could avoid a lot issues such as dbus, environment variables, and whatnot.

Patrick · December 31, 2020, 2:10pm

cgroups are not easy to use for this purpose either. Here is what I got.

set -x
set -e

## TODO
#sudo apt install cgroup-tools

testcommand="stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s"
#testcommand=hello
testcommand="dd if=/dev/zero of=/tmp/xxx bs=10M count=1"

sudo cgdelete cpu,memory,blkio:limit-low &>/dev/null || true

## TODO: user:user
sudo cgcreate -t user:user -a user:user -g cpu,memory,blkio:limit-low

## cpu.max default is 100000
#sudo cgset -r cpu.max=1 limit-low
#sudo cgset -r cpu.shares=1 limit-low

## https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt
## 20%
sudo cgset -r cpu.cfs_quota_us=10000 limit-low
sudo cgset -r cpu.cfs_period_us=50000 limit-low

sudo cgset -r memory.limit_in_bytes=100m limit-low

## https://andrestc.com/post/cgroups-io/
## To limit I/O to 1MB/s, as done previously, we write into the io.max file:
## $ echo "8:0 wbps=1048576" > io.max
#sudo cgset -r io.max="8:0 wbps=1048576" limit-low

## https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/blkio-controller.html
## Specify a bandwidth rate on particular device for root group. The format for policy is “<major>:<minor> <bytes_per_second>”:
## echo "8:16  1048576" > /sys/fs/cgroup/blkio/blkio.throttle.read_bps_device
## Above will put a limit of 1MB/second on reads happening for root group on device having major/minor number 8:16.
##
## https://lqhl.me/blog/2015/09/09/network-disk-bandwidth-limiting-on-linux/
## <major>:<minor>  <bytes_per_second>
## How to find out <major>:<minor>?
## TODO: Qubes ls -la /dev/xvda
# sudo cgset -r blkio.throttle.read_bps_device="202:0  1048576" limit-low
# sudo cgset -r blkio.throttle.write_bps_device="202:0  1048576" limit-low

cgexec -g cpu,memory,blkio:limit-low \
$testcommand

#ionice --class 3 \
#nice -n 19 \

sudo cgdelete cpu,memory,blkio:limit-low

Patrick · December 31, 2020, 3:00pm

Neither with cgroups one can easily say “use maximum hard limit of xx % of available system resources (CPU, RAM, IO) for this application”.

Maxmium RAM can be a hard limit such as xxxx MB.

Throttling a device IO (virtual) (harddrive) syntax is <major>:<minor> <bytes_per_second>. See https://lqhl.me/blog/2015/09/09/network-disk-bandwidth-limiting-on-linux/. Also a hardcoded value. Which value to choose? One would have to benchmark the disk first before deciding on a reasonable value. Also finding out <major>:<minor> is non-trivial from a script.

It seems cgroups resource moderation is more useful to say “within this cgroup this application can use xx % of CPU”. This might be useful when implementing this for the whole system.

Maybe just combining ionice with nice would be a good enough hack. I.e.

ionice --class 3 nice -n 19 application-name

But that wouldn’t limit maximum RAM the application can use. And once RAM is exhausted, system will probably freeze or get so slow that one would loose patience and reboot.

Would be best if Debian had this implemented for the whole system. For example guaranteeing that X always has sufficient system resources and cannot be starved by a broken application.

Patrick · December 31, 2020, 3:04pm

Patrick · January 4, 2021, 9:49am

Patrick · January 4, 2021, 9:56am