miscellaneous development log

Patrick · May 26, 2020, 3:47pm

Dev/About Infrastructure - Kicksecure

Patrick · May 30, 2020, 7:06pm

please add openssl-rsync script for use of encrypted rsync over TLS

Patrick · May 31, 2020, 11:51am

github.com/SpiderLabs/ModSecurity-nginx

lua-resty-waf - lua based WAF - performance similar to cloudflare WAF

opened 11:49AM - 31 May 20 UTC

closed 12:32AM - 08 Aug 20 UTC

adrelanos

question no-issue-activity

Are you aware of [lua-resty-waf](https://github.com/p0pr0ck5/lua-resty-waf) by @…p0pr0ck5? A LUA based WAF. > ## Performance > > lua-resty-waf was designed with efficiency and scalability in mind. It leverages Nginx's asynchronous processing model and an efficient design to process each transaction as quickly as possible. Load testing has show that deployments implementing all provided rulesets, which are designed to mimic the logic behind the ModSecurity CRS, process transactions in roughly 300-500 microseconds per request; this equals the performance advertised by [Cloudflare's WAF](https://www.cloudflare.com/waf). Tests were run on a reasonable hardware stack (E3-1230 CPU, 32 GB RAM, 2 x 840 EVO in RAID 0), maxing at roughly 15,000 requests per second. See [this blog post](http://www.cryptobells.com/freewaf-a-high-performance-scalable-open-web-firewall) for more information. > > lua-resty-waf workload is almost exclusively CPU bound. Memory footprint in the Lua VM (excluding persistent storage backed by `lua-shared-dict`) is roughly 2MB. Cloudflare WAF is LUA based. https://blog.cloudflare.com/cloudflares-new-waf-compiling-to-lua/ Unfortunately lua-resty-waf hasn't seen development (git commits) since 2018. Would it make sense to rebase ModSecurity-nginx on lua-resty-waf? Consider hire p0pr0ck5 if he's not hired by some CDN / WAF company yet? :) I am writing this on my own initiative. Didn't talk to p0pr0ck5.

Patrick · June 2, 2020, 4:08pm

Patrick · June 8, 2020, 12:21pm

mediawiki

Patrick · June 8, 2020, 7:09pm

github.com/apache/incubator-pagespeed-ngx

document gzip, gzip_static, brotli and brotli_static

opened 07:09PM - 08 Jun 20 UTC

adrelanos

Documentation is at the moment a bit sparse on [nginx gzip](https://nginx.org/en…/docs/http/ngx_http_gzip_module.html). Specifically nginx [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static). It is my understanding that gzip_static cannot be easily used in conjunction with nginx pagespeed. Source: testing myself and [this blog post](https://www.getpagespeed.com/server-setup/varnish-ssl-brotli-gzip-pagespeed-putting-things-together). I guess the way the architecture works is that nginx pagespeed needs/fetches the resource without compression, optimizes it and then adds its own dynamic gzip compression? The gzip compression might even be static once nginx pagespeed has cached the resource? Perhaps worth mentioning in pagespeed documentation that [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static) is not worth bothering with when using nginx pagespeed? [This blog post](https://www.getpagespeed.com/server-setup/varnish-ssl-brotli-gzip-pagespeed-putting-things-together) indicates that it's possible to get quasi gzip static. Perhaps you could shed some light on that? Also can't find any mentions of [brotli](https://github.com/google/brotli) / brotli_static. related google search terms: > site:https://www.modpagespeed.com gzip > site:https://www.modpagespeed.com gzip_static > site:https://www.modpagespeed.com brotli > site:https://www.modpagespeed.com brotli_static

Patrick · June 9, 2020, 12:10pm

github.com/jthingelstad/foreground

caching issue: static CSS is load thorugh load.php and therefore uncacheable

opened 12:10PM - 09 Jun 20 UTC

closed 03:19PM - 14 Dec 20 UTC

adrelanos

non-foreground issue

### Setup - Browsers and versions: firefox-esr 68.8.0esr-1~deb10u1 - MediaWi…ki version: 1.34.1 - Database version: - PHP version: - Foreground version: 2.3.0 ### Issue gtmetrix.com, google pagespeed and other website speedtest / caching test websites complain that the following resources do not have a long expiration date / aren't cacheable: * /w/load.php?lang=en&modules=ext.flaggedRevs.review&skin=foreground&version=18tly * /w/load.php?lang=en&modules=ext.purge&skin=foreground&version=3sx9r * /w/load.php?lang=en&modules=jquery.client&skin=foreground&version=1d1dj * /w/load.php?lang=en&modules=jquery&skin=foreground&version=1xgm5 * /w/load.php?lang=en&modules=jquery.getAttrs&skin=foreground&version=1p30m * /w/load.php?lang=en&modules=jquery.highlightText&skin=foreground&version=1mh8h * /w/load.php?lang=en&modules=jquery.makeCollapsible&skin=foreground&version=rscou * /w/load.php?lang=en&modules=jquery.suggestions&skin=foreground&version=1ucip * /w/load.php?lang=en&modules=mediawiki.String&skin=foreground&version=1bidf * /w/load.php?lang=en&modules=mediawiki.base&skin=foreground&version=m9box * /w/load.php?lang=en&modules=mediawiki.cldr&skin=foreground&version=laa59 * /w/load.php?lang=en&modules=mediawiki.libs.pluralruleparser&skin=foreground&version=ckbfi * /w/load.php?lang=en&modules=mediawiki.page.startup&skin=foreground&version=5umqm * /w/load.php?lang=en&modules=mediawiki.Title&skin=foreground&version=sx80d * /w/load.php?lang=en&modules=mediawiki.searchSuggest&skin=foreground&version=1ecdy * /w/load.php?lang=en&modules=mediawiki.storage&skin=foreground&version=1dq8q * /w/load.php?lang=en&modules=mediawiki.api&skin=foreground&version=6hk5g * /w/load.php?lang=en&modules=site&skin=foreground&version=6dr8h * /w/load.php?lang=en&modules=skins.foreground.modernizr&skin=foreground&version=aptx3 This is because these are fetched through `load.php`. That's probably the reason that these aren't links to `.css` or `.js` files. These files look fairly stable. Is this a mediawiki `load.php` issue or a foreground skin specific issue?

Patrick · June 12, 2020, 4:07pm

github.com/apache/incubator-pagespeed-ngx

document good origin HTTP header settings / nginx config / etags / nginx pagespeed output headers

opened 04:07PM - 12 Jun 20 UTC

closed 09:41PM - 28 Mar 21 UTC

adrelanos

What are some good combinations of HTTP headers? What should users of pagespeed …check their webapps are outputting wrt HTTP headers? Are there some good examples of recommended origin caching HTTP headers? Perhaps there are multiple valid combinations? Or good websites that have good caching settings which are a good role model? Could you please provide a basic nginx config for static contents? There's a lot. Expires; HTTP ETag; weak ETag; vs gzip; last modified. It's still unclear the me which combination of settings / HTTP headers are a good basis for pagespeed to work with. Pagespeed showing an `Etag: W/"0"` being for cachable static files is expected or a bug?

Patrick · June 14, 2020, 5:51pm

github.com/systemd/systemd

persistent ramdisk

opened 04:51PM - 13 Jun 20 UTC

closed 09:15AM - 16 Jun 20 UTC

adrelanos

**Is your feature request related to a problem? Please describe.** Ramdisks /… tmpfs are great because these are super fast. Ideal for cache. However, a server reboot leads to loosing the cache. File storage is great because persistent after reboot. I would like to combine the best of both worlds. **Describe the solution you'd like** Persistent ramdisk. **Describe alternatives you've considered** There are a lot attempts on persistent ramdisks but none of the projects I found seems to be either still maintained / stable enough for use in production / available from distribution package sources. * https://github.com/patrikx3/ramdisk * https://wiki.debian.org/SSDOptimization#Persistent_RAMDISK * https://thelinuxexperiment.com/persistent-ramdisk-on-debain-ubuntu/ * https://github.com/graysky2/anything-sync-daemon * https://github.com/wor/goanysync * https://web.archive.org/web/20191005110948/https://blog.jmdawson.co.uk/persistent-ramdisk-on-debain-ubuntu/ * https://gitlab.com/jamesdawson1995/persist * https://github.com/graysky2/profile-sync-daemon * https://wiki.archlinux.org/index.php/Profile-sync-daemon There are a lot corner cases such as incomplete snapshot / failed restoration. The one where I am confident that a persistent ramdisk could be implemented is systemd.

github.com/systemd/systemd

constrained system resources program starter wrapper

opened 05:50PM - 14 Jun 20 UTC

adrelanos

RFE 🎁 documentation run

**Is your feature request related to a problem? Please describe.** When runni…ng non-time critical things such as server backups, compression, log analysis it happens that these tools/script eat most (dunno 90-100%) of CPU or disk IO. This needlessly takes away system resources from more important tasks. **Describe the solution you'd like** An easy to use command line tool `limiter` (or so) which starts an application with little system resources. **Describe alternatives you've considered** There's `nice` and `renice` to lower priority of a process, `cpulimit` to lets say 30% maximum, `taskset` to limit to 1 core, `ionice`. Each of these tools has a different syntax. Specifically `cpulimit` seems harder to master. Syntax isn't trivial. Writing this for multiple tasks (on a server) would be a lot work. `nice` alone does not solve it. If I run for example `nice -n19 stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s` on my desktop system, it helps, but it is still less responsive until that process finishes. Would be useful for tasks (such as backups) that require a lot of CPU / IO where it does not matter if these finish in 5 seconds, 5 minutes or 30 minutes. More important is not to take away CPU shares from more important processes. For that purpose I wrote a script. ``` #!/bin/bash ## usage examples: ## ## limiter stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s ## ## sudo limiter stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s #set -x set -e if [ "$(id -u)" != "0" ]; then user_arg=-"-user" fi ## reasons for using some parameters: ## --scope - allows starting GUI applications ## useful to remove for debugging: ## --quiet \ ## useful to add for debugging: ## --unit=testtest \ ## useful for debugging (root): ## sudo systemctl show testtest.scope ## sudo systemctl show testtest.scope | grep CPU ## sudo systemctl status testtest.scop ## useful for debugging (user): ## systemctl --user show testtest.scope ## systemctl --user show testtest.scope | grep CPU ## systemctl --user status testtest.scop systemd-run \ $user_arg \ --quiet \ --same-dir \ --collect \ --nice=19 \ --property=MemoryMax=2G \ --property=CPUAccounting=yes \ --property=CPUQuota=5% \ --property=CPUWeight=1 \ --property=CPUShares=2 \ --property=TasksAccounting=yes \ --property=TasksMax=200 \ --property=IOAccounting=yes \ --property=IOWeight=1 \ --property=IOReadIOPSMax="/dev/disk/ 1K" \ --scope \ -- \ "$@" ``` Problem with that script is that I don't know what I missed, what's redundant, what's coming, etc. Hopefully would be useful enough to have such a tool shipped with systemd.

Patrick · June 20, 2020, 3:05pm

Patrick · December 28, 2020, 12:09pm

github.com/systemd/systemd

systemctl show-environment should not show environment variables of root when run by a non-root user

opened 12:08PM - 28 Dec 20 UTC

closed 12:45PM - 28 Dec 20 UTC

adrelanos

**systemd version the issue has been seen with** `241` **Used distribution…** Debian `buster` **Linux kernel version used** (`uname -a`) **CPU architecture issue was seen on** **Expected behaviour you didn't see** `systemctl show-environment` should not show environment variables of `root` when run by a non-root user. **Unexpected behaviour you saw** `systemctl show-environment` shows environment variables of root when run by a non-root user. Running `systemctl show-environment` should require root/sudo. Environment variables set for `root` are none of the business of non-root users. Although (probably) bad practice, environment variables might contain sensitive information such as passwords or other information leaks useful for a non-root attacker attempting to compromise root. This may be more so the case when using `systemctl import-environment`. **Steps to reproduce the problem** Run `systemctl show-environment` as user.

Patrick · December 29, 2020, 12:22pm

Patrick · January 28, 2021, 5:46am

github.com/QubesOS/qubes-issues

Secure USB Handling during Qubes (Re-)Installation

opened 11:21AM - 27 Jan 21 UTC

adrelanos

T: enhancement C: installer C: doc security P: default C: usb proxy

**The problem you're addressing (if any)** Qubes USB qube is great but protec…tions afforded by it might be invalidated as soon as the user re-installs Qubes using Qubes installer (such as for a major release upgrade). This is because Qubes installer is itself not based on Qubes but Fedora installer. To benefit from Qubes USB qube protections is has to be used always, consistently. Qubes installer does not yet contain any code to confine USB during installation. Qubes documentation currently does not address how USB devices should be handled. Would it be enough to just not connect any USB devices during installation? If so, if that helps, that should be documented. But then there are also non-removable internal USB devices which might have been previously compromised? These shouldn't have access to "dom0", i.e. no linux kernel (by Qubes installer) should load USB drivers on the host while these devices are attached. Even without any internal or external USB devices, is there a threat from having the USB controllers themselves? A previously malicious USB device might have compromised the USB controller firmware? Also in that case it should never again get access to a kernel running directly on hardware, i.e. outside of USB qube. **Describe the solution you'd like** * Make sure kernel does not load USB stack during Qubes installer? * Use something like `rd.qubes.hide_all_usb` during Qubes installer? (After asking the user.) * Perhaps a Qubes installer boot menu option with or without USB? * Document best practices how to deal with USB outside of Qubes such as during Qubes re-installation using Qubes installer or when dual booting. * As for dual booting it might suffice to document that this would mostly invalidate any protections previously afforded by Qubes. **Where is the value to a user, and who might that user be?** Consistent security from USB based threats. **Relevant [documentation](https://www.qubes-os.org/doc/) you've consulted** https://www.qubes-os.org/doc/usb-qubes/

Patrick · January 30, 2021, 5:32am

CVE - critical security bug - Exploitable overflow in Libgcrypt 1.9.0

Patrick · February 16, 2021, 2:59pm

github.com/QubesOS/qubes-issues

Qubes VMs ignore /etc/environment.d drop-in folder

opened 02:58PM - 16 Feb 21 UTC

adrelanos

T: bug C: Debian/Ubuntu P: default needs diagnosis affects-4.2

**Qubes OS version** R4 **Affected component(s) or functionality** Debian t…emplate. Possibly other templates and dom too, untested. **Brief summary** Qubes VMs ignore /etc/environment.d drop-in folder. **How Reproducible** Always reproducible. **To Reproduce** Steps to reproduce the behavior: 1. Create a file `/etc/environment.d/50_user.conf` with contents `testvar=x` in TemplateVM. 2. Restart TemplateVM. 3. `env | grep testvar` **Expected behavior** `env | grep testvar` showing `testvar=x`. **Actual behavior** `env | grep testvar` not showing `testvar=x`. **Additional context** `systemctl --user show-environment | grep testvar` is functional, shows `testvar=x`. `/etc/environment` also functional. This is functional in plain Debian (outside of Qubes). https://www.freedesktop.org/software/systemd/man/environment.d.html **Relevance** Fixing this would allow to stop (ab)using [1] `/etc/profile.d`and [2] `/etc/X11/Xsession.d` drop-in folders, which are currently used by Qubes, Whonix, Kicksecure, security-misc, and whatnot. Sometimes even the same environment variables are set in the in both [1] and [2] which is needless duplication. `/etc/environment.d` might also be covering more places than just [1] and [2]. I.e. systemd daemons. For example, `snapd` is also using that mechanism: cat /usr/lib/environment.d/990-snapd.conf > PATH=$PATH:/snap/bin Other applications are also using it. All related folders: * `~/.config/environment.d/*.conf` * `/etc/environment.d/*.conf` * `/run/environment.d/*.conf` * `/usr/lib/environment.d/*.conf`

Patrick · March 8, 2021, 12:46pm

FlaggedRevs: reviewAllPages.php CommentStore.php: $row does not contain fields needed for comment rev_comment
https://phabricator.wikimedia.org/T276782

Patrick · April 3, 2021, 4:35pm

github.com/QubesOS/qubes-issues

fix Qubes source code copyright / licensing declaration, machine readable copyright, use SPDX License Identifier

opened 01:19PM - 30 Mar 21 UTC

adrelanos

T: bug C: other project management P: default legal

There are various issues with Qubes source code licensing declaration. * Each… source file needs a copyright header. I've asked FSF about that ~9 years ago by private e-mail and they said yes. Nowadays FSFE is [recommending](https://reuse.software/tutorial/#step-2) the same. Also GPL-2 license file is saying that. [1] * `rpm_spec/package-name.spec`(`.in`) is using `License: GPL` instead of `License: GPLv2+`. * `GPL` is inot a valid `license Short License identifier` as per Fedora licensing guidelines ("RPM Spec"). Quote https://fedoraproject.org/wiki/Packaging:LicensingGuidelines#License:_field * > The License: field must be filled with the appropriate license Short License identifier(s) from the "Good License" tables on the [Fedora Licensing page](https://fedoraproject.org/wiki/Licensing:Main?rd=Licensing). * Each file `debian/copyright` must be duplicated to file `COPYRIGHT` in repository root file. (I'd suggest having a test that makes sure that these files always match.) ([Must not be a symlink.](https://github.com/Debian/lintian/blob/master/checks/source-copyright.desc#L11)) https://www.qubes-os.org/doc/license/ is not defining the copyright for each Qubes source file either. It says nothing about the source code for repositories by Qubes. No need to take my word for it. You could try e-mailing `licensing@fsf.org` (was helpful for me years ago) for confirmation or see https://reuse.software/ by FSFE. ---- On Debian, for Debian packages the linter tool `lintian --pedantic` is helpful to identify source code licensing / not machine readable issues. ---- [1] Quote GPLv2 license: (File `/usr/share/common-licenses/GPL-2` on Debian systems.) ``` How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. ```

Patrick · April 7, 2021, 6:26pm

Patrick · May 17, 2021, 7:44am

Patrick · May 17, 2021, 8:11am