miscellaneous development log

Patrick · April 17, 2020, 5:56pm

connectors settings page blank after adding gnusocial

opened 07:21PM - 12 Apr 20 UTC

closed 05:54PM - 25 Apr 20 UTC

Bug Addons Needs More Info

### Expected behavior Show settings page ### Actual behavior blank (whi…te) website ### Steps to reproduce the problem go to https://meld.de/settings/connectors Consumer key Consumer secret Base API URL:https://gnusocial.cc/api/ ### Friendica version you encountered the problem https://meld.de/profile/whonix

Patrick · May 2, 2020, 1:29pm

https://discourse.phabricator-community.org/t/phabricator-file-is-huge-whats-inside-how-to-shrink/3823

Patrick · May 2, 2020, 1:35pm

github.com/friendica/friendica

connectors settings page blank after adding gnusocial

opened 07:21PM - 12 Apr 20 UTC

closed 05:54PM - 25 Apr 20 UTC

adrelanos

Bug Addons Needs More Info

### Expected behavior Show settings page ### Actual behavior blank (whi…te) website ### Steps to reproduce the problem go to https://meld.de/settings/connectors Consumer key Consumer secret Base API URL:https://gnusocial.cc/api/ ### Friendica version you encountered the problem https://meld.de/profile/whonix

github.com/friendica/friendica

Can't login to your Diaspora account. Please check handle (in the format user@domain.tld) and password.

opened 08:54AM - 29 Apr 20 UTC

closed 03:16PM - 07 May 20 UTC

adrelanos

Addons Support Request

### Expected behavior functional diaspora export ### Actual behavior > …Can't login to your Diaspora account. Please check handle (in the format user@domain.tld) and password. ### Steps to reproduce the problem go to https://meld.de/settings/connectors go to diaspora export check Enable Diaspora Post Addon enter Diaspora handle enter Diaspora password check Post to Diaspora by default save settings ### Friendica source (git, zip) https://meld.de/profile/whonix //cc @copiis

Patrick · May 7, 2020, 8:11pm

github.com/friendica/friendica

gnusocial export stopped working

opened 08:05PM - 07 May 20 UTC

adrelanos

### Expected behavior post export to gnusocial ### Actual behavior post… not exported to gnusocial ### Steps to reproduce the problem set up gnu social export > Currently connected to: whonix check `Allow posting to GNU Social` check `Send public postings to GNU Social by default` save settings has worked before. https://meld.de/display/a8e4f565-145e-af1b-bac2-7a3576178511 -> https://gnusocial.cc/notice/1228585 OK but https://meld.de/display/a8e4f565-175e-b369-c9ac-f64047269530 or https://meld.de/display/a8e4f565-125e-b42a-e775-2a9559842554 was not forwarded to gnusocial. Didn't touch any gnusocial settings in meanwhile since I was happy that it was functional. ### Friendica version you encountered the problem https://meld.de/profile/whonix //cc @copiis

github.com/friendica/friendica

diaspora export not working

opened 08:11PM - 07 May 20 UTC

adrelanos

### Expected behavior post export to https://diasp.org/u/whonix ### Actual… behavior post not exported to https://diasp.org/u/whonix ### Steps to reproduce the problem Cet up diaspora export. Can see: > Connected with your Diaspora account whonix@diasp.org check `Enable Diaspora Post Addon` post to aspect `All Aspects` check `Post to Diaspora by default` save settings I then posted on twitter which was correctly imported as https://meld.de/display/a8e4f565-125e-b42a-e775-2a9559842554 but not exported to https://diasp.org/u/whonix ### Friendica version you encountered the problem https://meld.de/profile/whonix //cc @copiis

Patrick · May 22, 2020, 5:17pm

Patrick · May 26, 2020, 1:56pm

https://meta.discourse.org/t/print-long-topic-to-pdf-redux-again/44639/62

Patrick · May 26, 2020, 3:47pm

Dev/About Infrastructure - Kicksecure

Patrick · May 30, 2020, 7:06pm

please add openssl-rsync script for use of encrypted rsync over TLS

Patrick · May 31, 2020, 11:51am

github.com/SpiderLabs/ModSecurity-nginx

lua-resty-waf - lua based WAF - performance similar to cloudflare WAF

opened 11:49AM - 31 May 20 UTC

closed 12:32AM - 08 Aug 20 UTC

adrelanos

question no-issue-activity

Are you aware of [lua-resty-waf](https://github.com/p0pr0ck5/lua-resty-waf) by @…p0pr0ck5? A LUA based WAF. > ## Performance > > lua-resty-waf was designed with efficiency and scalability in mind. It leverages Nginx's asynchronous processing model and an efficient design to process each transaction as quickly as possible. Load testing has show that deployments implementing all provided rulesets, which are designed to mimic the logic behind the ModSecurity CRS, process transactions in roughly 300-500 microseconds per request; this equals the performance advertised by [Cloudflare's WAF](https://www.cloudflare.com/waf). Tests were run on a reasonable hardware stack (E3-1230 CPU, 32 GB RAM, 2 x 840 EVO in RAID 0), maxing at roughly 15,000 requests per second. See [this blog post](http://www.cryptobells.com/freewaf-a-high-performance-scalable-open-web-firewall) for more information. > > lua-resty-waf workload is almost exclusively CPU bound. Memory footprint in the Lua VM (excluding persistent storage backed by `lua-shared-dict`) is roughly 2MB. Cloudflare WAF is LUA based. https://blog.cloudflare.com/cloudflares-new-waf-compiling-to-lua/ Unfortunately lua-resty-waf hasn't seen development (git commits) since 2018. Would it make sense to rebase ModSecurity-nginx on lua-resty-waf? Consider hire p0pr0ck5 if he's not hired by some CDN / WAF company yet? :) I am writing this on my own initiative. Didn't talk to p0pr0ck5.

Patrick · June 2, 2020, 4:08pm

Patrick · June 8, 2020, 12:21pm

mediawiki

Patrick · June 8, 2020, 7:09pm

github.com/apache/incubator-pagespeed-ngx

document gzip, gzip_static, brotli and brotli_static

opened 07:09PM - 08 Jun 20 UTC

adrelanos

Documentation is at the moment a bit sparse on [nginx gzip](https://nginx.org/en…/docs/http/ngx_http_gzip_module.html). Specifically nginx [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static). It is my understanding that gzip_static cannot be easily used in conjunction with nginx pagespeed. Source: testing myself and [this blog post](https://www.getpagespeed.com/server-setup/varnish-ssl-brotli-gzip-pagespeed-putting-things-together). I guess the way the architecture works is that nginx pagespeed needs/fetches the resource without compression, optimizes it and then adds its own dynamic gzip compression? The gzip compression might even be static once nginx pagespeed has cached the resource? Perhaps worth mentioning in pagespeed documentation that [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static) is not worth bothering with when using nginx pagespeed? [This blog post](https://www.getpagespeed.com/server-setup/varnish-ssl-brotli-gzip-pagespeed-putting-things-together) indicates that it's possible to get quasi gzip static. Perhaps you could shed some light on that? Also can't find any mentions of [brotli](https://github.com/google/brotli) / brotli_static. related google search terms: > site:https://www.modpagespeed.com gzip > site:https://www.modpagespeed.com gzip_static > site:https://www.modpagespeed.com brotli > site:https://www.modpagespeed.com brotli_static

Patrick · June 9, 2020, 12:10pm

github.com/jthingelstad/foreground

caching issue: static CSS is load thorugh load.php and therefore uncacheable

opened 12:10PM - 09 Jun 20 UTC

closed 03:19PM - 14 Dec 20 UTC

adrelanos

non-foreground issue

### Setup - Browsers and versions: firefox-esr 68.8.0esr-1~deb10u1 - MediaWi…ki version: 1.34.1 - Database version: - PHP version: - Foreground version: 2.3.0 ### Issue gtmetrix.com, google pagespeed and other website speedtest / caching test websites complain that the following resources do not have a long expiration date / aren't cacheable: * /w/load.php?lang=en&modules=ext.flaggedRevs.review&skin=foreground&version=18tly * /w/load.php?lang=en&modules=ext.purge&skin=foreground&version=3sx9r * /w/load.php?lang=en&modules=jquery.client&skin=foreground&version=1d1dj * /w/load.php?lang=en&modules=jquery&skin=foreground&version=1xgm5 * /w/load.php?lang=en&modules=jquery.getAttrs&skin=foreground&version=1p30m * /w/load.php?lang=en&modules=jquery.highlightText&skin=foreground&version=1mh8h * /w/load.php?lang=en&modules=jquery.makeCollapsible&skin=foreground&version=rscou * /w/load.php?lang=en&modules=jquery.suggestions&skin=foreground&version=1ucip * /w/load.php?lang=en&modules=mediawiki.String&skin=foreground&version=1bidf * /w/load.php?lang=en&modules=mediawiki.base&skin=foreground&version=m9box * /w/load.php?lang=en&modules=mediawiki.cldr&skin=foreground&version=laa59 * /w/load.php?lang=en&modules=mediawiki.libs.pluralruleparser&skin=foreground&version=ckbfi * /w/load.php?lang=en&modules=mediawiki.page.startup&skin=foreground&version=5umqm * /w/load.php?lang=en&modules=mediawiki.Title&skin=foreground&version=sx80d * /w/load.php?lang=en&modules=mediawiki.searchSuggest&skin=foreground&version=1ecdy * /w/load.php?lang=en&modules=mediawiki.storage&skin=foreground&version=1dq8q * /w/load.php?lang=en&modules=mediawiki.api&skin=foreground&version=6hk5g * /w/load.php?lang=en&modules=site&skin=foreground&version=6dr8h * /w/load.php?lang=en&modules=skins.foreground.modernizr&skin=foreground&version=aptx3 This is because these are fetched through `load.php`. That's probably the reason that these aren't links to `.css` or `.js` files. These files look fairly stable. Is this a mediawiki `load.php` issue or a foreground skin specific issue?

Patrick · June 12, 2020, 4:07pm

github.com/apache/incubator-pagespeed-ngx

document good origin HTTP header settings / nginx config / etags / nginx pagespeed output headers

opened 04:07PM - 12 Jun 20 UTC

closed 09:41PM - 28 Mar 21 UTC

adrelanos

What are some good combinations of HTTP headers? What should users of pagespeed …check their webapps are outputting wrt HTTP headers? Are there some good examples of recommended origin caching HTTP headers? Perhaps there are multiple valid combinations? Or good websites that have good caching settings which are a good role model? Could you please provide a basic nginx config for static contents? There's a lot. Expires; HTTP ETag; weak ETag; vs gzip; last modified. It's still unclear the me which combination of settings / HTTP headers are a good basis for pagespeed to work with. Pagespeed showing an `Etag: W/"0"` being for cachable static files is expected or a bug?

Patrick · June 14, 2020, 5:51pm

github.com/systemd/systemd

persistent ramdisk

opened 04:51PM - 13 Jun 20 UTC

closed 09:15AM - 16 Jun 20 UTC

adrelanos

**Is your feature request related to a problem? Please describe.** Ramdisks /… tmpfs are great because these are super fast. Ideal for cache. However, a server reboot leads to loosing the cache. File storage is great because persistent after reboot. I would like to combine the best of both worlds. **Describe the solution you'd like** Persistent ramdisk. **Describe alternatives you've considered** There are a lot attempts on persistent ramdisks but none of the projects I found seems to be either still maintained / stable enough for use in production / available from distribution package sources. * https://github.com/patrikx3/ramdisk * https://wiki.debian.org/SSDOptimization#Persistent_RAMDISK * https://thelinuxexperiment.com/persistent-ramdisk-on-debain-ubuntu/ * https://github.com/graysky2/anything-sync-daemon * https://github.com/wor/goanysync * https://web.archive.org/web/20191005110948/https://blog.jmdawson.co.uk/persistent-ramdisk-on-debain-ubuntu/ * https://gitlab.com/jamesdawson1995/persist * https://github.com/graysky2/profile-sync-daemon * https://wiki.archlinux.org/index.php/Profile-sync-daemon There are a lot corner cases such as incomplete snapshot / failed restoration. The one where I am confident that a persistent ramdisk could be implemented is systemd.

github.com/systemd/systemd

constrained system resources program starter wrapper

opened 05:50PM - 14 Jun 20 UTC

adrelanos

RFE 🎁 documentation run

**Is your feature request related to a problem? Please describe.** When runni…ng non-time critical things such as server backups, compression, log analysis it happens that these tools/script eat most (dunno 90-100%) of CPU or disk IO. This needlessly takes away system resources from more important tasks. **Describe the solution you'd like** An easy to use command line tool `limiter` (or so) which starts an application with little system resources. **Describe alternatives you've considered** There's `nice` and `renice` to lower priority of a process, `cpulimit` to lets say 30% maximum, `taskset` to limit to 1 core, `ionice`. Each of these tools has a different syntax. Specifically `cpulimit` seems harder to master. Syntax isn't trivial. Writing this for multiple tasks (on a server) would be a lot work. `nice` alone does not solve it. If I run for example `nice -n19 stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s` on my desktop system, it helps, but it is still less responsive until that process finishes. Would be useful for tasks (such as backups) that require a lot of CPU / IO where it does not matter if these finish in 5 seconds, 5 minutes or 30 minutes. More important is not to take away CPU shares from more important processes. For that purpose I wrote a script. ``` #!/bin/bash ## usage examples: ## ## limiter stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s ## ## sudo limiter stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s #set -x set -e if [ "$(id -u)" != "0" ]; then user_arg=-"-user" fi ## reasons for using some parameters: ## --scope - allows starting GUI applications ## useful to remove for debugging: ## --quiet \ ## useful to add for debugging: ## --unit=testtest \ ## useful for debugging (root): ## sudo systemctl show testtest.scope ## sudo systemctl show testtest.scope | grep CPU ## sudo systemctl status testtest.scop ## useful for debugging (user): ## systemctl --user show testtest.scope ## systemctl --user show testtest.scope | grep CPU ## systemctl --user status testtest.scop systemd-run \ $user_arg \ --quiet \ --same-dir \ --collect \ --nice=19 \ --property=MemoryMax=2G \ --property=CPUAccounting=yes \ --property=CPUQuota=5% \ --property=CPUWeight=1 \ --property=CPUShares=2 \ --property=TasksAccounting=yes \ --property=TasksMax=200 \ --property=IOAccounting=yes \ --property=IOWeight=1 \ --property=IOReadIOPSMax="/dev/disk/ 1K" \ --scope \ -- \ "$@" ``` Problem with that script is that I don't know what I missed, what's redundant, what's coming, etc. Hopefully would be useful enough to have such a tool shipped with systemd.

Patrick · June 20, 2020, 3:05pm

Patrick · December 28, 2020, 12:09pm

github.com/systemd/systemd

systemctl show-environment should not show environment variables of root when run by a non-root user

opened 12:08PM - 28 Dec 20 UTC

closed 12:45PM - 28 Dec 20 UTC

adrelanos

**systemd version the issue has been seen with** `241` **Used distribution…** Debian `buster` **Linux kernel version used** (`uname -a`) **CPU architecture issue was seen on** **Expected behaviour you didn't see** `systemctl show-environment` should not show environment variables of `root` when run by a non-root user. **Unexpected behaviour you saw** `systemctl show-environment` shows environment variables of root when run by a non-root user. Running `systemctl show-environment` should require root/sudo. Environment variables set for `root` are none of the business of non-root users. Although (probably) bad practice, environment variables might contain sensitive information such as passwords or other information leaks useful for a non-root attacker attempting to compromise root. This may be more so the case when using `systemctl import-environment`. **Steps to reproduce the problem** Run `systemctl show-environment` as user.

Patrick · December 29, 2020, 12:22pm

Patrick · January 28, 2021, 5:46am

github.com/QubesOS/qubes-issues

Secure USB Handling during Qubes (Re-)Installation

opened 11:21AM - 27 Jan 21 UTC

adrelanos

T: enhancement C: installer C: doc security P: default C: usb proxy

**The problem you're addressing (if any)** Qubes USB qube is great but protec…tions afforded by it might be invalidated as soon as the user re-installs Qubes using Qubes installer (such as for a major release upgrade). This is because Qubes installer is itself not based on Qubes but Fedora installer. To benefit from Qubes USB qube protections is has to be used always, consistently. Qubes installer does not yet contain any code to confine USB during installation. Qubes documentation currently does not address how USB devices should be handled. Would it be enough to just not connect any USB devices during installation? If so, if that helps, that should be documented. But then there are also non-removable internal USB devices which might have been previously compromised? These shouldn't have access to "dom0", i.e. no linux kernel (by Qubes installer) should load USB drivers on the host while these devices are attached. Even without any internal or external USB devices, is there a threat from having the USB controllers themselves? A previously malicious USB device might have compromised the USB controller firmware? Also in that case it should never again get access to a kernel running directly on hardware, i.e. outside of USB qube. **Describe the solution you'd like** * Make sure kernel does not load USB stack during Qubes installer? * Use something like `rd.qubes.hide_all_usb` during Qubes installer? (After asking the user.) * Perhaps a Qubes installer boot menu option with or without USB? * Document best practices how to deal with USB outside of Qubes such as during Qubes re-installation using Qubes installer or when dual booting. * As for dual booting it might suffice to document that this would mostly invalidate any protections previously afforded by Qubes. **Where is the value to a user, and who might that user be?** Consistent security from USB based threats. **Relevant [documentation](https://www.qubes-os.org/doc/) you've consulted** https://www.qubes-os.org/doc/usb-qubes/

Patrick · January 30, 2021, 5:32am

CVE - critical security bug - Exploitable overflow in Libgcrypt 1.9.0