Ah, right. Looks like okular uses abstractions/user-download which seems unneccessarily strict from a usability point-of-view.
My own solution was this:
## Read-access to any non-dot file in /home/user/
## not contained in a /home/user/dot directory or private-files-strict
YMMV: I only use gwenview and okular as viewers, not editors (no write permissions).
EDIT: Don't do this. See below.