Offline Documentation Discussion

How much worth was the folder created by

?

I hadn’t got it to work yet. Nvevr got past that last error posted. I’m thinking about redoing the entire setup.

[quote=“HulaHoop, post:19, topic:413”]Works fine but gives:

[quote]WARNING: destination directory already exists, skipping initialisation
Creating static HTML dump in directory /home/user/Desktop/backup.
Using database localhost
Starting from page_id 1 of 1074
PHP Fatal error: Cannot access protected property LocalRepo::$thumbScriptUrl in /var/www/w/extensions/DumpHTML/dumpHTML.inc on line 1163
[/quote][/quote]
What did you mean by works fine? Did it complete or break after page id 1? If the result looks fine, we can ignore any warnings.

I meant that using ‘php’ makes the command work, but it results in a fatal error with no output of any HTML files.

user@host:~$ sudo php /var/www/wiki/extensions/DumpHTML/dumpHTML.php -d /home/user/Desktop/backup [sudo] password for user: WARNING: destination directory already exists, skipping initialisation Creating static HTML dump in directory /home/user/Desktop/backup. Using database localhost Starting from page_id 1 of 1159 PHP Fatal error: Cannot access protected property LocalRepo::$thumbScriptUrl in /var/www/w/extensions/DumpHTML/dumpHTML.inc on line 1163

https://lists.wikimedia.org/pipermail/mediawiki-l/2014-August/043248.html

A while back I reported this on the mediawiki mailinglists and they admit its a bug which they now have filed . Other suggestions are to use Kiwix as recommended by me independent of this inquiry.

Necropost update:

Kiwix is removed from Debian and the problems with the offline extension still apply however I found a new extension called “Collection” that you can run on your server and use to generate a help document for offline use in many formats that you then include in Whonix.

if u want help in this i can help with fast internet + fast pc + alot of storage .

not building/scripting but just i can report the issues and/or give trials to any beta ideas if available.

I not mind about necropost at all. If there is an existing topic, why
not continue to use it.

Thanks for the update!

Various wikis have this extension installed. Including wikipedia,
mediawiki and wikivoyage. Yet, I failed to use that extension as a user.

I don’t find the “add to book” button. Can you figure this out?

Here are the updated install instructions:

https://phab.wmfusercontent.org/file/data/4z4wye3wqc5v3j5dxqvx/PHID-FILE-i2bo7oh3zplimvzyrjed/4sj7iigxskwl4r2n/README.rst

Only follow the instructions in the “Installation and Configuration of the Collection Extension” part. You don’t need to configure a render server. A default public rendering server for wikis is defined by default in the extension settings: pediapress.com

Then look at the advanced section for information about the settings. I prefer we enable the text format exporting because its least problematic from a watering hole attack perspective. We can use multiple formats anyway.

File Link Has Expired.

Anyhow. Installation is not the issue for now. Even if it was installed. Please refer to my last post. I wouldn’t know how to use it as a user, as I failed to use this extension on other wikis that already have it installed.

Fixed:

https://phabricator.wikimedia.org/diffusion/ECOL/browse/master/README.rst?view=raw

Ah ok. I found a user support thread that details what they did to get it working. Note that the problem that they complained about was caused by them trying to set this up on a non publc wiki whihc does not apply to Whonix wiki:

What I understand is once its setup a PDF export button appears on the page otherwise there is something preventing this from working and we may have to look at setting up a local render server.

Would it be a wise idea security wise to connect to [a / the] public render server?

Just go to any wiki that has it installed already. For example en.wikipedia.org. Then click “create a book”. The problem is, you won’t find the “add to book” button appear. It should look like this.

But I can’t reliably find that menu then. Just try adding 5 or 10 random pages to such a test book.

Also even if it work work, would it be practical?

Pressing the ‘download as pdf’ button a thousand times?

Going manually through all user documentation and press the ‘Add this page to your book’ button? Doesn’t look like it supports adding a whole category. (category documentation)

Perhaps you have a different scope in mind. I thought the scope of this ticket is to have a whonix-documentation page that is updated at each release. A maintenance burden free process as simple as running some script.

Who did you suppose to run this? Developers at each release? Or users running this manually for pages of their selection?

My goal is to have a whole wiki conversion to an edocument that users can refer to offline and redistribute. There is a lot of useful general information on there besides the Whonix specific parts and it would be a shame to have it rely on the servers being up or very advanced non-practical requirements like settings your own webserver/mediawiki instance just to be able to access this information.

Update:

Limited Extensions (won’t work):

pdfbook - has the limitations you mentioned of converting categories. Nothing more unless you create a category that is a superset of everything else we have.

collection - has the problem of allowing some arbitrary chosen pages to be backed up on demand. No automated whole wiki solution.

pdfexport - very limited. Single page conversion only.

(Potential Solutions) Replacement extension recommended Carbon60: Cloud Consulting - Services and Solutions in place of the defunct DumpHTML one:

What is interesting too: This file was generated in less than 2 weeks
thanks to multiples recent innovations:

  • The Parsoid (cluster), which gives us an HTML output with additional
    semantic RDF tags
  • mwoffliner, a nodejs script able to dumps pages based on the Mediawiki
    API (and Parsoid API)
  • zimwriterfs, a solution able to compile any local HTML directory to a
    ZIM file

Thats a start and a great soltution for the Parsoid output conversion to an ebook format is the pandoc converter package which supports a lot of formats and can be run locally like any other program. Pandoc supports MediaWiki markup: http://pandoc.org

Agreed.

We already do have a category of all documentation.
Category:Documentation - Whonix

[It’s complete. Non missing.]

Does that qualify as category that is a superset of everything else we have? Does that help?

[Also adding all the wanted pages to a new category would be an acceptable one time task.]

[There is also category Development and Design.]

Yes that works great.

Pdfbook gets around of remote file rendering and does it locally too. Up to you but I think the Development and Design should be its separate book for interested would-be devs who want to get to the point.

https://www.mediawiki.org/wiki/Extension:PdfBook

HulaHoop:

Up to you but I think the Development and Design should be its separate book for interested would-be devs who want to get to the point.

Yes, Development and Design does probably not even need its own book. No
need to merge it with Documentation.

PdfBook extension installed.

The following single page pdf download works.

https://www.whonix.org/w/index.php?title=Qubes/Install&action=pdfbook&format=single

works, whole category. It’s by comparison a small category.

https://www.whonix.org/w/index.php?title=Category:Design&action=pdfbook

same for…

https://www.whonix.org/w/index.php?title=Category:Development&action=pdfbook

does not work, whole category.

https://www.whonix.org/w/index.php?title=Category:Documentation&action=pdfbook

Takes a while and the creates a 0 bytes file. Server config, php memory_limit surely has sufficient size. Looks like something hits the timeout after 30 seconds.

Any idea? @fortasse

@Patrick: “https://www.whonix.org/w/index.php?title=Category:Documentation&action=pdfbook” generated me a ~1500 page, 2MB PDF in about 40 seconds. Did you make any change between this post and when I tested just now?

No changes. Now it worked. Created a ~ 14 MB file with 1499 pages pdf file that works.