abolishing Whonix phabricator issue tracker, moving issue tracking to forums, migrating phabricator.whonix.org to forums.whonix.org

Patrick · December 19, 2023, 12:28pm

Redirections would be even better!

How many tickets we got approximately? 1000(s)? Can nginx handle such a large number of redirections without performance penalty? Any experience with that? I guess if it is an issue a separate nginx instance could be run?

Mycobee · December 19, 2023, 2:29pm

Just under 1000. 950ish. Cant remember the exact number.

The more I think about it, this is the simplest solution I can come up with

Query the mediawiki API and search for all occurrences of phabricator links.
Use the mediawiki api to programatically change all phabricator links to discourse links to the same issue.
Delete phabricator

It seems logical to me. Way better than fuckin with 1000 nginx redirects.

Although, there may be a clever way to send phabricator links to discourse with a single nginx regex. Which would be the simplest solution.

Patrick · December 19, 2023, 3:11pm

That is also nice. However there’s also links in the forums to phabricator or external links.

That would be super cool.

https://phabricator.whonix.org/T941 for example trying to guess the forums URL:
https://forums.whonix.org/t/lock_down_interpreters_/_compilers_(interpreter_lock)_(compiler_lock)

But might be different due to special characters, / and (. Even if the forums URL could be predicted I don’t see how T941 has any resemblance to the forums URL. Except…

If originally the forum topic is gets title T941.

Discourse will make this complicated:

rejects too short titles
does not allow all uppercase even T123456678901234567890 was rejected

This could be solved by reconfigure discourse if possible or using compliant topic titles instead.

T941 phabricator migrated ticket

But aren’t these horrible topic titels? Yes. But these can be renamed in a second step. Discourse internally creates reliable redirects. We’d have a double redirect but that is not so bad.

With that such a regex would be possible. Because there is a pattern here:

https://phabricator.whonix.org/T941
https://forums.whonix.org/t/T941_phabricator_migrated_ticket

But not sure it’s worth all of this extra complexity. Create topic + rename might be extensive development effort.

Mycobee · December 19, 2023, 5:07pm

Rename topics sounds annoying. What if we put the ticket ID in the discourse topic

Then we created a rewrite rule to send the ticket id in the url to a discourse search query

phabrictor.whonix.org/t941
https://forums.whonix.org/search?context=topic&context_id=7112&q=t941&skip_context=true

This seems like a simple enough rewrite. It should come up with the right ticket presuming it exists. If something is wrong, still a valid link

Not sure what context id means in the search. Also dont know whether or not we can assume it will never change

Mycobee · December 19, 2023, 5:13pm

Another possibility is a simple ruby program that does dynamic redirects.

Have a k:v map,

{
  "T941": " forum link to topic",
  "T325": " another link",
   # etc etc
}

Then just reroute. Pass everything that matches phabricator.* upstream to the ruby app

Patrick · December 21, 2023, 11:49am

True.

That is okish.

Also really good!

Did you ever redirect nginx to a ruby app?

Mycobee · December 21, 2023, 2:12pm

Yes. Usually just proxypass it to the port ruby is running on

Mycobee · December 28, 2023, 6:06pm

@Patrick for the Discourse API key, here is how you can send it to me

Almost ready to migrate

Patrick · December 29, 2023, 2:03pm

Done.

Sent by e-mail.

(Had to use gpg command line because Thunderbird refuses to use gpg keys without e-mail address.)

Mycobee · December 31, 2023, 7:20am

Okay I have some housekeeping tasks for you @Patrick

Lets get these three issues out of “review” and choose a better status for them, so they can align with our existing tag structure in Discourse

⚓ T902 disable removable drives auto-mounting - XFCE only
⚓ T551 enter Whonix firewall timesync-fail-closed mode before suspend / enter Whonix firewall full mode after resume and clock-fix
⚓ T492 research / document impact for tunnel users if Tor [exit] relays hosted at the same tunnel provider

We can keep them Open, mark them as resolved, or mark them as will not fix. But lets get rid of review, since we are in the process of reviewing all this shit now

Mycobee · December 31, 2023, 7:21am

Currently there are 208 open issues on phabricator, and 84 open issues on discourse. Will be nice to start mowing through these and culling the backlog. Feeling excited!

Patrick · December 31, 2023, 10:47pm

Done.

Mycobee · February 19, 2024, 1:53am

Okay everything is mostly working, all data is formatting and everything has been setup on the app side. I tried to migrate a few topics though and am getting a 500 error from Discourse.

I am going to need some help on this one @Patrick as I can’t see logs. All I get is a 500 response with nothing useful. I imagine whatever I am doing will trigger a stack trace.

Let me know when you are available and we can slam this out

Patrick · February 19, 2024, 10:59am

related: Forums for bug reporting and the github issue tracker for discussion?

Patrick · February 19, 2024, 10:59am

Any command that I can use to manually reproduce this so I can watch the logs as it happens?

Patrick · February 19, 2024, 4:03pm

NoMethodError (undefined method map' for "status_closed_issue_implemented":String) lib/discourse_tagging.rb:475:in filter_allowed_tags’ lib/discourse_tagging.rb:347:in `filter_tags_violating_one_tag

This?

Mycobee · February 19, 2024, 6:38pm

This shit is done son

Mycobee · February 19, 2024, 10:40pm

Archived the repo so that it is readonly and no confusion about it being maintained any longer

Mycobee · February 19, 2024, 10:43pm

Checklist of things todo for @Patrick

Remove admin access for @phabricator-migrator
Point all phabricator.whonix.org requests in nginx to https://forums.whonix.org/t/why-this-category-exists/19417
Delete phabricator, its database, and remove it from the server

Short and sweet

Mycobee · February 19, 2024, 10:46pm

At some point it might be worth going through all the phabricator tickets tagged status_open_issue_todo and seeing if we want to close any of them, and then deciding which ones we want to actively work