Support Forum Progress Report

I began backfilling all valid closed support forum topics and replies last Thursday. The current rate of about 500 topics/minute includes all topic tags, replies, and subscription information and discards spam and deleted topics. I’ve outlined the upgrade path with a tentative timeline for this week below.

Pre-maintenance window (this week):

  • Import all valid topics and replies from more than a year ago. Topics are considered valid if they are public, not a plugin or theme review (more on that later), and are not spam or trash. These are all topics that are closed and have no activity. I’m currently at offset 2,197,000 of 2,400,000 of the desired topic set. (@jmdodd)
  • Post a small notification message regarding the maintenance window once it is scheduled at the top of all pages in the support forum. (@jmdodd)

Switch over forum registration to be handled on login.wordpress.org. This will be done after the forum upgrade; “one catastrophe at a time”.

  • Change “topic closed” plugin date to two months rather than one year. Any topics older than two months will be considered closed in bbPress 1. This allows us to import much more of the existing database rather than relying on the maintenance window for all of the import. These topics will still be open once we switch over to bbPress 2. (@jmdodd)
  • Import the last ten months of topics from offset 2,400,000 to a set topic ID outside of the two month window. (@jmdodd)

Maintenance window (about 6 hours, very tentatively set for Friday, 26 August 2016):

  • Close the bbPress 1 forums. They will still be searchable, and users can browse them, but they will not be available for posting new topics or replies. (@jmdodd)
  • Run the import for valid topics and replies from the last two months. Do not include reviews. (@jmdodd)
  • Testing shakedown pre-nginx; make sure that registration/login/posting/etc. are all functional. (@jmdodd)
  • Update the forum site URL. (@jmdodd)
  • Deploy nginx rewrites for the support forums only. Rosetta forums still need nginx rewrites, but in the interests of restoring access to the support forum and not complicating the maintenance window, those can be done in a separate action. (@jmdodd)
  • Testing shakedown post-nginx; make sure that registration/login/posting/etc. are all functional. (@jmdodd)
  • Open all bbPress 2 forums except for reviews. (@jmdodd)

Post-maintenance window (at least 12 hours, hopefully on Saturday, 27 August 2016):

  • User role backfill for any users that are not the default bbPress user role. (@jmdodd)
  • Import all 300k reviews into bbPress 2. Reviews are editable by the poster and involve an external table that holds the actual rating data; the external table will need to be scanned and matched with the new post id of the imported topic. Reviews are also far more dense than normal topics; we gain some efficiency because most reviews have no replies, but no reviews will be discarded. The plugin and theme directories rely on the review data for their current designs; their queries will continue to work from stale pre-maintenance-window data until they are updated to use the new bbPress 2 tables. Doing the review import after the main switchover will allow us to deploy query updates against the plugin and theme directory code as well as the support forum to avoid errors, and reduce the time of the maintenance window from needing to import 360k topics to only needing to guarantee import of about 60k topics. (@jmdodd)
  • Import user favorites and topic, topic tag, and plugin subscriptions. Users will potentially miss some subscription emails during this period, but this should not be updated until all topics are available in the new tables. Plugin subscriptions will be the last data retrieved from the old bbPress 1 tables, as they are unrelated to any topics and will need to be crawled separately on a term-by-term basis. (@jmdodd)

#forums