DB stats table repair – new-api-stats

Looks like db2.stats has some corrupt tables, looks like wp_versions and archival_wp_versions are affected.

This has caused WP version stats to cease to operate as expected: https://wordpress.org/about/stats/

I’ve put the stats scripts into maintenance mode to prevent it trying to write to these tables (while still processing autoupdate stats/etc), if I’m not around to do it myself, please execute UPDATE `rev2_ingestion_settings` SET `value` = '0' WHERE `key` = 'maintenance_mode'; afterwards to re-enable it, if possible.

#prio1

Patterns API: maybe cache/direct to proper upstreams

https://api.wordpress.org/patterns/1.0/ will start to receive increased GET API requests following Gutenberg PR 28800 being merged, and ultimately being used in WordPress 5.8.

I’m unsure what the request load will be to this endpoint, or how varied the requests will be at this stage, so just flagging this early for attention.

Currently this APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. endpoint is handled by the catchall API upstream pool, it might make sense to shift this into the coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. or plugins/themes API pools.

The API currently assumes that it’ll be cached by nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/. (which it’s not), and loads WordPress on every API request. No caching is included at present, other than standard WordPress object caching.
The API only accepts GET requests.

Requests to this endpoint will be all GET and potentially of high volume with similar cacheable responses:

  • GET https://api.wordpress.org/patterns/1.0/?pattern-keywords=11
  • GET https://api.wordpress.org/patterns/1.0/?pattern-categories=1234
  • GET https://api.wordpress.org/patterns/1.0/?search=gallery
  • GET https://api.wordpress.org/patterns/1.0/?pattern-keywords=11&search=gallery
  • GET https://api.wordpress.org/patterns/1.0/?pattern-categories=1234&search=instagram

cc @iandunn

#prio1

Vary nginx cache by wporg_locale cookie value

This cookie gets set by the locale-detection pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party, which is used on the login site, and soon on the Learn WordPress site. In the case of Learn, the value of this cookie will determine the language that is displayed in the front end UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. of the site (see this PR). Caching the pages without consideration of the chosen locale has a lot of potential for bad UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it..

Could we update the nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/. config to take this cookie into account when caching wporg pages? This locale switching functionality is intended for use on various wp.org sites beyond just Learn, so ideally this change would apply for *.wordpress.org

#prio1 because we’d like to get the locale switcher launched on the Learn site soon.

Repair translations table.

Occasionally we’ve been seeing PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. warnings that the translation table is crashed:

Table ‘translate_translations’ is marked as crashed and should be repaired

This seems to have been happening sporadically for the last year or so, while running a translation job however, I ran into this and tracked it down to a range of IDs which appear to be inaccessible:

SELECT * FROM translate_translations WHERE id BETWEEN 43611019 AND 43611031;

Those IDs are accessible, but the 10 between them are not.

Can something be done, even if it’s to repair the table and remove those records in the process.

#prio1

DNS hosting for gutenberg.run

Hi, I’m working w/ @aduth to transfer the gutenberg.run domain to the Foundation. MarkMonitor has initiated the transfer, can you please setup DNSDNS DNS is an acronym for Domain Name System - how you assign a human readable address to a website’s exact numeric coded location (ie. wordpress.org uses the actual IP address 198.143.164.252). hosting on our end, and update the nameservers w/ MM after the domain lands?

I think the only needed record is an A to 138.197.4.192, and the www CNAME.

#prio1 (since the old registrar is currently doing DNS hosting)

Thanks!

Core build sync node version selection

As part of #52341-core there’s some work being done to standardise the build configuration for WordPress releases, but one of the changes is likely to cause breakage with the develop.svn -> core.svn sync.

Currently package.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. contains (for example) { engines: { node: "1.2.3" } } however the field should ideally contain >=1.2.3.
Due to the way that the nodeVersion switching code works though, I believe it’ll fail to find node ">=1.2.3" and then fail to find node ">=1*" not taking into account the version qualifiers.
See https://github.com/WordPress/wordpress-develop/pull/897/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R9-R12

While not an ideal way to parse the field, altering nodeVersion() {} in develop-core-svn-sync-config.sh like so should make this more reliable and work well enough until this can be replaced.

 nodeVersion() {
 	# Old branches don't have a node version defined
 	if [ "$1" = 'null' ]; then
 		NODE_VER=0.10.5
 	else
 		NODE_VER=$1
 	fi
+
+	# Remove any non-numeric characters and version qualifiers (>=1.2.3)
+	NODE_VER=$( echo "$NODE_VER" | grep -oE '[0-9.]+' | head -n1 )
+
 	echo Switching nodejs to $NODE_VER

This has been tested by copying it to a new file, and running with the following output:

$ cat test.sh && echo ----- && ./test.sh
#!/bin/sh

nodeVersion() {
        # Old branches don't have a node version defined
        if [ "$1" = 'null' ]; then
                NODE_VER=0.10.5
        else
                NODE_VER=$1
        fi

	NODE_VER=$( echo "$NODE_VER" | grep -oE '[0-9.]+' | head -n1 )

	echo $NODE_VER
}


nodeVersion "1.2.3"
nodeVersion ">=0.1.2"
nodeVersion ">=1.2.3 <2.0.0"
-----
1.2.3
0.1.2
1.2.3

After it’s standardised, I believe the build team intends to look at self-containing the build process so it’s less tied to the system it’s being built on.

cc @desrosj

#svn #build #prio1

#52341-core

Remove Support Forum bbpress 1 rules

As of r16878-dotorg and r16879-dotorg the Support forumSupport Forum WordPress Support Forums is a place to go for help and conversations around using WordPress. Also the place to go to report issues that are caused by errors with the WordPress code and implementations. bbPressbbPress Free, open source software built on top of WordPress for easily creating forums on sites. https://bbpress.org. 1 external has been removed.

Similarly to https://make.wordpress.org/systems/2021/01/11/remove-ideas-rewrite-rules-files/

Can these rules be altered in wporg-web’s wporg-wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ ?

  • location = /support/search.php {} can be removed
  • location = /support/register.php {} can be removed
  • location = /support/bb-login.php {} should be able to be removed. I haven’t seen anything linking to it, but it works and leaving it won’t harm anything other than being an extra nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/. rule.
  • In the location /support/ {} blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.:
    • #Profiles (Handled by bbPress1) needs to be removed
    • #bbPress Admin needs to be temporarily accessible. needs to be removed
  • Please ensure that public_html/support is removed from all production hosts (it should be)
  • Please remove public_html/support from sandboxes. The commits above removed it on production hosts, but not sandboxes.

Previously: https://make.wordpress.org/systems/2015/01/22/i-have-an-nginx-fix-for-a-fun/

See https://meta.trac.wordpress.org/ticket/5384 & https://meta.trac.wordpress.org/ticket/4135

#prio1

Remove Ideas rewrite rules (& files)

As of r16852-dotorg & r16853-dotorg the ideas forum external has been removed from WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/.

Can these rules be altered in wporg-web’s wporg-wordpress.org ?

  • Remove the location /ideas/ {} blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.
  • Remove rewrite ^/ideas/.*$ from @404-extend.
  • Remove rewrite ^/extend/ideas(/.*)?$ ...
    (That should remove all references to /ideas from server configs)
  • Verify that public_html/ideas/ no longer exists on disk on web nodes (it shouldn’t, due to the double-deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors.)
  • Remove public_html/ideas/ on sandboxes (svn up not needed, svn will gracefully update with that folder removed, but will not remove it, although it won’t error out, optional)

/ideas/ and /extend/ideas/ will be handled by a redirect in PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. (r10567-meta) and is temporarily also being handled by r16851-dotorg until these rewrite rules are removed.
If wanted, a server-level redirect for /ideas/ and /extend/ideas/ to https://wordpress.org/support/forum/requests-and-feedback/ can be added, but I don’t think it’s warranted.

#prio1

Could I get a deploy…

Could I please get a deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. on the #codex?

RevisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. [10700:10705] have added canonical and noindex tags, better 404 titles, and most importantly removed the permissions to create new Codex pages by regular users as it’s no longer being used for that (other than by spam bots). These changes were tested.

If the cache could also be cleared to remove the global survey r16681-dotorg and the above additions from the caches that would also be great.

#prio1

Change master branch name on GitHub wordpress-develop security mirror to public-trunk

We are now running automated tests on coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. using the GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ Actions runner with the goal of transitioning off of Travis; however, we need to both reduce the runs because of usage limitations on private repos and also because it’s unnecessary to run all those tests on the same code content between mirrors all the time.

To that end, I am requesting that trunk (the master branch in the git.wp.org mirror) be synced to public-trunk on the GitHub wordpress-develop security mirror instead of master.

We are not close to moving off of Travis entirely yet but if this can be fit in sooner than later, that would be helpful so we can continue work on some related testing setups.

#prio1