Update Git author script

During the svn => git sync process we rewrite the author using a bash script that fetches the user details. This is get-author.sh in the git home directory on svn servers.

Due to character sets, it appears that non-latin names are being corrupted.
For example:

$ git clone git://develop.git.wordpress.org/
$ cd develop.git.wordpress.org
$ git show f047b94d71e780cbd7595047f28a644955d35fff | head -n3
commit f047b94d71e780cbd7595047f28a644955d35fff
Author: Greg Ziółkowski <gziolo@git.wordpress.org>
Date:   Fri Apr 21 10:41:58 2023 +0000

Greg Ziółkowski should be Greg Ziółkowski as shown on his profile.

The SQL used for this is CONCAT(display_name, '|', user_nicename). I can’t test it as I don’t have mysqlMySQL MySQL is a relational database management system. A database is a structured collection of data where content, configuration and other options are stored. https://www.mysql.com/. tools on my sandbox, but I suspect either

  • Character sets need to be specified on the mysql command, I suspect either --default-character-set=latin1 or --default-character-set=utf8mb4 would work.
  • The above concat should do some character-set conversions; I think CONCAT( CONVERT( CAST( CONVERT( display_name USING latin1) AS BINARY) USING utf8), ‘|', user_nicename) would work.

To duplicate it, you should be able to run this on the svn host:
get-author.sh gziolo

Note: You can likely remove the 2015-era logging/debugging from the file at the same time, I don’t recall the outcomes of that, but I suspect it was long fixed.. Review the logs I guess!

Let me know if you’d like me to test or debug anything.

cc @dmsnell @gziolo (Apologies for the months long delay!)
#prio2 #git #svn

Update plugins.trac templates.

Similar to https://make.wordpress.org/systems/2023/02/06/update-plugins-trac-templates/ can we please have svnup-meta-checkouts.sh run on svn2?

Could we also consider adding it to a cron task that runs daily on svn2? (svn1 does not need this)

On svn1 it runs whenever a change to the paths it checks out is modified.

#trac #svn #prio3

GitHub SSH host key update – SVN-Git sync

https://github.com/WordPress/wordpress-develop and the Security mirror aren’t being pushed when a SVN commit is made – this started happening when GitHub rotated it’s host key.

https://github.com/WordPress/wordpress-develop is still receiving commits, as GitHub is pulling them from git://develop.git.wordpress.org/ every 15-20 minutes, but we need to push them in order for GitHub actions to work.

https://github.com/WordPress/WordPress has also ceased syncing, and it appears not to be managed by Systems, and likely to be managed by @markjaquith who has the same GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ host key issue. We should migrate this from Mark account to something Systems managed. I’m not sure how best to do that however, as the GitGit Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. Git is easy to learn and has a tiny footprint with lightning fast performance. Most modern plugin and theme development is being done with this version control system. https://git-scm.com/. commit hashes do not match git://core.git.wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org//, it uses a different branch naming (master, and branch-X.Y)

cc @sergeybiryukov

#svn #git #github #prio1

Update plugins.trac templates

Can we please update the plugins.tracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. checkouts?

On svn1.ord, various meta checkouts are updated via svnup-meta-checkouts.sh, but this doesn’t occur on svn2.ord, as a result, plugins.trac.wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ has the old WordPress.org headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. & Footer trac templates and now appears visually broken.

This should be as simple as running svnup-meta-checkouts.sh on svn2.. I don’t imagine it would cause any breakage, as the meta checkouts are in use on all other trac instances without issue.

Ref https://github.com/WordPress/wporg-mu-plugins/issues/326 and many others

#trac #svn #prio3

IP blocked from plugins.svn?

Per https://wordpress.slack.com/archives/C02QB8GMM/p1666717073063339 it looks like two wordfence IPs have been blocked from plugins.svn.wordpress.org – other svn.wordpress.org hosts are accessible.

It appears they had an automated release process go wrong, causing the IPs to be blocked.

See the above SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. thread for IPs and backstory.

cc @wfscottb

#svn #prio2

Theme Directory changes

The WordPress.org theme directory will now be accepting direct SVN access for theme updates. This means there’s two changes we need made on the systems side.

Everything else will remain the same for now, we’ll continue to accept ZIP uploads and apply the same theme checks on import. Initial theme submissions will continue to be ZIP-upload only, only updates will be available through SVNSVN Apache Subversion (often abbreviated SVN, after its command name svn) is a software versioning and revision control system. Software developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS). WordPress core and the wordpress.org released code are all centrally managed through SVN. https://subversion.apache.org/..

Dynamic SVN Auth file

The SVN auth file will need updated, similar to plugins:

  • Create a copy of update-wp-pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party-auth.sh & setup the cron task for it.
  • SVN Auth file is available at https://wordpress.org/themes/wp-json/themes/v1/svn-auth
  • Use Authorization: BEARER $THEME_SVN_AUTH_BEARER_TOKEN as defined in secrets.php

The file is currently minimal and only outputting a few lines, but will be ~25k lines when it’s opened to theme authors.
This can be tested by editing wp-content/plugins/theme-directory/rest-api/class-internal.php to either bypass auth temporarily or to output all authorship rules.

pre-commit ruleset for SVN commits

In order to prevent having to rebuild a bunch of things that are reliant upon themes.svn, we’d like to add some pre-commit rules to force commits to be in a specified format, rather than mostly free-form as plugins.svn is.

Rules:

  • Are always in a /Theme-Slug/1.2.3/ folder
  • Is a newer version than currently live
  • Does not alter existing versions (treat them as tags)
  • The /1.2.3/ folder matches the style.css Version:1.2.3 headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. value

I’ve put together some rules that work in my testing, but I’m unsure if there’s a different way you’d like to implement it. These were tested against my own SVN server mimicking svn.wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ configuration.

#prio1, #svn, #themes

Core build sync node version selection

As part of #52341-core there’s some work being done to standardise the build configuration for WordPress releases, but one of the changes is likely to cause breakage with the develop.svn -> core.svn sync.

Currently package.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. contains (for example) { engines: { node: "1.2.3" } } however the field should ideally contain >=1.2.3.
Due to the way that the nodeVersion switching code works though, I believe it’ll fail to find node ">=1.2.3" and then fail to find node ">=1*" not taking into account the version qualifiers.
See https://github.com/WordPress/wordpress-develop/pull/897/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R9-R12

While not an ideal way to parse the field, altering nodeVersion() {} in develop-core-svn-sync-config.sh like so should make this more reliable and work well enough until this can be replaced.

 nodeVersion() {
 	# Old branches don't have a node version defined
 	if [ "$1" = 'null' ]; then
 		NODE_VER=0.10.5
 	else
 		NODE_VER=$1
 	fi
+
+	# Remove any non-numeric characters and version qualifiers (>=1.2.3)
+	NODE_VER=$( echo "$NODE_VER" | grep -oE '[0-9.]+' | head -n1 )
+
 	echo Switching nodejs to $NODE_VER

This has been tested by copying it to a new file, and running with the following output:

$ cat test.sh && echo ----- && ./test.sh
#!/bin/sh

nodeVersion() {
        # Old branches don't have a node version defined
        if [ "$1" = 'null' ]; then
                NODE_VER=0.10.5
        else
                NODE_VER=$1
        fi

	NODE_VER=$( echo "$NODE_VER" | grep -oE '[0-9.]+' | head -n1 )

	echo $NODE_VER
}


nodeVersion "1.2.3"
nodeVersion ">=0.1.2"
nodeVersion ">=1.2.3 <2.0.0"
-----
1.2.3
0.1.2
1.2.3

After it’s standardised, I believe the build team intends to look at self-containing the build process so it’s less tied to the system it’s being built on.

cc @desrosj

#svn #build #prio1

#52341-core

Removing the PHP 5.4 plugin directory linting The…

Removing the PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. 5.4 pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party directory linting

The plugin directory has linting against PHP 5.4 for quite some time now. It used to be a feature to prevent accidental bad commits, but today it’s now regarded by many as a bug.

We need to do one or more of the following:

  1. Upgrade the PHP linting to PHP-latest-stable (7.0)
  2. Upgrade the PHP linting to PHP 5.6 with a plan to move to PHP 7.0 (latest stable) and keep up with future releases
  3. Switch PHP linting to being a warning instead of a blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.
  4. Remove the linting all together

Simply upgrading the linting to PHP 5.6 only kicks the can down the road, and still blocks people from using PHP 7 syntaxes in plugins (which is totally okay, if the plugin specifies that as a requirement).
Removing the linting seems like a bad idea, as I believe authors should still be notified they’re committing code that may not be compatible (Unfortunately we currently don’t do php fatal error prevention checks on plugin upgrades, so I’d like to retain it to prevent an unexpected WSOD).

So, I personally feel we should leave the linting at PHP 5.4 (which is what the majority of WordPress sites run today) and make it a warning, not a block.

In order to warn instead of outright blocking, the linting needs to be moved from the pre-commit hook to the post-commit hook and the script exiting with a error-level code, STDERR will then be sent to the client after the commit is made.

#plugins-svn, #svn

Hi could I have commit access to https…

Hi, could I have commit access to https://meta.trac.wordpress.org/browser/sites/trunk/wordpress.org/public_html/wp-content/themes/pub for #475-meta?
Thanks!

#meta, #svn

Could I please have commit access to http…

Could I please have commit access to https://meta.svn.wordpress.org/sites/trunk/wordpress.org/public_html/style in order to fix #555-meta?

#permissions, #svn