Sorry, I couldn’t help myself…

In November 2020, I posted about an initiative to update the updaters. It took a bit longer than expected to research the issue and an efficient way to manage the project, but here we are.

Context

The WordPress updaters are a set of PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher classes that assures that users can safely upgrade WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., plugins, themes, and translations.

The ability to manually update WordPress from the adminadmin (and super admin) area, and to install and update plugins and themes, has existed since 2.3 in 2007. Auto-update features were added in WordPress 3.7 (for minor releases), extended in 5.5 (as opt-in for plugins and themes), and 5.6 (major releases). To make the user experience of auto-updates even better, and build trust with users and extenders, it’s important that this mechanism works well and provides all the failsafe checks needed.

The WordPress Core update has proven to be generally reliable, but it doesn’t actually have many tests nor is well documented. There are also some reliability concerns around adding new files and the overall number of changed files, which is the reason WordPress currently tries to keep the number of changed files in minor releases to a minimum.

Plugins and themes updaters are older: in general, all of them can be improved.

Goal

The project goal is to review the existing state of the updaters and propose ways to improve them.

• WordPress
• Core
• Plugins
• Themes

Outcomes

The expected outcomes are:

1. Make sure the zips upload and unpacking are safe.
2. Create a mechanism to upgrade and rollback.
3. Have managed updates (database migrations).
4. Create a unified JSONJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. convention for requirements and dependencies.

Outcome 1 – Make sure the zip uploads and unpacking are safe

With the introduction of auto-updates, these processes run more frequently.. So relatively small issues get triggered more often, and with that become a bigger problem. They also now more often run unattended: an auto-update that breaks could lead to quite problematic results.

The goal here is to address most of the known issues related to:

• downloading and extracting update packages
• copying files and directories
• improving documentation for updates
• adding some automated tests
• making the update process more reliable in general

Related TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. tickets

Edited 2021-05-06T19:38 MDT: added #36710 and #36373 to the list of tickets. @pbiron.

• #14781 Allow upload of PHP files as plugin
• #15134 WordPress should not try to remove theme’s or plugin’s directory recursively if the directory is a symlink
• #17301 Keep the connection open when doing upgrades or long-running operations
• #20944 Changing plugin filename causes Fatal error on automatic activation after update
• #26626 WP_Upgrader::unpack_package() can overflow path name length limits
• #29408 Symlinked themes and plugins should not be updatable
• #33571 Improvements for plugin upgrades when disk space runs out (or other write failure)
• #34676 Optimize bulk plugin updates
• #35217 WP_Upgrader: writes temporary directory name to plugin folder if zip doesn’t have directory
• #36373 Plugin update fails silently when using zipfile without subdirectory
• #38084 Fix Inconsistency Theme and Plugin Update Process When No Package Found
• #44002 fread(): Length parameter must be greater than 0 – Happens on all updates to core, plugins, themes
• #44118 WordPress performs some unnecessary plugin update checks
• #44418 fs_unavailable while auto-updating plugins
• #50349 Plugin Update deletes plugin and showing “could not delete old plugin” message
• #50849 Incorrect action passed to hooks when updating plugin/theme by uploading ZIP file
• #51823 Native auto updater caused plugin update to fail and disappears plugin from directory (Should be fixed with Rollback Update Failure pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party)
• #52342 Update copy_dir and _copy_dir for $dirlist error reporting (this is fixed, but worth revisiting in case if it has dependencies with the other bugs)
• #36710 Symlinked directories should not be deleted recursively
• #36373 Plugin update fails silently when using zipfile without subdirectory

Outcome 2 – Create a mechanism to upgrade and rollback.

When a core auto-update fails, core has long had the ability to automatically attempt a “rollback” to the latest stable release (in the branchbranch A directory in Subversion. WordPress uses branches to store the latest development code for each major release (3.9, 4.0, etc.). Branches are then updated with code for any minor releases of that branch. Sometimes, a major version of WordPress and its minor versions are collectively referred to as a "branch", such as "the 4.0 branch". that the site is running). Note: for core auto-updates, “rollback” is not attempted for certain failures (e.g. “disk full”, etc).

This capability should be extended to be available for plugin and theme updates. Work on some of these issues is already undergoing as part the rollback update feature.

Related Trac tickets

• #37301 A way to test plugin/theme installs and updates before applying
• #49963 Security of failed update/rollback
• #50937 Auto-backup in core
• #51816 Implement auto recovery mode during auto update cron
• #51857 Add rollback for failed plugin/theme updates
• #51928 Provide plugin/theme update failure data to dot org

Outcome 3 – Have managed updates (database migrations)

Plugins should be able to easily run database migrations, so that update and rollback could be performed not only for files but also for data.

Yoast already does this by having a migrations library:

• https://github.com/Yoast/wordpress-seo/tree/trunk/lib/migrations
• https://github.com/Yoast/wordpress-seo/tree/trunk/src/config/migrations

If we had a unified process for data migrations in core, WordPress would be able to run the latest available migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. after an update or rollback:

• without the need to store anything about the previous state
• without requiring anything from the plugin author, apart from adding a “migrations” folder

Relevant Trac tickets

• #32606 Plugin data migration
• #36887 Database upgrades should fail gracefully
• #49188 Add a check for tables structures

Outcome 4 – Create a unified JSON convention for requirements and dependencies.

Different plugins have introduced compatibility tags (For example, Elementor and WooCommerce). In the long run, this could become harder to manage.

Relevant Trac tickets

• #11308 Handling plugin dependencies
• #20241 make it possible for plugins to activate other plugins
• #22316 Plugin Dependencies (Yet Another Plugin Dependencies Ticket)
• #23912 Add Composer package description
• #47285 Better Management of External Asset Dependencies
• #52154 Ability to install site with predefined configuration without Docker or console commands — analogue of package.json for full site installation with exact settings, plugins, themes and their settings

@afragen has written a lightweight library, wp-dependency-installer, that takes a JSON config file and can install a suggested or required plugin dependency. He mentions this as a lighter-weight solution to something like the TGMPA library. This makes it possible to solve many of the issues above.

How to contribute

1. Familiarize yourself with the existing classes, listed above.
2. Update your local WordPress SVN (use svn up) or Git repo (use git pull) to the latest version of WordPress trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision..
3. Pick a ticketticket Created for both bug reports and feature development on the bug tracker. from the above list, review its history, refresh it if needed or make a new patchpatch A special text file that describes changes to code, by identifying the files and lines which are added, removed, and altered. It may also be referred to as a diff. A patch can be applied to a codebase for testing..
4. Upload your patch to the Trac ticket you created, and add the keywords has-patch and needs-testing
5. If during the exploration of the above goals you can think of other enhancements, please create a new ticket and assign it to the Update/Install component

Keeping Discussions Focused

Any discussion about the specifics of a patch itself should happen on Trac. Any discussion about the broader scope of what this feature project is about should take place during the weekly chat in the #core-auto-update SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

What’s next?

See you on May 4, 2021 at 17:00 UTC for the kickoff meeting, during the weekly #core-auto-update.

In the meantime, if you already know that you want to work on a specific goal or a specific ticket, please leave a comment.

Thank you!

Thanks @afragen, @audrasjb, and @hellofromtonya for the peer review. Thanks @sergeybiryukov for combing through Trac to look for the tickets!

#auto-updates, #updater