Leaner, steadier PHPUnit runs for upcoming releases

In recent years, the CI load across coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. WordPress GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged by the repository owner. https://github.com/ repos increased as the project evolved. This is normal for an active project.

But, the load can be especially high when running the full PHPUnit matrix at once on many PRs, such as during a release with many backportbackport A port is when code from one branch (or trunk) is merged into another branch or trunk. Some changes in WordPress point releases are the result of backporting code from trunk to the release branch. branches. Cases like this mean the number of jobs can strain the capacity of the system.

Ahead of 7.1, we’ve landed a round of trims and reliability fixes aimed at increasing capacity and running leaner while ensuring quality checks still run.

This CI work builds on the test-suite optimization work from many contributors in the past.

Two key changes

  1. Trimmed the PHPUnit matrix to boundary PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 7.4 or higher versions (#12719 on trunktrunk A directory in Subversion containing the latest development code in preparation for the next major release cycle. If you are running "trunk", then you are on the latest revision., #12720 for 7.0). Full PHP coverage kept, redundant database combinations dropped. Per run: ~52% fewer jobs and ~54% fewer job-minutes.
  2. Fetch the GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ build once per run (#12701) instead of once per job, plus bounded retries on Docker image pulls (#12703). Runs needing a rerun to go green roughly halved, from ~68% to ~36%.

Source: measured per run from the GitHub Actions APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. (job counts and durations, and rerun counts) across comparable runs before / after the changes.

Next up

  • Trim the 6.8 matrix (#12726).
  • Pilot a dedicated larger-runner pool for use during release windows to help with concurrency issues (more on that in a future update).

Note: The PHPUnit tests themselves could be further tuned. The trims above cut job count, not job duration—so we could give more attention to the tests themselves to find more efficiency.

Thanks to contributions from adrianmoldovanwp, barry, garyj, @johnbillion, @jonsurrell, @jorbin, lucasbustamante, and @mukesh27.