Introducing GitHub Actions for Automated Testing

As of [49162], CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. is now running automated tests using GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. Actions as a runner, in addition to the existing Travis CI and Appveyor runs. This post is to publicize the change, document the reasoning, communicate next steps, and share how people contributing to WordPress Core will benefit.

GitHub Actions allows us to automate software workflows directly in GitHub, triggered by GitHub events. By switching, we are able to take advantage of a unified interface, inline annotations for linting issues in pull requests, the broader open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. ecosystem building and using Actions including existing work in Gutenberg, and free availability for public repositories. Note that private repositories do use the monthly bucket of included minutes.

For contributors, this continues to refine the experience of working on Trac tickets using GitHub pull requests, most notably by showing linting errors inline in the diff view of the PR (known as annotations). This also consolidates external tooling into one place. If you have not already, please take a moment to associate your GitHub account with your profile.

Screenshot of inline annotation examples

These 6 workflows cover all current testing and analysis performed in Travis CI and Appveyor:

There is also an additional 7th workflow that is meant to leave a welcome message when it’s the contributor’s first pull request, letting them know how we use GitHub pull requests and how to link them to a TracTrac An open source project by Edgewall Software that serves as a bug tracker and project management tool for WordPress. ticketticket Created for both bug reports and feature development on the bug tracker.. There appears to be an issue concerning permissioning when PRs are sent from forks, so this is pending.

Currently, Travis CI and Appveyor will continue to run for a transition period (ending TBD) to allow for any issues to be ironed out, and so that real-world usage data can be collected. So far, even in early testing, runs appear to be completing more quickly and with fewer/no false negatives, e.g. when Travis CI does not see the commit in the mirror yet. @desrosj will be collating run data in a spreadsheet, including but not limited to: overall build time, run time comparison (where 1:1 comparisons can be made), and frequency of false negatives.

Known next steps

  • Add and configure SlackSlack Slack is a Collaborative Group Chat Platform The WordPress community has its own Slack Channel at notifications. In addition to sending the results of the whole build of a core commit into #core, we may also want to consider a firehose channel for PRs.
  • Move to GitHub badges for build status indicators – note that these are per-workflow, which is different from the single badge for the entire Travis build for a given commit. However, GitHub does report an overall status for a commit/PR, so we may be able to use that information as well.
  • Report test results to the Host Test Results page.
  • Switch to ESLint from JSHint, as the latter does not appear to easily support inline annotations, and the former is in broader usage including in core for docs, GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. and many community projects. See #31823 for more – volunteers very much appreciated here.
  • Backportbackport A port is when code from one branch (or trunk) is merged into another branch or trunk. Some changes in WordPress point releases are the result of backporting code from trunk to the release branch. the workflow files to actively maintained older branches.

As always, please report any issues you are seeing with our GitHub Actions, as well as further ideas for use you may have. Major thanks to @desrosj for all the heavy lifting he’s done in just a couple of weeks, and to @ocean90 and @ayeshrajans for their help along the way.

#5-6, #build-tools, #unit-tests