Migrating WordPress E2E tests to Playwright

Context

The idea of using Playwright as an alternative to Puppeteer was initially mentioned in early 2020 during a JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. chat (meeting recap). At that time, the project was still in its early stages, and it was decided to postpone the decision until Playwright was more mature.

A few months ago, @bartkalisz proposed a proof of concept to migrate the GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ E2E test suite from Puppeteer to Playwright. This proposal was discussed several times in the above PR, during an editor chat, and during test team meetings.

WordPress test team members, Gutenberg team members, and Playwright engineers met to discuss in detail how this migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. could be done and what the possible blockers might be. You can find the minutes of that meeting here.

The infrastructure for writing Playwright tests in Gutenberg has been implemented by @kevin940726 here.

The purpose of this post is to present a migration plan to Playwright; as well as the implications this might have.

Areas in WordPress that will be affected by the migration

  • @wordpress/e2e-test-utils: It is a utility package with basic tests that are reused by other E2E test packages and suites in Gutenberg, in WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. and by third parties. This package is now deprecated and a new package @wordpress/e2e-test-utils-playwright was introduced internally for the migration process.
  • @wordpress/e2e-tests: This is the E2E test suite from Gutenberg. It contains tests for different parts of WordPress and Gutenberg. The package has also been deprecated, and new E2E tests exist now in the test/e2e folder.
  • WordPress Core E2E tests: This is the E2E test suite for WordPress Core. To date, it contains only a very small number of tests. It uses the @wordpress/e2e-tests-utils package, and will need to be migrated to use Playwright.

Migration plan

Here is the roadmap we will follow to perform the migration

  1. Add Playwright test infrastructure to Gutenberg (done)
  2. Gradually migrate Gutenberg E2E tests to Playwright:
    • Migrate tests from packages/e2e-tests to test/e2e one file at a time. See overview issue here.
    • Migrate test utils from packages/e2e-test-utils to packages/e2e-test-utils-playwright as required.
  3. Add Playwright test infrastructure to Core and migrate Core E2E tests to Playwright: once the e2e-test-utils used in Core are migrated, we can migrate the current suite of tests in Core to Playwright. 
  4. Remove old packages: remove @wordpress/e2e-tests, rename @wordpress/e2e-test-utils-playwright to @wordpress/e2e-test-utils.

Writing new tests during the migration

For now, developers may continue to write new E2E tests using Puppeteer. Once a significant amount of E2E tests have been migrated to Playwright, we can discuss enforcing that new E2E tests are written using Playwright. Waiting will ensure that enough test utils exist and that there are no problems with the new Playwright infrastructure.  See this issue for migration progress.

How to help

Helping migrate tests to Playwright is one way you can help right now. Playwright’s syntax is similar to Puppeteer’s, so it won’t be totally unfamiliar.

Here is some useful link to get you started:


Props to @kevin940726 @noisysocks for helping put this post together, and @aristath and @sergeybiryukov for reviews.

#e2e-tests, #gutenberg, #playwright, #puppeteer

Hallway Hangout: Fool me once — Writing end-to-end tests against regressions

On Thursday, I hosted a little walkthrough on how to write a simple end-to-end test to make sure a specific piece of editor functionality that had broken in the past wouldn’t break again.

As GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ grows, the project sometimes experiences regressions: Features that used to work suddenly don’t anymore. In order to prevent these regressions from happening, contributors can write end-to-end (e2e) tests that cover a given piece of functionality and alert us when that functionality is broken. Combined with a Continuous Integration system such as GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ Actions, this is a very powerful tool, since it can notify a PR author that their PR might break something before they even merge that PR.

I’ve recently tried to keep more track of such regressions, and have started filing issues requesting e2e tests to cover them. In the hangout, I’ve picked one of these issues and demonstrated how to write an e2e test for it.

Continue reading

#core-editor, #e2e-tests, #gutenberg, #hallwayhangout