Title: Core E2E – Make WordPress Test --- # Category Archives: Core E2E [ ](https://profiles.wordpress.org/bartkalisz/) [Bart Kalisz](https://profiles.wordpress.org/bartkalisz/) 3:49 pm _on_ October 4, 2021 Tags: [core-test ( 231 )](https://make.wordpress.org/test/tag/core-test/), [gutenberg ( 74 )](https://make.wordpress.org/test/tag/gutenberg/)    # 󠀁[Proposal: Migrate e2e to Playwright!](https://make.wordpress.org/test/2021/10/04/proposal-migrate-e2e-to-playwright/)󠁿 Howdy, good people! 👋 In the spirit of improving the E2E developer experience, I’d like to make a case for migrating CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.’s browser automation library to [Playwright](https://playwright.dev/docs/why-playwright). [I was asked](https://wordpress.slack.com/archives/C03B0H5J0/p1630587099134300?thread_ts=1630495198.109800&cid=C03B0H5J0) to write this post after [opening an experimental pull request](https://github.com/WordPress/gutenberg/pull/34089), where I migrated a selected portion of specs to Playwright, making them available for running both locally and on CI. That happened some time ago, so there’s already been quite a bit of discussion going on there! Having said that, I’m going to try making the case again here, taking the feedback I’ve received so far into consideration. I also encourage you to check out the PR to see the implementation details and Playwright advantages in action. ## Why drop Puppeteer in favor of Playwright? It’s easier to **write _stable_ tests**. There are a few reasons why. Please read on to find out which ones I think are the most relevant for the project. ### The APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. Playwright’s API is almost identical to Puppeteer’s, which means that the developer transition should be close to effortless. I think that’s a big factor here, as it significantly lowers the cost of this transition also from the migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. side. A good thing to start with! 🤞 ### The Auto-Waiting Mechanism From [the documentation](https://playwright.dev/docs/actionability): > Playwright performs a range of actionability checks on the elements before making > actions to ensure these actions behave as expected. It auto-waits for all the > relevant checks to pass and only then performs the requested action. (…) I think it’s the number one reason for the stability improvement over Puppeteer. In practice, it means the following: #### No need to perform any additional presence checks Most of the DOM changes happen asynchronously, so in order to avoid flaky behavior, a test usually explicitly wait for an element before performing an action on it. A good example would be the most frequently used `click` action, which usually looks like this when performed with Puppeteer: ```wp-block-code const button = await page.waitForSelector( 'button' ); await button.click(); ``` With Playwright, thanks to the auto-waiting mechanism it becomes just: ```wp-block-code await page.click( 'button' ) ``` #### No need to disable CSSCSS CSS is an acronym for cascading style sheets. This is what controls the design or look and feel of a site. animations This is thanks to the [stability check](https://playwright.dev/docs/actionability#stable), which makes sure the element has a bounding box and is not moving before the action is performed. I think this is a major advantage because it allows to fully test the application, including the CSS animations, which are an integral part of the user interface. In [my PR](https://github.com/WordPress/gutenberg/pull/34089), as a proof of concept, I removed the code that disables CSS animations and the forced reduced motion, which slowed down the refactored tests (21 suites) [by around 37 seconds](https://github.com/WordPress/gutenberg/pull/34089#issuecomment-932269462). This number will grow with every test, but judging by the current data it shouldn’t be more than a few minutes in total. I’d say the trade-off would still be worth it, but this can be discussed and decided upon later. What do you think about testing without animations? Should they be enabled if it’s possible, even for the cost of extra wait? #### Less code! In general, all of the above comes down to writing less and simpler code to get the same or better results than Puppeteer. If you go to [my PR](https://github.com/WordPress/gutenberg/pull/34089/files#diff-cf156cf748cf32e92fb3a529d7182fb6a8262c30892693e64713ce74cd6431bf), you’ll notice that there are more lines removed than added in the refactored tests! With Playwright, the tests and test utilities are easier to write and follow, and the environment requires less customization (e.g. disabled animations) which actually makes it closer to what users are experiencing. **Are there any downsides to the auto-waiting mechanism?** [There were some concerns](https://wordpress.slack.com/archives/C02QB2JS7/p1630494285149300) about how this mechanism could affect the performance tests. Because it could potentially become a blocking factor for this migration, I decided to [migrate Gutenberg’s performance specs to Playwright](https://github.com/WordPress/gutenberg/pull/34699) as a proof of concept and see what happens. So far, thankfully it looks like there isn’t much difference between Puppeteer and Playwright — the performance metrics are very close, which would be the desired outcome.Do you think there could be any downsides to the auto-waiting mechanism? Please let me know in the comments! ### The Advanced Selectors Support This part has changed a bit due to some feedback received in [the PR](https://github.com/WordPress/gutenberg/pull/34699). Originally, I mentioned [text selectors](https://playwright.dev/docs/selectors#text-selector) and [layout-based selectors](https://playwright.dev/docs/selectors#selecting-elements-based-on-layout) as the number one reason for making the tests and utilities easier to write and follow, as well as making them more resilient. While prioritizing user-facing attributes is still a good practice for most applications, GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. [https://wordpress.org/gutenberg/](https://wordpress.org/gutenberg/) is a bit different in this regard. Apparently, CSS classes are considered to be the API there, so they change less often than the user interface. Nevertheless, as some folks noticed advanced selectors are still a big win as they’d allow to, e.g. drop the use of cumbersome XPath selectors and more with powerful [selector chaining](https://playwright.dev/docs/selectors/#chaining-selectors). Currently, with Puppeteer, [CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) can only be used. Learn more about Playwright’s advanced selectors from [https://playwright.dev/docs/selectors](https://playwright.dev/docs/selectors). ### The Debugging #### Inspector The built-in [Inspector](https://playwright.dev/docs/1.11.0/inspector) is also a big advantage of Playwright. It’s quite intuitive and has some neat features like stepping through the script while running headfully or dynamically recording actions to a script — a really convenient way of quickly drafting the test. See the video below for a short demo of the script recorder. Code generation with Playwright inspector in action 💥 #### Trace Viewer Playwright offers a complete tracing solution. Trace can be recorded and stored in a zip file, which then can be viewed via the Trace Viewer GUI: ![](https://make.wordpress.org/test/files/2021/10/trace-viewer-2-1024x615.png) Viewing a recorded trace in Trace Viewer On the image above, you can see that the trace is displayed in a form of a film strip. Each frame can contain _Before_, _Action_, and _After_ snapshots visualizing a complete action execution. On the left-hand side is a list of all the actions Playwright performed. Each of them can be inspected in more detail in the section on the right-hand side, where you can switch between the action log, location in the source code, and the network log. I think it’s great to have that kind of functionality out of the box. It also shows how Playwright is intended to be a complete testing solution. With Puppeteer, there aren’t really any first-party tools for debugging, as far as I’m aware – The [suggested way](https://github.com/puppeteer/puppeteer/blob/main/README.md#debugging-tips) is to either slow down the tests in headful mode with [DevTools open](https://developer.chrome.com/docs/devtools/) or use the [Node.js Debugger](https://nodejs.org/en/docs/guides/debugging-getting-started/) when running headlessly. Learn more about Playwright’s debugging tools from [https://playwright.dev/docs/debug](https://playwright.dev/docs/debug). ### The Browser Support If there’s a goal to expand testing to browsers other than Chrome, it wouldn’t be an issue for Playwright as it supports all other major players: Firefox, WebKit, and Microsoft Edge. At the time of writing this, Puppeteer supports only Chrome and Chromium, and the official support of Firefox is [currently experimental](https://github.com/puppeteer/puppeteer#q-what-is-the-status-of-cross-browser-support). Learn more about Playwright browser support from [https://playwright.dev/docs/browsers](https://playwright.dev/docs/browsers). ### The Dedicated Test-Runner Playwright has a first-party [test-runner](https://playwright.dev/docs/intro), which is very similar to Jest test-runner (currently used for Puppeteer) but written from scratch. It contains a lot of end-to-end testing utils, tooling, [concurrency](https://playwright.dev/docs/test-parallel), [reporting](https://playwright.dev/docs/test-reporters), [assertions](https://playwright.dev/docs/test-assertions), [artifacts](https://playwright.dev/docs/test-configuration#record-test-trace), etc., and extensive configuration support. Another quite nice thing to have without having to install and rely on third-party libraries! Learn more about Playwright Test from [https://playwright.dev/docs/intro](https://playwright.dev/docs/intro). ### The Documentation I think it deserves a mention here, as it’s easy to navigate and really well written, in my opinion. Please take a few minutes and check it out for yourself at [https://playwright.dev/docs/api/class-playwright](https://playwright.dev/docs/api/class-playwright)– maybe you’ll find even more reasons to switch to Playwright? 😉 --- Writing good, stable E2E tests is often a struggle. If there’s a chance of improving that, especially with such a low cost, then it should be done. I would be happy to work on this task if there’s a consensus to move it forward. Thanks for reading. I’m looking forward to all the feedback! Props to [@hellofromtonya](https://profiles.wordpress.org/hellofromtonya/) and [@boniu91](https://profiles.wordpress.org/boniu91/) for proofreading! +make.wordpress.org/core/ [#core-test](https://make.wordpress.org/test/tag/core-test/), [#gutenberg](https://make.wordpress.org/test/tag/gutenberg/) * [Login to Reply](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fmake.wordpress.org%2Ftest%2F2021%2F10%2F04%2Fproposal-migrate-e2e-to-playwright%2F%23respond&locale=en_US) [ ](https://profiles.wordpress.org/justinahinon/) [Justin Ahinon](https://profiles.wordpress.org/justinahinon/) 6:51 am _on_ June 23, 2021   # 󠀁[State of e2e testing in WordPress Core](https://make.wordpress.org/test/2021/06/23/state-of-e2e-testing-in-wordpress-core/)󠁿 End-to-end (E2E) tests allow simulating user flows for an application, a site and validating them. They allow for automated testing of user interactions while providing a safety net for continuous development. Over the past two years, the groundwork for implementing e2e tests in WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. has been laid. However, there hasn’t been much communication about the technical stack that is used for these tests or the next steps for adding more tests in WordPress. The purpose of this post is to inform contributors about the current status of e2e testing in WordPress Core, the technologies being used, and what is currently being done to implement these tests. # What stack are we using for WordPress Core e2e tests? E2e testing in WordPress Core is implemented with Jest as the testing framework and Puppeteer for browser interactions. This stack is relevant for Core use case because it is present in other parts of the CMS codebase.  On the other hand, it is used by GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. [https://wordpress.org/gutenberg/](https://wordpress.org/gutenberg/) for its tests. So we are not starting from scratch for the implementation in Core, as there is already some preliminary work done with the [@wordpress/e2e-tests](https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-tests) package and the [@wordpress-e2e-utils utility](https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils) package, which includes several basic flows that can be reused in tests. # What tests are currently implemented in Core? Currently, only the setup for testing and a basic test of the WordPress Dashboard homepage are implemented in Core here. The [@wordpress/e2e-tests-utils utility package](https://www.npmjs.com/package/@wordpress/e2e-test-utils)( in the Gutenberg repository) contains various tests for installing, disabling, activating themes and plugins, navigating the different admin pages, etc. # What is the Test Team working on? There are currently efforts being made to implement more e2e tests in WordPress Core. Starting with [this ticket](https://core.trac.wordpress.org/ticket/52895), opened by [@isabel_brison](https://profiles.wordpress.org/isabel_brison/) that presents an overview of the different tests that will be implemented. There are also pull requests, and branches from contributors who have already implemented tests for some parts of WordPress: * PR for add/edit posts tests: [https://github.com/WordPress/wordpress-develop/pull/1089](https://github.com/WordPress/wordpress-develop/pull/1089) * Branch for Core categories tests: [https://github.com/JustinyAhin/wordpress-develop/blob/try/add-new-category-tests/tests/e2e/specs/add-edit-category.test.js](https://github.com/JustinyAhin/wordpress-develop/blob/try/add-new-category-tests/tests/e2e/specs/add-edit-category.test.js) * Branch for Core users tests: [https://github.com/JustinyAhin/wordpress-develop/blob/add/users-tests/tests/e2e/specs/users.test.js](https://github.com/JustinyAhin/wordpress-develop/blob/add/users-tests/tests/e2e/specs/users.test.js) The next step is to start on the overview of the e2e tests to be implemented by creating smaller tickets for each of the tests, and then write the tests. Any interested contributor is invited to comment on [the ticket](https://core.trac.wordpress.org/ticket/52895) to propose other test scenarios, learn, and/or help build tests. All are welcomed. # Resources ## Blog posts [Introducing the WordPress e2e tests](https://make.wordpress.org/core/2019/06/27/introducing-the-wordpress-e2e-tests/) [E2E (End-To-End) Testing in Core Proposal](https://make.wordpress.org/core/2020/08/07/e2e-end-to-end-testing-in-core-working-group-proposal/) ## Tickets [Overview of e2e tests to be written](https://core.trac.wordpress.org/ticket/52895) --- _Props [@francina](https://profiles.wordpress.org/francina/) and [@hellofromtonya](https://profiles.wordpress.org/hellofromtonya/) for reviews and edits._ * [Login to Reply](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fmake.wordpress.org%2Ftest%2F2021%2F06%2F23%2Fstate-of-e2e-testing-in-wordpress-core%2F%23respond&locale=en_US)