What
E2E (End To End) tests are a form of automated software testing which runs in the context of a user using an application. The intention is to test the complete “end to end” workflow that a user would take in various scenarios.
In web software development this means running and controlling a web browser pointing at a web page, either fully automatically by a machine, or by a user following the test steps.
Frameworks for running E2E tests allow a web browser to be controlled fully automatically, for instance by navigating to URLs, filling out forms, and clicking on links and buttons. The tests operate by checking assertions about the expected content or behaviour of the page and triggering a test failure when they are not met.
Why
Testing individual aspects of an application’s code — for example with unit tests or integration tests — reduces the chance of low-level bugs in the code that powers an application. Unit testing helps maintain the quality of the code that powers the application.
In contrast, E2E testing tests the higher-level functionality of an application that the user sees, typically including the interface and the inputs and controls that the user interacts with.
Unit testing and E2E testing can, should, and do co-exist.
How
E2E tests are usually human-readable and provide logical steps for a user to perform. For example:
- Given I am logged in as an Author
- When I view the list of posts in the dashboard
- And I see posts published by other users
- Then I should not see an “Edit” link next to posts which are published by other users
The scenario above can potentially form a completely automated E2E test, depending on the framework in use and the built-in steps that have been defined for it. The tests are written in such a way that a human can follow the same steps and come to the same result.
This means anyone, not just a developer, can write an outline of an E2E test.
Implementing E2E tests in WordPress Core Core is the set of software required to run WordPress. The Core Development Team builds WordPress.
WordPress Core right now has only one E2E test in place, to check that the wp-admin (and super admin) page loads. Gutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/, on the other hand, has quite a few.
There are some tickets and pull requests open about this:
https://core.trac.wordpress.org/ticket/43395
https://core.trac.wordpress.org/ticket/49507
There is also a test environment but it is not currently being used as part of the automated test suite: https://make.wordpress.org/core/2019/08/05/wordpress-local-environment/
It would be beneficial to create a project to work on this in a structured way in order to maintain the stability and reliability of WordPress core as new features are added and changes are made.
Moving Forward
Months ago, this topic was mentioned multiple times and @pandjarov from SiteGround has offered to set up the test environment plus some engineering time to work on the tests.
We could either set up a new one or check that the existing one is still the right way forward.
To really kick things off, the Core Team has to come up with requirements for the test environment, its tests, and the preferred workflow, so the tests are really useful.
What is needed
- A list of requirements – I think Core Committers, especially with a long history of contributing (the so-called historians) are best suited to start the list with the most pressing concerns
- Evaluating the existing test environment (existing vs creating new)
Ideas about what functionality in WordPress should be a priority in the automated tests, for example, the upgrade process, or managing users, or installing plugins - Developers who are familiar with Jest, which is the test framework in place in Gutenberg and WordPress core, so the test environment can be improved, and to write new tests
- A fork of the
wordpress-develop
repo on GitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ where the project can be managed and PRs opened - Volunteers to work on this. It’s a big task because it starts from 0 – this is very important because what the previous post about this has shown, in my opinion, is that an environment is not enough 😉
- Coordination with the Test team
- For making it easier to communicate I propose to add a dedicated weekly chat in the #core-test channel – From September 6th I am available to do that.
Deadline to comment
To make sure that the project lifts off I think it needs enough interest and volunteers to actively work on it.
Please leave a comment before September 6th (so you have a month to think about it!) with the requirements and/or (possibly AND) your availability to be part of this.