The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. here. With no agenda, the chat was open floor discussions.
@costdev asked: What projects or specific goals are targeted for 6.0? @hellofromtonya replied that 6.0 roadmap of the key features and refinements to be built for this major releaseMajor Release A set of releases or versions having the same major version number may be collectively referred to as “X.Y” -- for example version 5.2.x to refer to versions 5.2, 5.2.1, and all other versions in the 5.2. (five dot two dot) branch of that software. Major Releases often are the introduction of new major features and functionality.. The Test Team will support that effort.
4 Proposed Initiatives
Team discussed other projects and goals not necessarily specific to 6.0:
- Initiative 1: Audit and reorganize the testing suite – which includes
- Initiative 2: Ensure all patches ready for commit have test.
- Initiative 3: Grow the e2e test suites.
- Initiative 4: Testing Handbook – needs a lot of work to empower anyone to quickly get testing.
@hellofromtonya shared a bigger picture framework / thinking for testing:
Tests express the intended and expected behavior of how the code should and shouldn’t behave under different conditions. When tests do this, they are a wonderful source for learning.
Testing itself fuels the feedback loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. back into the development process to know what does and doesn’t work as expected.
- Get more people involved in testing -> meaning the handbook, tooling, and code examples need to empower anyone to get testing.
- Properly test both the expected (happy) and unexpected (unhappy) behavior in code at different levels > meaning tests need to be audited to ensure they are doing this and that the key areas have the tests needed to provide the feedback early enough.
- Get the tests organized, readable, and usable.
The goals aren’t to build more tests. Rather, the goals are to build the tests that are needed to validate the code works as expected under different conditions to further help improve the user experience and get feedback fed back into the development cycles.
Where to start? What to prioritize?
Team discussed whether to start with planning the test suite reorganization or getting the
@covers tags added / corrected.
The team decided to prioritize the
@covers tags in order to ensure the code coverage reports are accurate to point to where more work is needed.
Team then discussed how to divide the auditing work to finish the
@covers PR and get it merged.
- Mob review: @costdev and @hellofromtonya will focus on auditing.
- @pbearne will keep the PR up-to-date and handle modifying the PR from the audit.
- @hellofromtonya will commit finished work in batches.
Restructuring the handbook raises the potential for more team members to work on the other initiatives. Since it feeds the numbers, confidence and accuracy of the team, it helps with everything else. Coverage reports won’t help increase the number of test team contributors, so I’m tempted to say that our first big drive should be to work on the handbook. If there’s things that need to be done in advance of handbook entries, such as establishing a new structure for the tests, that should be done in order to help 1) achieve the goal of restructuring and 2) provide information for the handbook entries.by @costdev
Props to @boniu91 for proofreading.