The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., here.

Announcement: Test Team Reps nominations

Announcement starts here.

@hellofromtonya mentioned that two testers were nominated for this position to work together, both of them accepted the nomination, @hellofromtonya @boniu91

Team had no objections to those nominations

@hellofromtonya published and shared previously prepared introductions of newly nominated Test Team Reps

Discussion: What is needed for Testers to make their contribution easier and better

The discussion starts here.

@francina shared her list, people present on the meeting agreed:

• Clear testing instructions in tickets
• Clear, unified, synched instructions to setup different testing environments
• Streaming to go through tests together and learn by doing
• A roadmap and action plan to automated testing

@lucatume added, that’s not clear for him:

• Where and how he could contribute in a best way. He’s willing to help with what he’s the best in: coding.
• It’s also not clear where we should collect discussions about the tickets and other important things.

@hellofromtonya replied to the second point, saying that:

• If the discussion is related to ticket, we should collect them in ticket. If the discussion happens on Slack, it’s a good practice to link it inside the ticket.
• Slack is great for adding collaboration, seeking guidance, Team chats, etc.

@lucatume expanded the previous (first) point and for the most effective contribution, the following things should be clear:

• A clear indication of what tickets will need test code, not manual testing.
• A clear path to how test code can be contributed. If tools are required, what tools are required.
• A definition of “good” and “bad” friction.

@hellofromtonya answered, that tickets with needs-unit-tests keyword are the ones for the PHPUnit side of things. We don’t have anything for the e2e tests though.

@lucatume suggested adding needs-e2e-tests keyword and the Team agreed that’d be useful.

@francina mentioned, that the current test setup is Jest + Puppeteer. We need developers to set up a tool for the Team that will make creating e2e tests easy.

Team agreed that it’s not possible to have 100% of automated tests, with no human review.

@francina shared what the Team needs right now to kick off the e2e testing:

• Skilled QA engineers to setup up the infrastructure
• Documentation to teach people to write testing specs
• People to write the tests
• Automated test engineers to review and maintain

@hellofromtonya said, that most likely, the bottleneck will be the group of automation test engineers to do the review, tuning, and maintenance work.

@lucatume explained how the “autogenerating” of e2e tests looks like (here)

@hellofromtonya confirmed the workflow:

• Human does the manual test
• Tooling records those steps
• Tooling converts the steps into code

Mai mentioned, that generated code can be worse in terms of quality than the one written by a human. Team agreed that it’ll need to be refined and maintained. @hellofromtonya shared improved workflow:

• Human tester does the manual testing steps
• Tooling records those steps
• Tooling converts the steps into code
• Test code is attached to the ticket
• Code is reviewed by a human is skilled in e2e tests and the thing under test
• Once approved, a coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. committer commits the test code into the project
• Skilled e2e test humans maintain the tests (including tuning and refinement)

Team agreed to start a pilot initiative:

• Build a prototype
• Start small with a handful of impactful e2e tests
• Get those tests stable
• Learn
• Iterate

@lucatume will start creating the prototype
@francina offered reviewing the test handbooks

#build-test-tools, #test-team