The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
If you’d like to help test Full Site Editing, please join the FSE Outreach Program. You can find current calls for testing for this program here and you can join the fun in #fse-outreach-experiment.
The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. here.
@hellofromtonya summarised last week in tests:
5.9 e2e strategy session was a big step forward for WP CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., as:
@hellofromtonya mentioned, that the next step is to do the same for PHPUnit integration tests. Both of the above will become a 5.9 Test Roadmap
@justinahinon shared his concerns about shipping Playwright to Gutenberg now as it’ll need lots of work. He thinks that it’s better to make this experiment in the Core before.
@hellofromtonya shared her thought that Puppeteer and Playwright might exist in the suite together
@costdev agreed
@hellofromtonya wrote the Bigger Scope which is:
@costdev reminded, that we need to agree on standard for writing e2e with Playwright
@hellofromtonya mentioned that she’s working on the Webfonts APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.. The API will handle the theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. webfonts. There are some tickets that need unit/integration tests:
If anyone’s interested in writing tests please leave comment either on Slack or in ticket itself.
This is a summary of a Hallway Hangout that was wrangled in the #fse-outreach-experiment channel as part of the FSE Outreach Program. It was intended to be a casual walkthrough of the current call for testing with @sparklingrobots taking the lead sharing their screen. Big thank you for being brave enough to meander around the call for testing publicly.
As a reminder, there is still 1 week left to participate so please join in if you can.
Attendance:
Thank you to everyone who attended 🙂 @karmatosed @mkaz @shaunandrews @paaljoachim @annezazu @sparklingrobots @richtabor
Video Recording:
Feedback Overview
Throughout the session, some specific issues were noted as problem areas to follow up on, including some previous reported and some new:
Outside of specific points of feedback about the experience, there were also high level themes that became apparent as @sparklingrobots made her way:
We ended the call talking about how the future of the Query Loop block powering more user friendly variations, an integrated block pattern library, an overhaul of the IA of the various design tools, & more will help ease this current experience. For now though, we’re in an in between state where loads can still be learned to improve the default tools themselves.
Next Steps
@annezazu will follow up next week after she returns (she’s out Thurs/Fri) to replicate and open issues.
#fse-hallway-hangout, #fse-outreach-programThe meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. here.
@hellofromtonya introduced agenda of the meeting and shared a post that describes where the testers are needed during the week. At the end of introduction, Team Test Update was shared.
The team was able to decrease the numbers of tickets.
@hellofromtonya described how this changes impact CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. testers. Anyone that’s running the PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. test suite or buil automated integration tests, should read the dev note.
For running the PHP test suite with Docker, nothing changes in your workflow:
npm install npm run build:dev npm run env:start npm run env:install
and for running the tests:
npm run test:php
for running a specific group:
npm run test:php -- --group groupname
When not using Docker to test, then there are extra steps. The handbook and Dev Note provide the details.
In 5.9 release, Test Team will introduce the roadmap.
Everyone agreed that it’s necessary and help with the process of analyse and future improvement. @hellofromtonya also mention that we’ll implement a longer term vision, called North Star.
@justinahinon shared his document that is a great roadmap for automate tests focal group.
@hellofromtonya shared an idea to have 5 separate focal groups in Test Team, each will need folks to shepherd the group:
If anyone is interested in being a co-lead of any group, it’s definitely possible and the Team is looking forward for new members.
@hellofromtonya shared expectations:
Team agreed that we need a bit of advertisment of our group.
Here is the agenda for the upcoming Test Team Chat scheduled for 2021-09-28 13:00.
This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..
Do you have something to propose for the agenda? Please leave a comment below.
Can’t make the meeting? Share anything relevant for the discussion in a comment below.
Props: @boniu91 for peer review.
With efforts in progress to introduce end-to-end (e2e) testing in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., this working session brings folks together to collectively define the big picture goals and what to accomplish during the 5.9 cycle (i.e. roadmap). These goals can then serve as the North Star 🌟 to guide contributors for maximizing impact.
When: RESCHEDULED for 2021-10-06 10:00
Where: Zoom call (link will be shared in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. #core-test channel prior to the start of the session)
Topic: Strategize e2e goals for the 5.9 release cycle
Props to @justinahinon for proofreading.
The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. here.
@hellofromtonya reminded, that this bi-weekly meeting is where people who test WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. gather to discuss important things regarding test contribution. Testing includes manual testing, attempting reproduce reported issues, and automated testing.
You can find the agenda meeting here.
@hellofromtonya described the purpose of this weekly publication, in short, it includes:
Team agreed that it’s very helpful.
@hellofromtonya reminded that it’s a Team update, an overview of what has happeed last week. It also includes stats that are related to our team.
The Team agreed that the queue of the items that need testers’ attention is long.
@hellofromtonya explained that it’s a time for testers to share updates for testing project initiatives. This section has started with:
@hareesh added:
@francina opened this discussion, asking what instructions we want to give. The main reason was related to setting up environment.
@hellofromtonya proposed to create a docs empowering everyone to contribution, we need clear entry points for:
Team agreed that this documentation should be present in Make Test website, Make Core should only refer to it.
@hellofromtonya mentioned, that PHPUnit docs page lacks a reference to where contributors can set up their local machines. That should be a priority.
@francina @juhise @hellofromtonya agreed that the Team should start documenting the steps for Docker testing environment and later repeat the same pattern for other solutions. On of the concerns is that screenshot are getting outdated very quickly.
@mkaz shown blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-editor’s development site setup instructions based on wp-env. Team agreed that all projects should be pointing to one place where everything related to development environment is stored and maintained.
@hellofromtonya proposed a step-by-step instrctions of creating documentation for different types of test environments:
@francina proposed working session with setting up environments. It’s going to happen on 2021-09-23 14:00
People with different OS will be essential for the successful meeting.
@boniu91 shared a reminder that on 2021-09-17 13:15 the Test Scrub is going to happen taking care of the following tickets:
Here is the agenda for the upcoming Test Team Chat scheduled for 2021-09-14 13:00.
This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..
Do you have something to propose for the agenda? Please leave a comment below.
Can’t make the meeting? Share anything relevant for the discussion in a comment below.
Props: @boniu91 for peer review.
The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., here.
@hellofromtonya described that this is our time together to talk about things for our team: blockers, needs, roadmap, learning, docs, etc.
@boniu91 reminded, that there’s open poll to decide when we’ll be meeting in the future, twice a week:
@hellofromtonya asked to post the answers as comments inside the mentioned post.
@hellofromtonya described what is the goal:
And also what’s the current status:
CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. tests were blocked for over 2 years from running PHPUnit 8 and 9. They are not blocked anymore!!
@pbearne asked if we have any list of backports that needs to be finished
@hellofromtonya write a list of PRs:
There are 2 PRs for backports:
Plus, there’s still work to do in #53904
Also, the backports are waiting for PR 1587 to be fully reviewed and committed, we need to be sure that all paths are covered.
@lucatume offered, that he can take a look
@boniu91 asked if there’s any calendar with future WP Core releases
@francina answered that not yet
@pbearne asked what’s the plan for the @covers
@hellofromtonya answered that the plan for @covers is to address those during the reorganization and namespacing effort.
Why?
That’s when we’ll be thoroughly reviewing each test. At that point, we’ll know what each test is actually covering.
Why wait?
To avoid duplicating effort. We’re focusing first on PHP 8.1 and modernizing for latest PHPUnit. Once that’s done, then we plan to switch to the reorganization, namespacing, covers effort.
@hellofromtonya mentioned that there are many ways of contribution:
For those who don’t know how to build automated tests, you can contribute too:
@boniu91 commented that it would be good to have some kind of document that would describe the possible ways of contribution for new Test Team members.
Team agreed, @francina agreed to lead the effort, @hellofromtonya and @boniu91 offered help.
Test Team is meeting twice a week:
This schedule was discussed while ago, when the team wasn’t crystalized as it’s right now. This is why it’s a good time to ask members of this Team whether the time of meetings is okay, or shall we reschedule it.
Please post in the comments your opinion about that, we’ll be collecting votes until 30th of August and then make a decision.
Thank you for reading!
The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., here.
Announcement starts here.
@hellofromtonya mentioned that two testers were nominated for this position to work together, both of them accepted the nomination, @hellofromtonya @boniu91
Team had no objections to those nominations
@hellofromtonya published and shared previously prepared introductions of newly nominated Test Team Reps
The discussion starts here.
@francina shared her list, people present on the meeting agreed:
@lucatume added, that’s not clear for him:
@hellofromtonya replied to the second point, saying that:
@lucatume expanded the previous (first) point and for the most effective contribution, the following things should be clear:
@hellofromtonya answered, that tickets with needs-unit-tests keyword are the ones for the PHPUnit side of things. We don’t have anything for the e2e tests though.
@lucatume suggested adding needs-e2e-tests keyword and the Team agreed that’d be useful.
@francina mentioned, that the current test setup is Jest + Puppeteer. We need developers to set up a tool for the Team that will make creating e2e tests easy.
Team agreed that it’s not possible to have 100% of automated tests, with no human review.
@francina shared what the Team needs right now to kick off the e2e testing:
@hellofromtonya said, that most likely, the bottleneck will be the group of automation test engineers to do the review, tuning, and maintenance work.
@lucatume explained how the “autogenerating” of e2e tests looks like (here)
@hellofromtonya confirmed the workflow:
Mai mentioned, that generated code can be worse in terms of quality than the one written by a human. Team agreed that it’ll need to be refined and maintained. @hellofromtonya shared improved workflow:
Team agreed to start a pilot initiative:
@lucatume will start creating the prototype
@francina offered reviewing the test handbooks
