The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
Please drop by any time in SlackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. with questions or to help out.
After one month of very intensive activity, we have finally reached the end of the test team program. We would like to thank all the participants for their dedication and hard work throughout this period. The program has been a great success in many areas, and we have gathered valuable insights and feedback that will help us improve our whole contribution onboarding process.
During the program, we started with a total of 9 participants, but after some expected dropouts, we ended with 6 members, with most participants doing a fantastic job during the entire process. They were involved in tasks such as testing, documentation improvements, leading meetings, and a lot of feedback to support the team’s growth.
In a dedicated SlackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel, we have been able to work very closely with the participants, gathering information about their experience through the process and also sharing the progress of this program. There was no clear starting program structure, but one happened to begin shaping as weeks went by that could be described as follows for the record:
Program Weekly Structure
The first week was focused on onboarding all members on the testing protocol as soon as possible, because one of the main targets was to go through a significant amount of tickets through the program period.
During the second week, we started introducing the meeting protocols, both for patch testing scrubbing and how to run the weekly test meetings with the corresponding agenda and summary post publishing. We also started to gather feedback on the testing protocol because the initial test results started to pop up.
For the third week, we switched the focus to documentation improvements, and we started to gather feedback on the meeting protocols and keep it up on the testing part. The contributor pathway video program began to come together.
Finally, for the last week, we tried to clear up all the final questions and analyze the current state of WordPress in correlation with the testing team to set future goals for the coming months.
Program Results
Overall, the program has exceeded our expectations in terms of engagement and results. Some goals were shared with the participants in the first interview, but from the experience we had from past programs, we knew that generally these goals were challenging to meet and could not be met. However, in this case, we have been able to achieve most of the goals and even exceed some of them. Here are some of the key results we have achieved:
Testing Reports
At the beginning of the program, there were a total of 487 tickets with the needs-testing label. By the end of the program, this number has dropped to 264, which is a significant decrease of almost 50%. This is by far one of the biggest achievements. We are pleased to observe that the protocol has been refined to a point that members were able to go through tickets at an excellent pace, understanding the whole process with proficiency. This will probably translate into a more efficient process in the future.
Documentation Improvements
Improving internal protocol documentation is something that requires more experience and time inside the team. However, we have been able to gather a lot of feedback and proposals for documentation improvements in our GitHub repository, which is a great starting point for the future. We have already started working on some proposals, and we hope to have them published in the following weeks.
The Crown Jewel: Test Contributor Pathway in Progress
One of the main goals of this program was to create a video training with a clear pathway for future contributors. We are delighted to announce that the program is almost completed, and we are planning to have it ready in a couple of weeks. A lot of feedback has been gathered through the program, and soon there will be an announcement in case anyone wants to join the “betaBetaA pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. program” to test the training and provide feedback before the official launch.
Participant Engagement Analysis as a Blueprint for Future Test Team Aspirants
We believe that sharing the results of the program participants could be useful for future WordPress contributors to understand which level of engagement is expected from them if they want to be part of the Test Team. And furthermore, to discover the different ways they can contribute to the team.
1. Ozgur (@ozgursar): Worked on a total of 68 testing reports, drove a test-chat and started leading to a documentation improvement regarding email testing. For the next few weeks, we expect the docs page to get published and a patch testing scrubbing meeting to be led to complete the whole circle. He is the first participant proposed to join the Test Team and continue his journey with us.
2. Huzaifa (@huzaifaalmesbah): Worked on a whopping total of 89 testing reports, which has been massive, and also proposed a documentation improvement regarding the `Getting started for testing` page structure. The only thing he has missed is leading some different meeting sessions, but there is already one scheduled for next week, and we are sure that with all the knowledge he has now, he is more than ready to lead more sessions in the future. He is the second participant suggested for joining the team.
3. Juanma (@juanmaguitar): He has been extremely active leading proposals for documentation improvements and providing a ton of feedback during all sessions, including a triaging guide, test-chat protocol guide, and some tips on post-tag improvement during a test-chat session. He has also led one test-chat session, but the only downside is that he has only been involved in testing 3 tickets, clearly the only weaker point that we hope could be improved in the following weeks to be somewhat on par with the rest of the participants. He is the third participant proposed for joining the team, and we are sure that with a bit more involvement in the reporting part, he can be a great asset for the team.
4. Erick (@r1k0): He worked through a grand total of 52 testing reports and also led one of the patch testing scrubs. There is only one thing that he has missed to go through the whole process, and it is the documentation improvement work and jumping into a couple more meeting leading sessions, but we are sure that he is more than ready to do it in the following weeks. He is the fourth participant recommended for joining the team, and we hope he jumps into the documentation part as soon as possible to be able to be on the team with a more complete profile.
5. Shazzad (@sajib1223): He was already active as a test team contributor before the program, but during the program, he has been able to consolidate many doubts he had about the test team protocols. He was able to run a patch testing scrub, but still in the other areas he has been lagging a bit. With no documentation proposals yet, only 10 tickets, and no test-chat sessions, we hope he can get quickly up to speed in the following weeks to be able to join the Test Team officially.
6. Mohammed (@mohkatz): He has been the last participant that has attended the whole program sessions, but unfortunately, he has not met the minimal requirements. With no testing reports, no documentation proposals, and no meeting-leading sessions. Hopefully, if he gives the team more time, he can get up to speed in the following weeks and be able to join the Test Team in the near future.
As I commented in the beginning, the other 3 participants that were selected dropped out in the beginning of the program for different reasons, and we hope that they can get involved in the future if they’d like to.
Future Directions and Organizer Personal Thoughts
As the organizer, I can’t stress anymore that this program has gone great, but simultaneously, I have to acknowledge that it has been very exhausting to organize. Running future programs like this is uncertain, and probably more organization and resource gathering will be required to be able to make it happen again in the following months. The dedication required from the organizers and participants is very high. Not only the two hours required for the live weekly sessions, but also the time to go through questions in the Slack chat, manually review most of the reports done and the documentation proposals, and also the time to create the training ideas.
Results of a program like this are proven to be excellent, but we need to find a way to make it more sustainable for the future. Luckily, now we have a couple of members onboarding in the team now and some additional ones probably soon, and we hope that they can take some of the leading load that has been driven by the current members in all testing areas, including, maybe, running future programs like this one.
The program is designed to provide comprehensive training for new and existing members of the Test Team. It will cover various aspects of testing, including testing practices, test team processes, and a lot of feedback and introspection to improve collaboration pathways.
The training sessions are not strictly structured. They will include a mix of presentations, hands-on activities, group discussions, and Q&A sessions to ensure an engaging learning experience. This training is meant to be interactive and to shape the future of the Test Team.
There will be no recorded sessions to encourage live participation and interaction, and one of the main intentions will be to materialize the participants experiences into a comprehensive guide for future members of the Test Team.
Introductory Material
To prepare for the training sessions, participants are encouraged to review the following introductory materials:
If you don’t have good experience with local testing environments, please check the Local Testing Environment Setup Guide (Part 1)(Part 2).
Disclaimer: This briefing was primarily generated using an AI tool from the video transcription.
Last 25th of July 2025, an informal meeting of the WordPress CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress. Testing Team, focusing on a proposed new workflow aimed at improving the quality and efficiency of bug resolution and patch delivery within the WordPress project. The discussion covered challenges with the current system, the rationale behind specializing in components, and a revised process for triaging, testing, reviewing, and committing tickets.
Key Themes and Most Important Ideas/Facts
1. The Problem: Wasted Effort and Lack of Confidence
The primary motivation for the new workflow stems from a significant amount of wasted effort in addressing tickets that are ultimately closed due to many unforeseen conflicts.
High Rate of Unsolved Tickets Growth: Approximately 80% of the reports end either in a close or a position that will not receive a successful fix for a long time (if ever). This means considerable time and effort (sometimes multiple rounds of reproduction, patching and refreshing) are invested in tickets that are ultimately discarded.
Lack of Confidence in Closing Tickets: Many contributors lack the “confidence to close tickets or proposing closing,” leading to continued work on unviable issues.
Difficulty in Predicting Conflicts: The “hard complexity” of visualizing “what are all the elements that could be conflicting” early in the process leads to late-stage discovery of unsolvable problems.
Broad Contributor Scope: Contributors currently “are working thorough the project at the same time,” most working across literally all the components. This prevents them from developing deep expertise in any single area and overlook the most important things with ease.
2. The Solution: Component Specialization and Enhanced Review
The proposed solution centres on encouraging contributors to specialize in specific WordPress components, fostering deep expertise that will enable more efficient and accurate triaging and review processes.
Specialization to earn confidence: Specialization by selecting a single WordPress component because approaching the project as a whole might not be feasible for an individual.
Mastering a Component: The goal for contributors in this group should be to master a just one component in an affordable time frame.
Improved Review Process: Specialists will be able to read any report within their selected component and triage with confidence. This early identification of unviable tickets is crucial.
The ultimate purpose is to provide definitive patches, so the committers can trust to a degree that the last review is almost bureaucratic, being so confident that the ticket is the best possible that they wouldn’t have any doubt because they already know they are treating with a specialist in that component.
3. Proposed Workflow: A “Mini-Tracker” System
A modified workflow is proposed, utilizing a GitHubGitHubGitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ project board (a “mini-tracker”) to streamline the process for tickets handled by specialists, aiming to ensure higher quality controls over the period of implementation.
New Ticket Triage by Specialist: A new report comes in (i.e., on “media”). A component specialist (i.e., “Prashant”) reads the report. Due to their expertise, they can quickly assess if it’s a problem that needs reproduction in a minimal amount of time.
Needs Reproduction TagTagTag is one of the pre-defined taxonomies in WordPress. Users can add tags to their WordPress posts along with categories. However, while a category may cover a broad range of topics, tags are smaller in scope and focused to specific topics. Think of them as keywords used for topics discussed in a particular post.: The specialist tags the ticket in TracTracTrac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. as needs-testing (or ideally generating a new tag called needs-reproduction that doesn’t exist nowadays) and then creates a corresponding entry on the GitHub project board with basic info (Trac ID, Trac URLURLA specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org, component). This board acts as a filtered list of “super clear” tickets that have been reviewed by a specialist.
Bug Reproduction by Tester: A tester (i.e., “Krupa”) picks a ticket from the “Needs Reproduction” column on the GitHub mini-tracker board, reproduces the bug, writes a report and tags it as needs-patch in Trac (or closes if it’s not valid). Then they move it to an “In progress” column on the GitHub board.
Patch Development: Someone (within or outside the group) develops the patch. Periodically, someone within the team will be checking all the “In Progress” tickets in the mini-tracker to see if they already have a patch, and move it into the next step of the mini tracker, Needs Review.
Patch Review by Specialist: The component specialist (e.g., “Prashant”) reviews the patch, suggesting modifications if needed They may tag it in Trac as changes-requested if modifications are needed, or needs-testing if code is adequate and could move into the next phase, Needs Patch Testing.
Patch Testing by Tester: After the patch is modified and approved by the specialist, a tester (e.g., “Krupa”) tests the patch with the corresponding Patch Testing Report in Trac.
Ready to Commit: Once tested and reviewed by specialists, the ticket is deemed “technically perfect and ready to be committed” minimizing the review burden on core committers.
Addressing a concern by @krupajnanda about “why having two places with the same testing information“, this “mini-track” ensures that tickets worked on by this group are those “that you know 100% that have been reviewed by a specialist,” contrasting with the “open sea of needs-testing tickets” in Trac where “anyone can put needs testing.”. We have to remember, that currently anyone that signs into Trac, can 1. Open a Ticket, 2. Add needs-testing tag at the same time as the ticket is opened. This is creating a massive inefficiency to the Test Team that has been more than proven. Anyway, testers can freely choose from either the Mini-Tracker or the Trac’s open sea, but we will encourage prioritizing first the Mini-Tracker ones for the reasons stated above.
4. Component Status Examples discussed in meeting
Media: Second most opened tickets after “General.” Probably will require more than one contributor in this area.
Posts (Post Types): Massive number of unanswered tickets due to lack of a current component maintainer.
Editor: High number of tickets without answers (over 100), good option for active GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ contributors.
TinyMCE: This type of component may be moving towards a “legacy situation” so, in the first stage, they are a not the best selection, and we should prioritize on the rest.
Application Passwords, TaxonomyTaxonomyA taxonomy is a way to group things together. In WordPress, some common taxonomies are category, link, tag, or post format. https://codex.wordpress.org/Taxonomies#Default_Taxonomies.: Suggested as good starting points for those unsure, being affordable components.
REST APIREST APIThe REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Mentioned by @pmbaldha as a component of interest.
Comments & Formatting: Mentioned by @yashjawale as components of interest for their smaller size.
5. Focus on Quality vs. Testing
The initiative emphasizes “improving the quality” of tickets, rather than just testing. It was clarified that we are not testing. The core test team’s name is acknowledged as potentially misleading, suggesting “Core Quality” might be more accurate.
6. Relationship with Component Maintainers
The new role of “reviewer” (component specialist) is closely aligned with the existing “component maintainer” role. The initiative aims to empower more contributors to ultimately become official component maintainers, addressing the current inactivity in many areas.
Reviewing as a New Skill: The initiative aims to introduce “reviewing” as a skill within the test team.
Path to Component Maintainership: The project serves as a “testing protocol” for contributors. It aims to address the perceived difficulty in becoming a component maintainer, as some contributors feel they need to “demonstrate your skills” before asking.
7. Concerns and Counterarguments
Jonathan (@desrosj), a committer, raised several concerns regarding the proposed workflow and underlying assumptions.
Duplication of Work: “We’re duplicating a lot of the work here, and what I’m hearing is that Gutenberg and track is not either accurate enough or effective enough in helping you all find things to test.“
Confusing for New Contributors: Introducing a separate project board is “confusing, especially for people that are newer to contributing,” as it adds complexity to where information should be added and tracked.
“Wasted Effort” Redefined: Jonathan argues that efforts on tickets that are ultimately closed are “are not necessarily wasted, that’s good effort that we test things, and we find that they don’t work, or they’re not appropriate“.
Committers Always Review: Jonathan refutes the idea that committers will eventually “just commit something that is tagged a certain way,” stating, “We’re always going to do our own review, our own research, our own testing.”
Documentation of Component Maintainership: A significant takeaway from the discussion is the lack of documentation on “the expectations and the requirements and the path to become a component maintainership.” Jonathan committed to documenting this in the handbook.
Focus on Improving Existing Tools: Jonathan suggests focusing on “why are the keywords not working, like how can we get better reports” within existing systems, rather than introducing new ones.
8. Pilot Program Mindset
@SirLouen emphasizes that this is an “informal”, “expiring proposal” and a “testing protocol,” not a permanent change.
Not Set in Stone: “This is not something that will be forever”. Probably the original idea will change as we will be redefining through iterative improvement“.
About Duplication: We have to remember that this is not going to replace Trac. Everything should be logged in Trac as mentioned before. All official tracking and issue management will continue to happen in Trac to ensure consistency and centralization of information. In the “Mini Tracker” only very simple information like a link to the report, the status of the report and the component associated for filtering purposes will be used. The amount of duplication will be minimal and will only serve for Quality improvement purposes.
Addressing Potential Confusion: New contributors are not going to receive this protocol without a learning session, explaining how it works. Any new contributor that lands into WP project, GitHub or Trac, can continue doing his things as always. This is not being aimed as a replacement but an improvement. Only people that voluntarily join the Quality group through any of our Onboarding sessions, will be instructed on how to use this and help on developing this new paradigm.
Reasons to avoiding anything Official in the First Stage: The new process is not being immediately pushed to the main WordPress organization because “almost anything that goes into WordPress organization is a slow process meant to last for a while“. The sole modification of a keyword that was causing filtering conflicts, took @SirLouen almost two months of multiple discussions and follow-ups through multiple dev-chats. Our proposal cannot wait for this long for every single change, as this is still a work in progress. Frequent changes are expected as the proposal becomes more concrete and mature with time: This will be happening simultaneously as the Quality group develops, Just as we did with the First Quality Report, analysing real data to draw meaningful conclusions: we’re excited to provide concrete evidence showing how adjusting certain workflows in Core could be a smart move. This way, we can build on solid facts rather than getting stuck in endless debates without the data to support our ideas.
Actionable Items for Participants
Select a Component: Attendees are encouraged to choose a specific WordPress component they wish to specialize in.
Communicate with the group: If unsure about component selection or for any questions, participants can send a message in #test-chat or directly PM any of the test team members.
Think About Component Maintainership: Consider the possibility of becoming a component maintainer in the future, with support from the presenter to build the necessary skills. This is not required, but we will help through the process.
Attend Future Sessions: Subsequent sessions will be less theoretical and focus on practical application, component selections, and addressing ongoing questions or problems.
Review Documentation: Participants are encouraged to read a future post explaining the proposed workflow and the GitHub “Mini-Tracker”
On the 4th and 18th of July, two feedback sessions were held with many contributors in the current Test Team. Valuable insights were gathered to shape a future to improve the overall Quality of CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress. contributions. If you are here just for the steps to join, go straight to the bottom.
According to the last report, several key areas for improvement were identified, including a communication gap and the misguided efforts driven by this team. On top of that, the committer activity base has been significantly declining in the past few months. The gap between contributors and committing attempts has been increasing overtime without a clear solution upfront.
To address these challenges, the idea with the biggest consensus that has raised is the following: Developing a focused group with those Test Contributors looking to gather and share knowledge, a group open to anyone to join at anytime without seniority restrictions. This initiative is made to develop a way to fill that gap: Networking comes first, and Seniority comes next.
Vision & Mission
The vision of this project is to provide such a degree of quality that any committer could be so confident that the code has gone through enough quality checks that it could be ultimately merged blindly and fearlessly.
And the mission will be to guide contributors making concentrated efforts to build specialized talent and encourage networking with the rest of the team to polish their results over time.
Tackling the Hindrances: A Workflow Overview
Looking at the current simplified but flawed workflow:
Both Report Triaging (Bug Reproduction Report) and Patch Testing (Patch Testing Report) are the areas where historically the Test Team has been doing main efforts. But we have identified two main problems with this:
The triaging challenge
Reviewing and triaging a new ticket could feel the easiest task in this world: checking if what the user reports is happening and confirming in case all seems good.
However, this simplicity can lead to overlooking more in-depth issues or missing important details that might be relevant to the whole component.
For example, not being able to recognize if the ticket poses a real problem or if this is what’s expected for many reasons (like supporting backwards compatibility expectations).
Confirming that the issue is there, and tagging it subsequently as it Needs a Patch without enough perspective, is always leading to another contributor to take it blindly, and then making a patch on a report that is probably going to end being ditched for multiple reasons we did not consider. Without a top-level triage, we are wasting the efforts of contributors unnecessarily.
The testing dilemma
Most of the reasons were already explained here. But summarizing a bit, the main issue here is that most of the patches are not valid on a first attempt because the code provided could not be attending to similar concerns as stated in the triaging part: It could cause a backwards compatibility troubles or even a potential regression, code could be too repetitive, or too verbose, it could be poorly optimized, missing coding standards and further checks etc.
So testing these weak patches, without a code review, could be valid but not ideal and definitely insufficient for further progressing on a ticket. Testing should be done in a very late-stage and just like the last check that everything is perfectly in order, ready to commit.
Not only to mention that a lack of Unit Tests is one of the main elements that leads to a lower Quality Score (according to the Quality Report). No Playwright tests have been submitted in months, and most of the PHPPHPPHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. https://www.php.net/manual/en/preface.php. code commits don’t even have enough Test Coverage.
First Proposal
The proposal that has brought more attention to achieve a new level of Quality to the organization has been the idea of having contributors specializing in one single area or component within WordPress or GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/.
This approach aims to deepen expertise, improve efficiency, and enhance the overall quality of contributions within specific domains.
For example, contributors focusing exclusively on the Media component to identify and resolve issues more effectively, gaining knowledge, knowing their real limitations, and becoming an expert voice being able to discuss on the same level as any other more experienced generalist contributor.
Moreover, being able to achieve such a degree of knowledge could enable fluid and informal conversations with other members of the group when conflicts arise among different parts of the whole system and also, while a contributor’s self-confidence gets significantly boosted through the process more polished ideas will probably arise in the future.
Structure & Organization
The intention is to organize this group in a flat leadership structure, ensuring maximum autonomy and collaboration among the members. The mentorship figure could always be present, but the target is that each member becomes their mentor in his area to provide help on such and instruct and maybe introduce future generations.
Which are the Steps to Join and Participate?
Choose a Component in Core could you be willing to start with. Note: You can pick an area you are interested in to dive deeper, or a component you have successfully contributed to already. In the Gutenberg, we have still not decided any component subdivision because there are too many Blocks & Features (92 in total) so we might need a little more time to put more thought on how to regroup them to be more affordable and logical. But if Gutenberg is your way, and you want to put some effort into it as soon as possible, just open a new conversation and will start looking at it with thoughtfulness. Also, don’t be too mindful of the selection: there is no perfect component for you, and ideally choosing one that doesn’t overlap with another contributor is preferable than overthinking this.
Once you have selected a Component (or at least you are undecided with a handful of them), join the group on our weekly onboarding sessions (First session 25th of July @ 1PM GMT, and from there, every Thursday @ 1PM GMT, 3PM CET, 6.30 IST), or directly PMing any of the current members of the group.
As soon as we agree on where you are going to start, any current member will help you to get started and progress, to make your journey as comfortable and fun as possible.
Provide the feedback and share your experience with the group.
What to expect
Nothing here is written in stone. We will be adapting to everyone and to what actually works best. We will be in a learning process for long, and maybe, in 5 or 6 months, we might start to see the initial results when the group members start to reach that level we are talking about here. Mastering the whole WordPress codebase is an almost impossible task, but mastering on a full focus is very affordable, and we definitely have big expectations for all of you.
You must be logged in to post a comment.