The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
Please drop by any time in SlackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. with questions or to help out.
On the 4th and 18th of July, two feedback sessions were held with many contributors in the current Test Team. Valuable insights were gathered to shape a future to improve the overall Quality of CoreCoreCore is the set of software required to run WordPress. The Core Development Team builds WordPress. contributions. If you are here just for the steps to join, go straight to the bottom.
According to the last report, several key areas for improvement were identified, including a communication gap and the misguided efforts driven by this team. On top of that, the committer activity base has been significantly declining in the past few months. The gap between contributors and committing attempts has been increasing overtime without a clear solution upfront.
To address these challenges, the idea with the biggest consensus that has raised is the following: Developing a focused group with those Test Contributors looking to gather and share knowledge, a group open to anyone to join at anytime without seniority restrictions. This initiative is made to develop a way to fill that gap: Networking comes first, and Seniority comes next.
Vision & Mission
The vision of this project is to provide such a degree of quality that any committer could be so confident that the code has gone through enough quality checks that it could be ultimately merged blindly and fearlessly.
And the mission will be to guide contributors making concentrated efforts to build specialized talent and encourage networking with the rest of the team to polish their results over time.
Tackling the Hindrances: A Workflow Overview
Looking at the current simplified but flawed workflow:
Both Report Triaging (Bug Reproduction Report) and Patch Testing (Patch Testing Report) are the areas where historically the Test Team has been doing main efforts. But we have identified two main problems with this:
The triaging challenge
Reviewing and triaging a new ticket could feel the easiest task in this world: checking if what the user reports is happening and confirming in case all seems good.
However, this simplicity can lead to overlooking more in-depth issues or missing important details that might be relevant to the whole component.
For example, not being able to recognize if the ticket poses a real problem or if this is what’s expected for many reasons (like supporting backwards compatibility expectations).
Confirming that the issue is there, and tagging it subsequently as it Needs a Patch without enough perspective, is always leading to another contributor to take it blindly, and then making a patch on a report that is probably going to end being ditched for multiple reasons we did not consider. Without a top-level triage, we are wasting the efforts of contributors unnecessarily.
The testing dilemma
Most of the reasons were already explained here. But summarizing a bit, the main issue here is that most of the patches are not valid on a first attempt because the code provided could not be attending to similar concerns as stated in the triaging part: It could cause a backwards compatibility troubles or even a potential regression, code could be too repetitive, or too verbose, it could be poorly optimized, missing coding standards and further checks etc.
So testing these weak patches, without a code review, could be valid but not ideal and definitely insufficient for further progressing on a ticket. Testing should be done in a very late-stage and just like the last check that everything is perfectly in order, ready to commit.
Not only to mention that a lack of Unit Tests is one of the main elements that leads to a lower Quality Score (according to the Quality Report). No Playwright tests have been submitted in months, and most of the PHPPHPPHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. https://www.php.net/manual/en/preface.php. code commits don’t even have enough Test Coverage.
First Proposal
The proposal that has brought more attention to achieve a new level of Quality to the organization has been the idea of having contributors specializing in one single area or component within WordPress or GutenbergGutenbergThe Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/.
This approach aims to deepen expertise, improve efficiency, and enhance the overall quality of contributions within specific domains.
For example, contributors focusing exclusively on the Media component to identify and resolve issues more effectively, gaining knowledge, knowing their real limitations, and becoming an expert voice being able to discuss on the same level as any other more experienced generalist contributor.
Moreover, being able to achieve such a degree of knowledge could enable fluid and informal conversations with other members of the group when conflicts arise among different parts of the whole system and also, while a contributor’s self-confidence gets significantly boosted through the process more polished ideas will probably arise in the future.
Structure & Organization
The intention is to organize this group in a flat leadership structure, ensuring maximum autonomy and collaboration among the members. The mentorship figure could always be present, but the target is that each member becomes their mentor in his area to provide help on such and instruct and maybe introduce future generations.
Which are the Steps to Join and Participate?
Choose a Component in Core could you be willing to start with. Note: You can pick an area you are interested in to dive deeper, or a component you have successfully contributed to already. In the Gutenberg, we have still not decided any component subdivision because there are too many Blocks & Features (92 in total) so we might need a little more time to put more thought on how to regroup them to be more affordable and logical. But if Gutenberg is your way, and you want to put some effort into it as soon as possible, just open a new conversation and will start looking at it with thoughtfulness. Also, don’t be too mindful of the selection: there is no perfect component for you, and ideally choosing one that doesn’t overlap with another contributor is preferable than overthinking this.
Once you have selected a Component (or at least you are undecided with a handful of them), join the group on our weekly onboarding sessions (First session 25th of July @ 1PM GMT, and from there, every Thursday @ 1PM GMT, 3PM CET, 6.30 IST), or directly PMing any of the current members of the group.
As soon as we agree on where you are going to start, any current member will help you to get started and progress, to make your journey as comfortable and fun as possible.
Provide the feedback and share your experience with the group.
What to expect
Nothing here is written in stone. We will be adapting to everyone and to what actually works best. We will be in a learning process for long, and maybe, in 5 or 6 months, we might start to see the initial results when the group members start to reach that level we are talking about here. Mastering the whole WordPress codebase is an almost impossible task, but mastering on a full focus is very affordable, and we definitely have big expectations for all of you.
