The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
Please drop by any time in SlackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. with questions or to help out.
After one month of very intensive activity, we have finally reached the end of the test team program. We would like to thank all the participants for their dedication and hard work throughout this period. The program has been a great success in many areas, and we have gathered valuable insights and feedback that will help us improve our whole contribution onboarding process.
During the program, we started with a total of 9 participants, but after some expected dropouts, we ended with 6 members, with most participants doing a fantastic job during the entire process. They were involved in tasks such as testing, documentation improvements, leading meetings, and a lot of feedback to support the team’s growth.
In a dedicated SlackSlackSlack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel, we have been able to work very closely with the participants, gathering information about their experience through the process and also sharing the progress of this program. There was no clear starting program structure, but one happened to begin shaping as weeks went by that could be described as follows for the record:
Program Weekly Structure
The first week was focused on onboarding all members on the testing protocol as soon as possible, because one of the main targets was to go through a significant amount of tickets through the program period.
During the second week, we started introducing the meeting protocols, both for patch testing scrubbing and how to run the weekly test meetings with the corresponding agenda and summary post publishing. We also started to gather feedback on the testing protocol because the initial test results started to pop up.
For the third week, we switched the focus to documentation improvements, and we started to gather feedback on the meeting protocols and keep it up on the testing part. The contributor pathway video program began to come together.
Finally, for the last week, we tried to clear up all the final questions and analyze the current state of WordPress in correlation with the testing team to set future goals for the coming months.
Program Results
Overall, the program has exceeded our expectations in terms of engagement and results. Some goals were shared with the participants in the first interview, but from the experience we had from past programs, we knew that generally these goals were challenging to meet and could not be met. However, in this case, we have been able to achieve most of the goals and even exceed some of them. Here are some of the key results we have achieved:
Testing Reports
At the beginning of the program, there were a total of 487 tickets with the needs-testing label. By the end of the program, this number has dropped to 264, which is a significant decrease of almost 50%. This is by far one of the biggest achievements. We are pleased to observe that the protocol has been refined to a point that members were able to go through tickets at an excellent pace, understanding the whole process with proficiency. This will probably translate into a more efficient process in the future.
Documentation Improvements
Improving internal protocol documentation is something that requires more experience and time inside the team. However, we have been able to gather a lot of feedback and proposals for documentation improvements in our GitHub repository, which is a great starting point for the future. We have already started working on some proposals, and we hope to have them published in the following weeks.
The Crown Jewel: Test Contributor Pathway in Progress
One of the main goals of this program was to create a video training with a clear pathway for future contributors. We are delighted to announce that the program is almost completed, and we are planning to have it ready in a couple of weeks. A lot of feedback has been gathered through the program, and soon there will be an announcement in case anyone wants to join the “betaBetaA pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. program” to test the training and provide feedback before the official launch.
Participant Engagement Analysis as a Blueprint for Future Test Team Aspirants
We believe that sharing the results of the program participants could be useful for future WordPress contributors to understand which level of engagement is expected from them if they want to be part of the Test Team. And furthermore, to discover the different ways they can contribute to the team.
1. Ozgur (@ozgursar): Worked on a total of 68 testing reports, drove a test-chat and started leading to a documentation improvement regarding email testing. For the next few weeks, we expect the docs page to get published and a patch testing scrubbing meeting to be led to complete the whole circle. He is the first participant proposed to join the Test Team and continue his journey with us.
2. Huzaifa (@huzaifaalmesbah): Worked on a whopping total of 89 testing reports, which has been massive, and also proposed a documentation improvement regarding the `Getting started for testing` page structure. The only thing he has missed is leading some different meeting sessions, but there is already one scheduled for next week, and we are sure that with all the knowledge he has now, he is more than ready to lead more sessions in the future. He is the second participant suggested for joining the team.
3. Juanma (@juanmaguitar): He has been extremely active leading proposals for documentation improvements and providing a ton of feedback during all sessions, including a triaging guide, test-chat protocol guide, and some tips on post-tag improvement during a test-chat session. He has also led one test-chat session, but the only downside is that he has only been involved in testing 3 tickets, clearly the only weaker point that we hope could be improved in the following weeks to be somewhat on par with the rest of the participants. He is the third participant proposed for joining the team, and we are sure that with a bit more involvement in the reporting part, he can be a great asset for the team.
4. Erick (@r1k0): He worked through a grand total of 52 testing reports and also led one of the patch testing scrubs. There is only one thing that he has missed to go through the whole process, and it is the documentation improvement work and jumping into a couple more meeting leading sessions, but we are sure that he is more than ready to do it in the following weeks. He is the fourth participant recommended for joining the team, and we hope he jumps into the documentation part as soon as possible to be able to be on the team with a more complete profile.
5. Shazzad (@sajib1223): He was already active as a test team contributor before the program, but during the program, he has been able to consolidate many doubts he had about the test team protocols. He was able to run a patch testing scrub, but still in the other areas he has been lagging a bit. With no documentation proposals yet, only 10 tickets, and no test-chat sessions, we hope he can get quickly up to speed in the following weeks to be able to join the Test Team officially.
6. Mohammed (@mohkatz): He has been the last participant that has attended the whole program sessions, but unfortunately, he has not met the minimal requirements. With no testing reports, no documentation proposals, and no meeting-leading sessions. Hopefully, if he gives the team more time, he can get up to speed in the following weeks and be able to join the Test Team in the near future.
As I commented in the beginning, the other 3 participants that were selected dropped out in the beginning of the program for different reasons, and we hope that they can get involved in the future if they’d like to.
Future Directions and Organizer Personal Thoughts
As the organizer, I can’t stress anymore that this program has gone great, but simultaneously, I have to acknowledge that it has been very exhausting to organize. Running future programs like this is uncertain, and probably more organization and resource gathering will be required to be able to make it happen again in the following months. The dedication required from the organizers and participants is very high. Not only the two hours required for the live weekly sessions, but also the time to go through questions in the Slack chat, manually review most of the reports done and the documentation proposals, and also the time to create the training ideas.
Results of a program like this are proven to be excellent, but we need to find a way to make it more sustainable for the future. Luckily, now we have a couple of members onboarding in the team now and some additional ones probably soon, and we hope that they can take some of the leading load that has been driven by the current members in all testing areas, including, maybe, running future programs like this one.
You must be logged in to post a comment.