The Test Team helps manage testing and triage across the WordPress ecosystem. They focus on user testing of the editing experience and WordPress dashboard, replicating and documenting bug reports, and supporting a culture of review and triage across the project.
One candidate was nominated during the open nomination period and accepted their nomination. The new interim Test Team RepTeam Rep A Team Rep is a person who represents the Make WordPress team to the rest of the project, make sure issues are raised and addressed as needed, and coordinates cross-team efforts. for the current term is Brian Alexander ( @ironprogrammer ).
Meet Brian Alexander
Brian has a background that ranges from graphic design to full-stack development, digital marketing, and customer support. Since being introduced to WordPress in 2007, he has loved exploring the many ways the platform can be extended and used as a serious content management system.
Within the project, he is a full-time sponsored contributor, served as co-Test Lead for 6.0, and is the current interim Test Team Rep.
Brian lives in Portland, Oregon with his wife and daughter, and six egg-laying hens. He runs, cycles, plays the drums, and makes terrific homemade soap. And he never turns down a chance for a cup of good coffee.
You can read more about him on his profile page.
Olá! We’d like to express our gratitude to everyone who stopped by the Test Team (or CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.:Test) table at this year’s WCEU Contributor Day 🙇. Your ideas, perspectives, and open discussions help foster initiatives critical to testing WordPress. Thank you to everyone who participated!
Participants at the event covered the following topics (some of which were also referred to #core-test in Slack):
Test Report Templates
- In the proposed Test Reports guidelines, clarify how the green checkmark (✅) and red “X” (❌) emoji should be used in reports: expected vs unexpected.
- Break out the report templates into subpages under a main “Test Report” description in the Test Handbook to improve readability.
- Proposal to provide ticket creation templates for TracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. issue reporting, similar to Gutenberg Issues (e.g for Bugs vs Enhancements).
@todoInvestigate whether Trac supports pre-populated template options, for initial post and/or comments.
Easier Test Contributions
- Improve/update the Test Handbook guidance for creating a local WordPress environment.
- The desire for ephemeral test environments (no local installLocal Install A local install of WordPress is a way to create a staging environment by installing a LAMP or LEMP stack on your local computer. needed) to test PRs and patches. Some ideas:
- Add Test Handbook guidance for applying patches from GitHubGitHub GitHub is a website that offers online implementation of git repositories that can can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ or Trac, covering the various “best practice” methods.
- Reiterate the importance of different environment flavors across the test contributor group (Docker/
wp-env, VVV, Laravel Valet, Local, etc). There shouldn’t be a preference for “the best” or “one way” to run/test WordPress, since it should reflect the real-world variation across the WordPress community.
- Assign a new
week-in-testcategoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging. to Week in Test posts, for easier filtering of “where to start” in testing (currently grouped under the more generic
End-to-End (E2E) Testing
- Questions as to where to begin E2E testing in WordPress:
- It was noted that it’s common for E2E tests to fail intermittently, which can confuse and hamper development. This is often attributed to unexpected delays in DOM updates.
- Consider that E2E testing can passively validate accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) (“a11yAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility)”) as a beneficial byproduct.
- Add E2E section to Week in Test to increase awareness of this aspect of WordPress testing.
Tuesday Meetings Time Change
- It was suggested that Tuesday meetings for
<test-triage>be shifted from 17:00 to 16:00 UTC to allow broader participation from European contributors. Please share your vote or thoughts here.
Props to @boniu91 for peer review of this post.
Briefly touched on two recent blog posts to be aware of:
What’s missing and what’s stopping people from switching to blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. themes
- Responsiveness continues to come up with Ellen sharing how she built their own system to handle this for now, knowing that they can always switch over. She believes this is one of the main reasons people are holding back from switching to block themes.
- We chatted briefly through intrinsic responsiveness ideas related to this and how that’ll ease much of these tensions in time.
- Onboarding to the FSE experience was brought up, particularly around how confusing it is that the BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. label still exists since that seems to imply it’s not usable. There’s an open discussion around removing the beta label (in time) on this exact topic.
- The question came up around “How do we get folks to use block themes if there’s a beta label?” and how difficult that can be.
- @poena has a post on switching over to a block theme, Ellen is working on a post for a 10 step process, and there are clear areas that can be improved to ease this process from a technical point of view in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. (see “Ease switching to a block theme/between block themes” in this post).
- People are very confused about when to switch in general though, even if folks like Ellen are building things that are ready to go for production sites.
Communicating the value of FSE
- Nick has done some hardcore testing with folks who are new to the site editor and when watching them go into the experience and they change the typography of paragraphs but then can’t with headings! Why? We need to take a look at consistency across the tools we’re providing people. People get very frustrated and confused when one block has controls and another doesn’t.
- Block themes truly is a better experience for getting a design into WordPress but the confusion added is a bit sad for the project that it gets a rough reputation.
- Right now, it feels like more of a communication issue to the end user around what they should actually do and what they can do with consistent communication. Figma does this well.
- This has come up in DevRel for WP Engine. When you’re talking about the basics of how to do XYZ, this should be on Learn and in docs. When you’re talking about the cool, cutting edge stuff, we need more of that. “Here’s how to learn the basics of creating a block theme but here’s how to take it to the next level.”
- We discussed how if we can standardize block settings across all core blocks but allow agencies to turn on/off easily that’ll be huge for the user experience.
Patterns and opening up tooling
- We spent some time chatting about issues for unifying the pattern modals and patterns as sections work since having consistency in the interfaces for patterns and in the larger editor can really help folks take advantage of what’s possible.
- In many ways, it feels like users can rely on patterns and/or learn by doing over time as they explore more tools. As a result, exposing those tools doesn’t feel as risky as a pattern can guide the experience and, if they do want to dive in more, they can have access to the tools outright.
- We discussed how valuable locking is when it comes to patterns as a way to curate and guide the experience more.
- We went back and forth on the question of “How do we get people excited about what’s possible rather than worrying about folks breaking things?”
Difficulty with terminology
- Terminology is still very difficult and unfortunately folks need to be generally aware of what to use when.
- After 5.9, this led to the creation of support docs like this that compare reusable blocks, patterns, and template parts.
Naming of bock themes and the theme directory
- We talked about how there have been different names for block themes, like “full site editing theme” or “block-based theme”. This is causing confusion and also differs from what shows up in the Theme Directory.
- We discussed how difficult it is to find block themes in the directory since the tag you have to use is “full site editing” , which both isn’t intuitive and hard to find.
- This led to questions around having a separate menu item for themes or improving the filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output..
- Ellen shared how it’s unattractive to put free themes in the directory – “who clicks on FSE to filter?” Discoverability is so low – it’s not featured enough. She shared that they don’t put any effort into free themes.
- We all felt that the entire theme directory was due for an overhaul but were curious about what some quick solutions could be for now to make it more attractive and interesting to add block themes there.
- Perhaps there could be a block label in the section below:
Limitations of the pattern directory
- In talking about the theme directory, we discussed how neat it would be to find patterns associated with different themes, partially as a way to entice people to download that specific block theme and improve the user experience.
- Ellen brought up how it’s not possible to add patterns to the pattern directory that use third party blocks. This sometimes prevents submissions for block themes who have specific blocks for their theme.
- We discussed how the pattern directory is overwhelming for users yet also limited: you can’t use named variables for color palette + can’t use third party blocks + no curation.
- The crux of the problem is t hat block themers are creating blocks to fill gaps with core right now which then limits what can be added to the pattern directory.
- We discussed how there perhaps could be a filter to allow for third party blocks vs Core blocks. For those who want to venture into needing third party blocks, they could then opt in by filtering to show those.
- @shaunandrews recently shared a post about pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party dependencies so some of that thought/design could likely be re-used there.
- Nick shared how for the new feature in 6.0 where block themes can feature specific patterns from the directory, it’s still not granular enough. It would be nice to be ale to disable all patterns but then bring in a few from the directory to feature. There’s an issue open for this topic already!
- We ended the call talking about how these dynamics often fragment the community – people building premium themes or patterns rather than using the Core pathways. This then moves everything away from Core distribution channels and harms the community/branding/experience of WP.
- Ellen described it as feeling like you’re building against something.
- We ended the call talking about how important it is to share feedback, engage in discussions, and help influence the direction of where things go so we can get to where we need to.
This post attempts to summarize testing activities for WordPress 6.0. As currently there’s no established process for tracking testing activities within GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/, the following stats are only from TracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/. tickets.
Please note, this summary is not a complete account of testing activities, but rather a gathering of what can be tracked within Trac.
There were 234 Trac tickets closed in 6.0, excluding copy or docs only tickets.
Of these tickets:
- 5% (11 tickets) have testing instructions
- 28% (66 tickets) have automated tests
- 8% (18 tickets) have one or more test report(s)
Data Gathering Process
This section shares how the data was gathered for the stats.
- Tickets with testing instructions were identified using the Trac keyword
has-testing-info. This keyword is manually added during triage when step-by-step instructions (written or video / gif) are present within the ticket.
- Disclaimer: Other tickets may have instructions, but were not flagged with the keyword.
- Tickets with automated tests (PHPUnit, integration, and e2e tests) were identified using Trac’s
- Test Reports were gathered by manually scanning each closed 6.0 Trac ticket for the phrase
Test Reportand then verifying the presence of an actual test report.
- Disclaimer: Some tickets may have a non-formal test report such as sharing a gif or screenshot to show before and after state / behavior.
This post is a summary of the fourteenth call for testing for the experimental FSE outreach program. As always, I want to highlight those who helped to bring others along with them in this latest effort:
- @courane01 and @arasae for running a fun and informative social learning space on this call for testing.
- @piermario for creating an Italian translation with a much better recipe to bring more folks into this call for testing.
Shout out to @alixnotes for being the sole first-time contributor for this call for testing. Get excited – you will soon have a testing contributor badge on your WordPress profile!
Across many of the responses to this call for testing, it was quickly clear that folks found the Template Editor via the Post Editor uniquely confusing, especially after growing used to the Site Editor. Because of how the Template Editor interacts or doesn’t with the Post Editor, a few folks struggled to understand when they were editing the template vs the post itself. In terms of the next editions of the List and Quote blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience., there was general excitement around the new capabilities, especially once some refinements and bugs are addressed around keyboard controls for the List block.
To help ground the following feedback, here are some quotes about the overall experience to keep in mind:
The new Quote block worked well. It is now possible to add nested blocks inside it, one of the features I have long needed as a writer here at the Tavern when quoting from third-party resources…Overall, I am eager to see the finalized versions of these blocks. They will bring back some of the missing functionality from the classic editor and give users the flexibility to do even more.@greenshady in this WP Tavern post.
I only got halfway through the FSE #14 because I got too frustrated with the comments part of the challenge. I spent 40 minutes on it, and here’s my biggest takeaway. The slightly different variations of the template editing screen were just too confusing for me. As someone who has been trying to work in the FSE for a few months now, I was completely thrown off by the slightly different screen you got when you launch the template editor directly from a post vs the template editor you get when you go to edit site, and then select a template to edit.@beckej in this comment.
I really prefer not to use the Post Editor template system and instead keep all templates in the Site Editor. As it creates a consistency in how templates are created. The Post Editor template system is very different compared to the Site Editor template system. It creates a confusion in how templates are created. I look forward to being able to create multiple Post and Page templates in the Site Editor and have a simple system to where I can choose which posts and pages to attach any template to.@paaljoachim in this comment.
For any designers and developers who want to see someone walk through the experience, I want to also mention the following videos to check out and skip around to see the call for testing in action:
- Social Learning Space recording (1 hour 11 min).
- @beckej’s YouTube recording of his testing experience with timestamps here for important moments (33 min).
Here’s an example of what was created from @greenshady for this call for testing (he went above and beyond per usual):
Here’s another example of what was created with a lovely color palette by @alixnotes:
One of the testers for @courane01’s session experienced the editor crash when testing list view and attempting to modify a color but I was unable to replicate this across a few different attempts. Outside of that, the following bugs were found:
- List v2: add focus next list item after using sibling inserter (fixed in 13.3).
- List V2: Fix outdent on empty list item when pressing enter (fixed in 13.3).
- List V2: deleting sub items requires deleting twice.
When I added a sibling list item, the cursor was not in the item to start typing; I had to manually place the cursor in the list item.@antigone7 in this comment.
Since this test explored two experimental iterations of current blocks (List and Quote), much of the feature requests centered on these items.
- List v2: better paste handling.
- List v2: add the ability to have more options than just bullets or 1,2, 3.
- List v2: convert markdown based lists to list blocks.
- Quote V2: show citation block at all times.
- Quote v2: Desire for a citation block with padding/margin controls (previously reported). This is under discussion and noted for now.
- Radius controls: Show location on hover (fixed in 13.3).
- Borders: enable box-shadow options (previously reported).
It would be nice if I could copy and paste a list and have it automatically detect that and make it a list.@courane01 in this comment.
I assumed that the Add citation would be an inner block. I also assumed that I would be able to add padding/margin to the block. Both things are missing.@paaljoachim in this comment.
A new template can only be created by going to a post and clicking on “New” in the settings under Template on the right. It would be much more intuitive if you could create a new template directly in the editor.@hage in this comment.
Markdown-based lists are also not transformed into a List block when pasted into the editor. The formatting is lost, and each item gets absorbed into a Paragraph block.@greenshady in this WP Tavern post.
As lightly mentioned in the high level overview, much of the feedback fell into this categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging. with folks confused by the Template editor via the Post Editor, unsure of when they were editing the post vs template, and struggling to get layouts to cooperate (especially when it came to width controls). Sometimes these issues all combined with folks editing post content rather than the template and unable to adjust the width as they wanted as a result. Additional quotes are added below to better provide context:
- Feedback on sibling inserter experience in List V2 including how best to escape a list block (new issue).
- Desire for the ability to add a new post/page template in the Site Editor since the current Template Editor experience is visually different and disconnected.
- Improved clarification between editing the contents of a template vs a post possibly through providing a way to view a template while editing a post and colorizing more global pieces.
- Discard changes when moving between editing modes and/or allow for a way to discard changes since the interface will continually prompt you to save.
- Update the font size options to be t-shirt sizes (small, medium, etc).
- Better communicate that adding a background color also adds padding.
- Desire for consistent dimension controls across blocks since it’s currently commonplace for some blocks to have controls and options that others do not (ex: Post Author block vs AvatarAvatar An avatar is an image or illustration that specifically refers to a character that represents an online user. It’s usually a square box that appears next to the user’s name. block).
- Simplify the layout concept in order to make the controls easier to use to better ensure that when creating a new template the widths aren’t drastically different from the rest of the Post Content block. Tied to this, there needs to be a way to make it clear that a Group block needs to be added at all to have the widths be consistent.
For this final item, here’s a quick video demonstrating the problem so folks can better understand this specific pain pointPain point Pain points are “places where you know from research or analytics that users are currently getting hung up and have to ask questions, or are likely to abandon the site or app.” — Design for Real Life in the template creation process:
I also had difficulty making the Comments section the same width as the Content Group. If I used a Group Block to contain the Comment Query LoopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop., then the comments themselves could be reduced in width (using the Inherit Layout option) , but the borders were still the full width.@antigone7 in this comment.
Still trying to get “edge space” – ie margin from edge to orange borders I tried toggling on the “inherit default layout” This didnt make any different to the margin. It just changed the padding. I also added a zero to the block spacing field. Nothing changed. I tried changing the layout toggle to 80 % wide. This changed the internal padding of the block and didnt move the block away from the edges…Why are some settings in the block toolbar and some in the inspector? Why arn’t the same settings (where appropriate) in all similar blocks – ie padding, margins, width available in both groups, and columns?@alixnotes in this comment.
Still in the template editor I went to view the post. Then trying to get back to the template editor I got the page editor which informed me that there was a saved version that contained changes. (I have been caught by this before – on my own site when making changes in the templates and then going to the page editor, if I had clicked on the revert to saved I potentially might have lost all the changes that I had just tried to make. This is confusing!).@alixnotes in this comment.
The slightly different variations of the template editing screen were just too confusing for me. As someone who has been trying to work in the FSE for a few months now, I was completely thrown off by the slightly different screen you got when you launch the template editor directly from a post vs the template editor you get when you go to edit site, and then select a template to edit.@beckej in this comment.
The different controls for different blocks makes it really hard to make something that is consistent and nice. I decided it would be cool to make the user pictures a little bit bigger, like that might make the comments more inviting. Since I made the commenters pictures so big, I said, let’s add in a Post Author block so that the post author’s picture will be shown too! Wait, the Avatar block in the Comments Query loop and the Post Author have completely different control? I can make the avatar any size, but the post author I have a dropdown with 3 choices? I can but a border radius on the avatar block, but not the post author block? If it’s a picture, I should have all the same tools available to me as any other block that uses a picture.@beckej in this comment.