When you conduct user testing, you are measuring to see if the feature or patch accomplishes its intended purpose.
The types of user testing that WordPress needs help with are:
- Manually testing new features yourself
- Testing those same features with other users
Before You Test Before You Test
- Setup a test site either locally or with your web hosting provider
- Install the WordPress Beta Tester plugin
- Find a feature or patch to test
- Make sure you know the intended purpose of the feature
- Install the feature or patch you want to test onto your test site
- Make sure to reset the test site between every test
Do a Walk-through Do a Walk-through
With the end goal in mind, do a walk-through of the feature and take notes as you go, or record your session and take notes when you play it back. If you are testing a feature early in the design process, more open-ended suggestions and potential points of confusion are appropriate to note. If you are testing a feature later in the development process, then your main focus should be to look for blockers in the design decisions that have already been made.
One of the fastest and easiest ways to write up a task list for usability testing is to find out the feature’s end goal and then do a walk-through yourself. Take notes as you go through each step, or record the walk-through and then write down the steps when you play back the video. This will also help you set an expectation for how long the test may take others.
If you use any assistive technologies, such as JAWS or any other screen reading software, testing using those would be a great addition to user testing.
Write a Task List Write a Task List
Start with a specific scenario. These are imaginary situations that help people get into the right mindset for completing a task. Feel free to have fun with it! Choose a scenario that mimics the real world as much as possible, so that people can engage with the tasks as if they are real. Here is an example:
You’re setting up a website for a new department. You need to set some people up so they can add content, and others will need to be able to submit content that must be reviewed before it can be published.
Tasks should be realistic and clearly written without describing the steps in too much detail. Use language you think would be used by users, and avoid giving clues like specific words used on the site. You’ll also want to avoid asking people what they “think,” because you want to focus on what they actually do and not what they think they might do.
- Goal: Look up a user role.
- Poor task: You want to see the role for your co-worker. Go to the website, sign in, and tell me where you would click to find user roles.
- Better task: Look up the rights for your co-worker, Sally, to make sure she is able to publish new content.
Test with Other Users Test with Other Users
Choose a location where you are likely to find your target audience. WordPress meetup groups are a great resource. Another potential source of volunteer testers is your local WordCamp, where people can sign up to help test during breaks—ask your local WordCamp organizers if they have space for a breakout area for user testing. Some people also find testers at coffee shops.
It is common to compensate users for their time, though many WordPress contributors will volunteer their time if you ask up front. At meetup groups, buying food like snacks or pizza for the group can be a good incentive for adding some time to the meetup for testing. When testing, don’t underestimate the power of chocolate, or offering to cover the cost of a cup of coffee in exchange for a few minutes of their time. Giving a small gift card is reasonable for a short test. Longer tests should be paid. If you don’t have a budget for testing, consider asking your local WordPress group if they might be able to work with you to raise money for user research.
When conducting a test, encourage users to think out loud and either record their onscreen interactions with a tool such as QuickTime or take notes during the test. Try not to be leading or answer questions. Just pay close attention to what they do onscreen as they think aloud and find their way through the tasks. Your goal should be to take out as much bias as possible and pay attention to what you can observe someone doing.
Reporting Results Reporting Results
Whether you are reporting results from a walkthrough or from a test with another user, the most valuable results you can report are bugs that block you from completing the goal or trends about pain points and satisfaction rates. A short “Top 5” list of bugs and pain points should be at the top of any report followed by notes, personal suggestions (labeled personal as they may have bias), and details of the test.
If you recorded a video, it’s helpful to include a link to it in your report or a highlight reel if you have tested with several people.