Now that we’ve got a couple of rough patches (welcome screen, color picker, “view” my site) it’s time go back and test a few users in our first discovery cycle with these patches applied, and see if we’ve made any progress.
Here’s the video, and here are my notes:
Step two notes – Log in
Step two notes – Explain what you see
Step Three notes – Preview your blog
- 4:00 – She’s looking for a link
- 4:14 – “Oh here it is” – She clicks “Visit my blog” in toolbar (Win for “view” my site patch).
Step Four notes – Change your background color
- 5:45 – When asked to change the background color, she immediately (instinctively) clicked the big blue “customize my site” button (woot!).
- 5:55 – she easily found the BG color option in the customizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings..
- 6:00 – Oh noooes, she found a bug in Iris (so painful to watch…)! clicking the right column of Iris does’t change the color in the text-field (After a bit of testing, it looks like if you click the right column first, before clicking the left gradient box, nothing happens. If you click, if you click the left box first, then the right column, it works as expected.) cc// @mattwiebe
- 8:07 – “Each time I view my blog it gives me a slightly different picture”.
- 9:10 – still trying to figure out why BG color isn’t changing – “I’m so confused”
- 10:30 – Moved on without being able to figure out how to change BG color. 🙁
Step Five notes – Change your site title
- 10:50 – Looking in Appearance -> Header The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. for a place to change her site title (which really makes sense, we need to add a link there).
- 11:10 – didn’t find it in Appearance -> Header, no she’s checking all up and down the left nav
- 12:20 – Found it in the customizer
- 12:30 – What the? She couldn’t add spaces? That is weird. There is a bug in one of the patches? cc// koopersmith can you spot any reason why this would be happening in one of the patches listed above?
- 13:20 – “I’m very confused – I don’t know how I would insert spaces”
- 14:22 – gave up without being able to add spaces…
Step Six notes – Add your first post
Step Seven notes – Preview your new post
Step Eight notes – Publish an image
She found how to upload the image – not going to worry about anything after that for now, since media manager is being revamped. We’ll do plenty of user testing on the new media manager once it’s complete.
- Bug in Iris prevented her from changing her background color
- Can we please NOT have rotating header images in twenty twelve? None of the users we’ve tested have really comprehended what’s going on there.
- We really should add a link on “Appearance -> Header” to “Settings -> General” to change their site title or tagline.
- There was a bug in the customizer when she tried to update her site title which wouldn’t allow her to add spaces.
I’d like to get the color picker bug fixed before I run another user through these changes, but overall, I’d say these patches have already really improved things for first-time users. Thoughts?
In reviewing the first 6 user tests, one thing really stands out to me (it’s a glaring problem in each of the tests). Can you guess what it is?
It might not be what you’d expect…
Continue reading →
Round 3 of the “WP as a CMS” cycle is ready for your viewing enjoyment. 🙂 Previous user tests for this discovery cycle can be found here and here.
This one’s audio is a bit fuzzy at times. I changed up a few of the tasks to be a bit more clear on what I was wanting this guy to do. Have a look, and please highlight anything you spot (that could be fixed) in the comments.
Experiencing technical difficulties with the video – Sorry – I’ll get it posted as soon as I get that sorted…
Here’s round 2 of the “WordPress as a CMS” user test. You can view round one here.
As much as I love spending 2+ hours making notes on each user test video, I’m going to try a different approach with this one. As you watch this one, please make a note in the comments if you spot anything that could be fixed. I’ve found the “2:23 – Something happened here” format to work pretty well.
Here’s the start of another discovery cycle (#2). This one is focused around WordPress as a CMS (using it to build a website vs. for a blog). I’ll need to make some edits to this script before I run any other users through it.
You can download the video and follow along if you’d like (however, this one is super painful to watch – you’ve been warned!)
Continue reading →
It’s summary time.
We’ve run 3 users through the same set of scenarios (via usertesting.com). You can review the results here, here, and here.
Now… What exactly are we trying to accomplish with all of this?
It’s important to note that we have to think about these discoveries in the context of the typical WP user. But to be honest, I’m not entirely sure I know what the “typical WP user” looks like (but I hope to dig into that over the next couple of months ;-)). Sure, I have assumptions, as I’m sure you do, but my assumptions aren’t backed by any sort of data.
The important thing to keep in mind here is that these 3 people were paid to perform a list of tasks. While we can benefit from watching their interactions (and looking for patterns of complexity/confusion), we need to recognize that the flow they follow is prescribed, and likely different in some ways from an actual user.
Overview of discoveries
Here’s an unfiltered list of observations across all of the tests:
- Had trouble figuring out how the color selector worked (x3)
- Unsure what “QuickPress” was
- Unsure where “1 post”, “1 page” came from under “Right Now”
- Had lots of trouble finding a link to view their site
- User clicked “Select files” when they actually had a URL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org to paste (and should have used the “From URL” tab). (x3)
Post Add New
- Lost all changes when she clicked a link taking her away from the new post page (after already making some changes)
- Didn’t notice “Insert into post” button when adding an image in the media modal
- Clicked “Preview” button, but didn’t realize a new tab had opened (causing confusion)
- Doesn’t appear to be a clear path for users that just wish to post a photo
- Had trouble knowing where to go to change her site title (x3)
- User didn’t notice links in toolbar (x2)
- User never noticed “+ new” dropdown (x2)
- The blog dropdown menu is completely different when in the admin vs. on site (should it be?)
- Confusion over why the header The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. image was changing (x2)
This is where I need your insight/input.
1) First, is there anything that you noticed that’s not on this list?
2) Next, which of these do you think we should attempt to address (are there any quick wins that would benefit all WP users)?
3) If you had to prioritize your list, what would you tackle first?
I’ll hold off for a day before I post my thoughts. Don’t be shy! 😉 I’d love to hear what you think.
Round 3 of this “Discovery Cycle” is ready to preview. Check out round one & round two, if you’ve not already.
Here’s the video for this round.
Continue reading →
I’m back with results from round 2 of our “Discovery Cycle“. This lady was certainly a bit more savvy. If you haven’t seen round one, you might want to check that out first. I left the detailed user instructions out of the summary this time, since they’re the same for both users.
Here’s the video, if you’d like to follow along.
Continue reading →
I thought I’d introduce myself, as I hope to start spending some time contributing to core Core is the set of software required to run WordPress. The Core Development Team builds WordPress..
I’ve been designer at Automattic for close to 2 years (working on Akismet, VideoPress, and CodePoet). Before Automattic, I was the lead UI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. designer at Campaign Monitor for 2 years. I look forward to meeting you.
I’m just getting my feet wet, and quite honestly haven’t a clue where to get started ;-), so I thought I’d set up a quick user test (I’m a big fan of user testing). I set up a temporary WP install, and ran a user from usertesting.com through a couple scenarios.
You can download the video here if you’re interested.
Here are my notes (sorry, they’re kind of long):
Continue reading →