Tag Archives: user-testing

Mel Choyce-Dwan 10:22 pm on January 18, 2017
Tags: customization, , user-testing   

Share a walkthrough of setting up a WordPress site

A really powerful way to share your experiences using WordPress, is to actually show us how you use it to build sites. As the Editor and CustomizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. teams start working on researching and designing better experiences for coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., now is a great time to share examples of how you use them in your day-to-day work.

So, what does that mean?

Please walk us through your process of setting up a new WordPress site. I’d suggest is to pick a subject — for example, today I’m going to redesign my blog, or I’m going to build a website for a local restaurant — and then walk through the setup process, from installing WordPress, to picking a theme, to setting up some pages and writing some blog posts, to completing something you’d feel comfortable launching.

As you do your walkthrough, be sure to take notes about your experience and keep a list of any bugs you encounter.

Once you’ve reported back from your own experience, asking someone else such as a friend, family member, a coworker, and even a client would also be helpful. For some useful pointers as to how to run user tests, you can check out this handbook link.

What format is useful?

How you report your walkthrough is truly up to you.The important thing is to feel comfortable with what you share. Here are some suggestions:

  • Video (with or without audio)
    • If you do a video with audio, please also create a transcript. If you do a video without audio, feel free to just highlight the biggest pain points, “gotcha!” moments, and bugs. You can post your video to Youtube, or upload it somewhere like Cloudup or CloudApp to share.
  • Screenshots
    • You can do a series of screenshots. With each screenshot, add a caption or comment about what you’re doing, along with any highlights or observations.
  • Text
    • If you’re not comfortable or capable of doing video or screenshots, it’s also okay to just post your observations via text. Lots of times when I’m casually testing a feature, I’ll write some freeform, stream-of-conscious style notes. These are good for capturing your state of mind while you’re working and revealing particularly frustrating moments.

However you decide to do your walkthrough, please post your results here as a comment. If you’re comfortable submitting bugs you find to TracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/., add those tickets to your comments. Otherwise, describe the bug you encountered and we’ll create a ticket for you.

Thanks!

+make.wordpress.org/flow #user-testing #editor #customization

(Thanks to @karmatosed for getting the draft for this started!)

jerrysarcastic 7:24 pm on July 22, 2014
Tags: , user-testing   

Media Grid – User testing round 1

We recently performed some quick user tests on the Media Grid (at BetaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process. 2) to see how users fare when performing various tasks, and came out with some interesting results.

TL;DR – The concept of the Media Grid was intuitive and easy for users to understand, especially navigation and editing metadata; however, uploading media and bulk selection tasks were still problematic for most. A lot of work could be done to make these more straightforward.  

We also tested hiding all metadata (title, mime type, etc.) from the grid view, and saw no difference in performance, which hints at the possibility that metadata is not as important here as it is in the list view.

Follow the jump to see more details about this round of user tests. Continue reading →

#media-grid, #user-testing

designsimply 2:08 pm on February 20, 2014
Tags: user-testing,   

I’ve tested the Widget Customizer 0.14 plugin with…

I’ve tested the Widget Customizer 0.14 pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party with the Twenty Fourteen theme on WordPress r27060.

Summary

  • No trouble adding, rearranging, or removing widgets
  • Widgets can be tough to spot if they are added “below the fold”
  • Widgets that require extra prep are easily dismissed

Continue reading →

#user-testing, #widgets

designsimply 2:25 pm on January 31, 2014
Tags: user-testing,   

User Testing the Widget Customizer 0.13 Plugin I’ve…

User Testing the WidgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user. CustomizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. 0.13 PluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party

I’ve tested the Widget Customizer 0.13 plugin with the Twenty Fourteen theme on WordPress r26863.

Summary

  • No problems adding, reordering, or removing widgets
  • Content and Primary labels for widget locations weren’t clear to the user
  • Trouble with scrolling the slide-out, but it could have been a one-time user error</
  • Widgets can get a little lost if you add one to an already long list
  • ”Find widgets” search seems to obscure the last widget description
  • Feels messy to see all the widget controls stay open
  • Its not clear, at first, how to exit the customizer

Successes

No trouble adding a text widget. She clicks “Save & Pubilsh” a lot—seems to assume it’s needed to see the changes. (0:23)

No trouble reordering widgets, but again hesitates when looking for right vs left sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme.. (0:56)

No trouble removing widgets. (1:01)

Points of Confusion

She has no idea what sidebar is which. The quick red glow seems to have helped a bit. (0:31)

Scrollbar jumps back to the top, not sure what she did to trigger that. (0:12)

She expected the new widgets to be added to the top. (0:18)

Has some trouble reading the “Twenty Fourteen Ephemera” widget description at the bottom of the slide-out panel. (0:17)

Other Insights

It doesn’t seem to bother her that all the widget controls stay open — but boy does it feel messy. (0:19)

Its not clear, at first, how to exit the customizer. But she gets it eventually. (0:33)

If you’d like to see the full video, you can download it here: 2b86a9a1.mp4 (9:25)

(For Reference) Tasks Used in the Test

  • Go to Appearance > Customize Add some text in a new widget at the top of the left sidebar.
  • Look through the widgets and pick two that you like. Add both of them to the sidebar and say why you picked them.
  • Rearrange the widgets so the bio appears at the bottom of the right sidebar.
  • Remove all of the widgets from the left sidebar except the text you added before and one other widget you think is the most important to keep.
  • Verify that the updates you’ve made are visible on the site separate from the live preview in the dashboard.

User’s Computer Info

  • Operating System: Windows 7 6.1
  • Browser: Windows Firefox 26.0
  • Display: 1920

Next up: a couple more tests with version 0.14 of this plugin.

#user-testing, #widgets

designsimply 4:32 pm on November 20, 2013
Tags: user-testing,   

User Testing the Better Widgets 0.1 Plugin

Let’s keep going with the work on widgets! I’ve tested the Better Widgets 0.1 pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party with the Twenty Eleven theme on WordPress 3.7.1 and there is some interesting stuff in the results.

Summary

  • No problem at all rearranging or deleting widgets
  • Had trouble with the drag-and-drop target area at first
  • Suggests adding a way to add new widgets directly from the Widgets page
  • Uses the wrong widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user. area, completely misses the pre-expanded “Main SidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme.”
  • CustomizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. is slow

Successes

No problem rearranging widgets:

No problem deleting widgets:

Points of Confusion

Unsure where to go when asked to add a bio to the sidebar:

Drag-and-drop misfires:

Completely misses the pre-expanded “main sidebar” widgets (longest clip at 4:11):

Other Insights

Wants to know how to add more widgets from the Widgets screen:

Reasons for picking search and calendar widgets:

Unrelated to Widgets

Slow customizer is slow, takes ~24s for this user to load it up:

If you’d like to see the full video, you can download it here: Better Widgets v0.1

(For Reference) Tasks Used in the Test

  • Add a bio to the bottom of the sidebar.
  • Look through the widgets and pick two that you like. Add both of them to the sidebar and say why you picked them.
  • Rearrange the widgets so the bio appears at the top of the sidebar.
  • Remove all of the widgets except the bio and one other widget you think is the most important to keep.
  • The test website is using the Twenty Eleven theme. It has a widget area called “Showcase Sidebar.” What do you think that’s for, and how would you go about figuring out how to set it up?

User’s Computer Info

  • Operating System: Windows NT (unknown) 6.2
  • Browser: Windows Chrome 30.0.1599.101
  • Display: 1366

My Observations & Suggestions

  • The drag-and-drop misses in this video are interesting, makes me wonder if having widget areas side-by-side is good or not—or perhaps arranging them more masonry style instead of in a grid would be better
  • Interesting that she didn’t connect “sidebar” and “widgets” right away, but getting into that label is tricky business and I’d like to watch that area with more users in order to spot trends
  • Love the suggestion to have an “Add New” button on the widgets page
  • Different sidebar areas for themes is tough, I think it’s pretty hard to tell what in the world the “Showcase Sidebar” is for if you’re not familiar with Twenty Eleven
  • It’d be great to focus on customizer performance speed

Next up: same test, except run it on WP 3.8 trunk. What specifics are you interested in finding out from tests like these?

#user-testing, #widgets

designsimply 8:04 pm on October 24, 2013
Tags: user-testing,   

User Testing Widgets+MP6 v1 Layout

I’ve analyzed user testing for two videos shaunandrews kicked off for v1 of Widgets + MP6. The clips represent the main points of interest and user confusion from the tests and range from 12 to 49 seconds, so they should be quick and easy to watch.

Expects clicking “Save” will close the widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.:

Troubles with drag and drop:

  • Windows 7 6.1
  • Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
  • 1920 x 1080

Hover tip proves useful when figuring out the MetaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. widget:

No trouble with drag and drop to add a widget:

Expects widget to close after saving:

  • Windows Vista 6.0
  • Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
  • 1280 x 800

If you’d like to see the full videos, you can download them here: Widgets+MP6 001, Widgets+MP6 002.

Key Observations

  • One user spotted the hover tips and the other didn’t
  • Both users said they expected the widget to close after saving
  • May need to increase sensitivity of the widget drop area, check Chrome 30 on Win7 for issues

Just because a couple users mention they’d like to see a widget area close after saving doesn’t necessarily mean you should do that. Additional testing in situations where someone is working for a longer time with one particular widget and may want to save periodically would be good. Adding some other visual indicator to show when saving is finished might be sufficient, as opposed to closing the widget on save.

#user-testing, #widgets

designsimply 3:15 am on October 23, 2013
Tags: , user-testing   

THX38 0.7.1 User Testing

I put THX38 0.7.1 through some testing. Here are some clips showing the main points of interest. Most are around a minute or so in length, some are shorter.

Accidentally clicks into the customizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. and gets disoriented, first click on the “Customize” button doesn’t work (not sure why), thinks the magnifying glass hover icon indicates search:

While the themes page is blazingly fast 🙂 the customizer is slow:

Suggests the “Delete Theme” button should be red all the time:

Overlooks “Customize” button on the active theme:

“Delete Theme” is a little hard to find for this user:

Wishes clicking on the large screenshot in the modal would show a full view:

Wants more screenshot detail:

“Delete Theme” process not immediately visible, but found fairly easily:

Theme description details are a little too technical, i.e. “post format-packed,” HTMLHTML HTML is an acronym for Hyper Text Markup Language. It is a markup language that is used in the development of web pages and websites., SEO:

“Delete Theme” is easy:

Keeps clicking the back button instead of the “x” in the modal and it disorients her a little:

Tries right-clicking to delete:

If you’d like to see the full videos, you can download them here: THX38 001, THX38 002, THX38 003, THX38 004, THX38 005.

What would make finding really good themes easier? (survey question responses)

  • Tags or categories for subject matter, colors, or any number of style ideas and needs
  • Marketing maybe? Make it free?
  • If I were given an option of more than 20 themes to choose from, I would like to be able to choose by categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging., color theme, or complexity.
  • I can’t think of anything to really beat browsing. I mean, you could categorize them and even list them according to common uses (like Church) but I think I’d have to keep looking to find the perfect one. If you categorized based on HTML5 and all that, I think you’d lose a chunk of your user base (although having more technical categories for advanced users isn’t a bad idea at all).
  • The ability to search by genre. e.g. if I was a hairdresser key in the term hairdresser and get back a range of themes that could work for that business model. Complete with a choice of images would be fantastic.

Key Observations

  • The magnifying glass/search icon was only mentioned by one user. The visibility icon could still be a good alternate because an eye symbol is often used for “view” online.
  • Themes load very fast! This makes it especially noticeable when the customizer loads slowly.
  • Viewing most themes on a brand new WordPress install is lackluster because there isn’t much data to customize per se.
  • Users had no trouble finding “Preview,” “Activate,” and “Delete Theme” in the modal when hover buttons were removed. Consider taking the hover buttons out. I know this adds one click to activating a theme (pushback if you feel strongly about it).
  • Noteworthy user suggestion:  make the “Delete Theme” button red all the time
  • Noteworthy user suggestion:  make it so you can click the screenshots in the modal to enlarge them (carousel style?) or make them zoomable

All in all, the pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party is in really good shape. It’s fast! The “delete theme” link doesn’t get triggered accidentally but is still relatively easy to find. All of the bugs found via user testing were fixed in plugin version 0.8. If I could vote for two updates for v1, I would recommend: remove preview and activate hover buttons, change hover icon to a “view” symbol.

#thx38, #user-testing

Dave Martin (lessbloat) 8:40 pm on October 2, 2012
Tags: , user-testing   

Decided to run another user through this afternoon…

Decided to run another user through this afternoon:

Here’s the video, and my notes:

Step two notes – Log in

No problems.

Step two notes – Explain what you see

No problems.

Step Three notes – Preview your blog

No problems.

Step Four notes – Change your background color

3:22 – She tried typing “blue” into the hex text field – didn’t work.
3:26 – No issues figuring out the color selector. Yay!

Step Five notes – Change your site title

No problems.

Step Six notes – Add your first post

No problems.

Step Seven notes – Preview your new post

No problems.

Step Eight notes – Publish an image

7:55 – Our first successful test of the new media flow. Double yay!

Recap

Smashing success. Came in at 8:32 for all tasks. Quickest time yet I do believe…

#discovery-cycle-1, #user-testing

Dave Martin (lessbloat) 5:37 pm on October 2, 2012
Tags: , user-testing   

Ran another user through Discovery Cycle 1 this…

Ran another user through Discovery Cycle 1 this morning.

Here’s the video, and my notes:

Step two notes – Log in

No problems.

Step two notes – Explain what you see

No problems.

Step Three notes – Preview your blog

No problems.

Step Four notes – Change your background color

  • 3:36 – He had to play around a bit, but he figured out how to change the color to blue fairly quickly
  • 3:52 – He clicked the “default” button resetting the work he had done to choose a color.

Step Five notes – Change your site title

No problems.

Step Six notes – Add your first post

No problems.

Step Seven notes – Preview your new post

No problems.

Step Eight notes – Publish an image

  • 7:20 – Cool, he clicks into new media workflow. But unfortunately he’s looking to paste a URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org, which isn’t currently in the new media workflow just yet.
  • 9:05 – He discovers the “Upload/insert” link, and has no trouble adding the URL.

Recap

  • Overall, I’m super happy with this test. He completed everything in 10 min which is a great time (compared to where we started off at the beginning of the 3.5 cycle).
  • Happy to see the first user try the new media flow, even if it didn’t work out for him. We’ll continue to to test new users as that section fills out.
  • 7:30 – When he entered a URL into the media uploader, it appeared to succeed for a brief second, you saw a file upload happening briefly and then it disappeared (just a heads up @koopersmith).
  • Do we need to do anything about the “default” button in the color picker? Would be a shame if someone took a bunch of time to find just the right color, then lost it with a click.

#discovery-cycle-1, #user-testing

Dave Martin (lessbloat) 4:03 pm on September 3, 2012
Tags: , user-testing   

I sent two additional users through discovery cycle…

I sent two additional users through discovery cycle 1 on Friday afternoon. The four initial tests can be found here.

User #5 through these scenarios

Here’s the video, and my notes:

Step two notes – Log in

No problems.

Step two notes – Explain what you see

No problems.

Step Three notes – Preview your blog

Found it after a bit of trial and error

Step Four notes – Change your background color

5:20 – Yay! She was able to use the color picker.

Step Five notes – Change your site title

5:55 – No problems.

Step Six notes – Add your first post

6:42 – No problems. Used the “+ new” menu

Step Seven notes – Preview your new post

No problems.

Step Eight notes – Publish an image

9:40 – No problems – Another instance of a user literally copying and pasting the image into the visual editor.

Recap

Quickest user test so far. By far. Yay!!!!

User #6 through these scenarios

Here’s the video, and my notes:

Step two notes – Log in

  • Takes 2 times to log in, but no problems.
  • She see’s the big IE is out of date notice.
  • 2:20 – “It’s a little distracting”, referring to dashboard in general.

Step two notes – Explain what you see

No problems.

Step Three notes – Preview your blog

No problems. Clicked link in welcome screen.

Step Four notes – Change your background color

  • 3:30 – hunting and pecking through left nav
  • 5:00 – tries using color picker (which semi-works in IE7/8?), she selects a color from the right hue slider, but it doesn’t update the color. 🙁
  • 5:50 – comes back a second time, and only see’s hex text box – user gives up.

Step Five notes – Change your site title

  • 6:30 – User ends up in Appearance -> headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. (this is about the 3rd user out of 6 that have tried it here). Note: we need to add a link on this page.
  • 7:40 – Still trying to find it in appearance -> header
  • 8:24 – Finds it in settings -> general

Step Six notes – Add your first post

9:30 – IE7/8? woes adding title in “Add new post”

Step Seven notes – Preview your new post

No problems.

Step Eight notes – Publish an image

No problems.

Recap

  • So we’ve still got problems with the color picker. I think it’s slightly unfair to judge this too critically since the user is using IE7, or 8?
  • I’m going to work up a quick patch to add a link to edit the users site title from the appearance -> header page.

#discovery-cycle-1, #user-testing