WordPress.org

Ready to get started?Download WordPress

Make WordPress UI

Tagged: user-testing Toggle Comment Threads | Keyboard Shortcuts

  • Jerry Bates (jerrysarcastic) 7:24 pm on July 22, 2014 Permalink | Log in to leave a Comment
    Tags: , user-testing   

    Media Grid – User testing round 1 

    We recently performed some quick user tests on the Media Grid (at Beta 2) to see how users fare when performing various tasks, and came out with some interesting results.

    TL;DR - The concept of the Media Grid was intuitive and easy for users to understand, especially navigation and editing metadata; however, uploading media and bulk selection tasks were still problematic for most. A lot of work could be done to make these more straightforward.  

    We also tested hiding all metadata (title, mime type, etc.) from the grid view, and saw no difference in performance, which hints at the possibility that metadata is not as important here as it is in the list view.

    Follow the jump to see more details about this round of user tests. (More …)

     
    • Helen Hou-Sandi 2:49 am on July 23, 2014 Permalink | Log in to Reply

      Bulk select is a tricky interaction – the persistent checkboxes are already round 2. What can we do there to make it better? Fade more things out? Re-explore a separate mode?

      @melchoyce and I discussed hiding all those extra fields by default and just letting people turn them on if they really want them (that is, the power users that tend to volunteer for interviews). I think that is a good route to go, though a part of me feels that “off by default” makes it silly.

    • Eric Andrew Lewis 2:51 pm on July 23, 2014 Permalink | Log in to Reply

      I’d like to revisit and user test a fixed-to-bottom toolbar where bulk action buttons can go.

    • Mickey Kay 1:05 am on July 24, 2014 Permalink | Log in to Reply

      I like this idea. Would love to see/test.

    • Matt McLaughlin 8:41 pm on July 29, 2014 Permalink | Log in to Reply

      I just fundamentally don’t get the “Select function from drop down, then hit this separate button for ‘activate function'” UI. Why would you not simply have a delete button? I’ve never really understood the way word press does bulk actions. And if you’re going to go with buttons, why would you put them at the bottom rather than the top?

      Isn’t the simplest solution here to just replace the bulk actions pull down + apply buttons with buttons for the actions themselves? If you have more actions than fit in the bar, make the rightmost a “More” pulldown for the least used actions.

      You should also separate out the “Changes the filters or views” functionality from the “Does something to the images I’ve selected” functionality. My suggestion would be to put View, Filters and Search on one bar and the “do something” buttons like delete on a second bar just below it.

    • Matt McLaughlin 8:56 pm on July 29, 2014 Permalink | Log in to Reply

      Sorry for the late replies, but a few more thoughts:

      1) Not only is the meta-data not necessary, but the check box thing isn’t necessary either. A much more visually elegant way to do this would be to use the bright blue border to indicate selected, and at the same time grayscale out the non-selected images. Use standard optional keystrokes for multi-select (shift selects everything in between and command does non-contiguous multi-select). If you’re worried about discoverability put a little tool tip somewhere.

      2) A “Deselect All” Button would be nice

      3) If you’re going to add a button bar with Edit, Delete, etc. then it makes sense to put the Upload/Add media button in that bar instead of weirdly floating by itself next to the title.

      4) It’s a little hokey, but if you’re worried about people not realizing they’re editing multiple photos you can have the buttons change names on multi select i.e. “delete” becomes “delete all”.

  • designsimply 2:08 pm on February 20, 2014 Permalink
    Tags: user-testing,   

    I’ve tested the Widget Customizer 0.14 plugin with the Twenty Fourteen theme on WordPress r27060.

    Summary

    • No trouble adding, rearranging, or removing widgets
    • Widgets can be tough to spot if they are added “below the fold”
    • Widgets that require extra prep are easily dismissed

    (More …)

     
  • designsimply 2:25 pm on January 31, 2014 Permalink
    Tags: user-testing,   

    User Testing the Widget Customizer 0.13 Plugin

    I’ve tested the Widget Customizer 0.13 plugin with the Twenty Fourteen theme on WordPress r26863.

    Summary

    • No problems adding, reordering, or removing widgets
    • Content and Primary labels for widget locations weren’t clear to the user
    • Trouble with scrolling the slide-out, but it could have been a one-time user error</
    • Widgets can get a little lost if you add one to an already long list
    • ”Find widgets” search seems to obscure the last widget description
    • Feels messy to see all the widget controls stay open
    • Its not clear, at first, how to exit the customizer

    Successes

    No trouble adding a text widget. She clicks “Save & Pubilsh” a lot—seems to assume it’s needed to see the changes. (0:23)

    No trouble reordering widgets, but again hesitates when looking for right vs left sidebar. (0:56)

    No trouble removing widgets. (1:01)

    Points of Confusion

    She has no idea what sidebar is which. The quick red glow seems to have helped a bit. (0:31)

    Scrollbar jumps back to the top, not sure what she did to trigger that. (0:12)

    She expected the new widgets to be added to the top. (0:18)

    Has some trouble reading the “Twenty Fourteen Ephemera” widget description at the bottom of the slide-out panel. (0:17)

    Other Insights

    It doesn’t seem to bother her that all the widget controls stay open — but boy does it feel messy. (0:19)

    Its not clear, at first, how to exit the customizer. But she gets it eventually. (0:33)

    If you’d like to see the full video, you can download it here: 2b86a9a1.mp4 (9:25)

    (For Reference) Tasks Used in the Test

    • Go to Appearance > Customize Add some text in a new widget at the top of the left sidebar.
    • Look through the widgets and pick two that you like. Add both of them to the sidebar and say why you picked them.
    • Rearrange the widgets so the bio appears at the bottom of the right sidebar.
    • Remove all of the widgets from the left sidebar except the text you added before and one other widget you think is the most important to keep.
    • Verify that the updates you’ve made are visible on the site separate from the live preview in the dashboard.

    User’s Computer Info

    • Operating System: Windows 7 6.1
    • Browser: Windows Firefox 26.0
    • Display: 1920

    Next up: a couple more tests with version 0.14 of this plugin.

     
    • shaunandrews 2:33 pm on January 31, 2014 Permalink | Log in to Reply

      Awesome. Thanks for doing these tests — they’re super insightful and helpful.

      She has no idea what sidebar is which. The quick red glow seems to have helped a bit. (0:31)

      That red glow has always been a sort of “placeholder” — we’ll work on making it more obvious, and slightly better looking.

    • Weston Ruter 9:58 pm on January 31, 2014 Permalink | Log in to Reply

      She expected the new widgets to be added to the top.

      Actually, widgets used to be added at the top. There was a dropdown SELECT to choose a widget, and then it got prepended to the list of widgets. @shaunandrews Should the “Add a widget” link move to the top?

      • Sheri Bigelow (designsimply) 2:00 pm on February 20, 2014 Permalink | Log in to Reply

        I could see it appearing at the top and bottom if there were a ton of widgets in use… but if I had to pick between one or the other, I’d probably leave the “Add a Widget” link at the bottom because that’s where new widgets get placed automatically in the live preview so it makes sense that the button would also be at the bottom of the widget list.

    • Weston Ruter 10:00 pm on January 31, 2014 Permalink | Log in to Reply

      Content and Primary labels for widget locations weren’t clear to the user

      Would this actually be an issue with the theme itself? These labels are defined by the Theme.

    • Weston Ruter 10:03 pm on January 31, 2014 Permalink | Log in to Reply

      It doesn’t seem to bother her that all the widget controls stay open — but boy does it feel messy. (0:19)

      We talked about automatically collapsing a widget control when another is opened, but I thought that maybe they’d want to have more than one open to make it easier to copy/paste between widgets, or to compare two widget controls. Also, the widgets admin page allows multiple widgets to be expanded at a time.

    • Weston Ruter 10:07 pm on January 31, 2014 Permalink | Log in to Reply

      “Find widgets” search seems to obscure the last widget description

      This seems to be a browser rendering bug, but I can’t reproduce on Firefox 26 on OS X nor on Chrome on OS X. @shaunandrews can you reproduce this bug?

    • Weston Ruter 10:09 pm on January 31, 2014 Permalink | Log in to Reply

      Widgets can get a little lost if you add one to an already long list

      What if we added the new widget icons to prepend the widget titles? We have them in the widget addition panel, but we’re not using them in the widget controls. The icons would help locate and differentiate between widgets in a long list. Specifically, I’m talking about adding the icon at .widget-title > h4:before. @shaunandrews?

  • designsimply 4:32 pm on November 20, 2013 Permalink
    Tags: user-testing,   

    User Testing the Better Widgets 0.1 Plugin 

    Let’s keep going with the work on widgets! I’ve tested the Better Widgets 0.1 plugin with the Twenty Eleven theme on WordPress 3.7.1 and there is some interesting stuff in the results.

    Summary

    • No problem at all rearranging or deleting widgets
    • Had trouble with the drag-and-drop target area at first
    • Suggests adding a way to add new widgets directly from the Widgets page
    • Uses the wrong widget area, completely misses the pre-expanded “Main Sidebar”
    • Customizer is slow

    Successes

    No problem rearranging widgets:

    No problem deleting widgets:

    Points of Confusion

    Unsure where to go when asked to add a bio to the sidebar:

    Drag-and-drop misfires:

    Completely misses the pre-expanded “main sidebar” widgets (longest clip at 4:11):

    Other Insights

    Wants to know how to add more widgets from the Widgets screen:

    Reasons for picking search and calendar widgets:

    Unrelated to Widgets

    Slow customizer is slow, takes ~24s for this user to load it up:

    If you’d like to see the full video, you can download it here: Better Widgets v0.1

    (For Reference) Tasks Used in the Test

    • Add a bio to the bottom of the sidebar.
    • Look through the widgets and pick two that you like. Add both of them to the sidebar and say why you picked them.
    • Rearrange the widgets so the bio appears at the top of the sidebar.
    • Remove all of the widgets except the bio and one other widget you think is the most important to keep.
    • The test website is using the Twenty Eleven theme. It has a widget area called “Showcase Sidebar.” What do you think that’s for, and how would you go about figuring out how to set it up?

    User’s Computer Info

    • Operating System: Windows NT (unknown) 6.2
    • Browser: Windows Chrome 30.0.1599.101
    • Display: 1366

    My Observations & Suggestions

    • The drag-and-drop misses in this video are interesting, makes me wonder if having widget areas side-by-side is good or not—or perhaps arranging them more masonry style instead of in a grid would be better
    • Interesting that she didn’t connect “sidebar” and “widgets” right away, but getting into that label is tricky business and I’d like to watch that area with more users in order to spot trends
    • Love the suggestion to have an “Add New” button on the widgets page
    • Different sidebar areas for themes is tough, I think it’s pretty hard to tell what in the world the “Showcase Sidebar” is for if you’re not familiar with Twenty Eleven
    • It’d be great to focus on customizer performance speed

    Next up: same test, except run it on WP 3.8 trunk. What specifics are you interested in finding out from tests like these?

     
    • Weston Ruter 5:55 pm on November 20, 2013 Permalink | Log in to Reply

      Perhaps the slow loading for the customizer was just a network problem? It seems like an external script was blocking the page loading. In my experience the customizer loads very fast.

      • designsimply 7:27 pm on November 20, 2013 Permalink | Log in to Reply

        Could be. I’ve seen the problem show up more than once in tests, so I’ll keep watching to see if it continues to be a trend.

    • danhomer4 2:15 am on December 17, 2013 Permalink | Log in to Reply

      Windows NT 6.2 is Windows 8, for those confused

  • designsimply 8:04 pm on October 24, 2013 Permalink
    Tags: user-testing,   

    User Testing Widgets+MP6 v1 Layout 

    I’ve analyzed user testing for two videos shaunandrews kicked off for v1 of Widgets + MP6. The clips represent the main points of interest and user confusion from the tests and range from 12 to 49 seconds, so they should be quick and easy to watch.

    Expects clicking “Save” will close the widget:

    Troubles with drag and drop:

    • Windows 7 6.1
    • Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
    • 1920 x 1080

    Hover tip proves useful when figuring out the Meta widget:

    No trouble with drag and drop to add a widget:

    Expects widget to close after saving:

    • Windows Vista 6.0
    • Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
    • 1280 x 800

    If you’d like to see the full videos, you can download them here: Widgets+MP6 001, Widgets+MP6 002.

    Key Observations

    • One user spotted the hover tips and the other didn’t
    • Both users said they expected the widget to close after saving
    • May need to increase sensitivity of the widget drop area, check Chrome 30 on Win7 for issues

    Just because a couple users mention they’d like to see a widget area close after saving doesn’t necessarily mean you should do that. Additional testing in situations where someone is working for a longer time with one particular widget and may want to save periodically would be good. Adding some other visual indicator to show when saving is finished might be sufficient, as opposed to closing the widget on save.

     
    • Ulrich 8:24 pm on October 24, 2013 Permalink | Log in to Reply

      I think the problem with the saving of the widgets is that there is no confirmation anywhere that the widget is saved.

    • Archetyped 8:28 pm on October 24, 2013 Permalink | Log in to Reply

      Save on close

      Another way to pose this is: why shouldn’t the widget button close on save?

      Or perhaps the better question is: Are the actionable options provided to the user the right ones?

      If you break it down, the actions available to users are basically:

      • Edit widget settings
      • Close widget without making any further changes
      • Delete widget

      Therefore, perhaps more appropriate user actions would be:

      • Done — Save widget and close
      • Cancel — Close widget without saving changes
      • Delete — Completely remove widget

      In the current UI, you could make the argument that the widget should not close on save because the user may just want to save their progress as they continue to work on the widget’s settings.

      But why should the user have to do this? This is a burden that the machine should be offloading from the user.

      Post progress is saved automatically as you edit a post, why shouldn’t widgets have the same intelligence?

      With the machine saving your progress, the UI no longer needs a “save” button. You can focus on your work until you’re done. Then you just have to decide what your final action will be– save or cancel your changes (or remove widget entirely).

      • Jon Brown 12:02 am on October 25, 2013 Permalink | Log in to Reply

        WordPress saving changes is confusing enough as it is (metadata/featured image on posts. I wouldn’t want incremental changes to widgets saving prior to clicking save. Especially if the widget settings are versioned and undo-able.

        The buttons should read “close” and then upon making changes the button should change “save” then after saving change back to “close”. While there were changes to save a link for cancel changes and close would replace the current link for close next to delete.

        Personally I don’t want the widget closing on save, I just want a stronger confirmation that my changes were actually saved. I often want two widgets open at the same time so I can copy settings between the two.

      • designsimply 12:30 am on October 25, 2013 Permalink | Log in to Reply

        Take user suggestions like this one with a grain of salt. Sometimes people say one thing and then turn around and do another. So you have to look at a group of tests and find trends, and even then I think there’s a little magic that goes into interpreting the tests. :)

        Post progress is saved automatically as you edit a post, why shouldn’t widgets have the same intelligence?

        I imagine it’s because you don’t always want your widget changes live until they are finished—if you auto-saved, your widgets would be all skewampus on the front end during editing.

        I’m not advocating for one solution over another btw. I think it’d be cool to test the different scenarios more.

      • designsimply 12:32 am on October 25, 2013 Permalink | Log in to Reply

        Thanks for the feedback btw!

      • shaunandrews 2:34 am on October 25, 2013 Permalink | Log in to Reply

        Post progress is saved automatically as you edit a post, why shouldn’t widgets have the same intelligence?

        The key difference here is that the auto-save functionality that you see in the post editor is saving a revision. Widgets don’t have the concept of revisions. So, if widgets auto-saved, they’d be pushing out new versions of your widget to your live site. Not ideal.

        Now, one day we may have the ability to save revisions of widgets — then auto-save makes a lot of sense. However, you’d still want a “Save” button, or perhaps a “Publish” button, to push your changes to your site.

      • Archetyped 7:03 am on October 25, 2013 Permalink | Log in to Reply

        Yes, I understand the data infrastructure that allows posts to autosave progress as revisions, etc. My point is that users should not need to care about any of that.

        Obviously nothing should be live until the user commits those changes. Apologies if it seemed the opposite was implied. I’m also not saying autosave should be implemented without the necessary infrastructure changes, but that infrastructure should conform to user needs, not other way around.

        Take user suggestions like this one with a grain of salt.

        Indeed. My comments are not based on a single user’s comments but rather based on the usability issues observed working with clients over the years.

        • designsimply 2:13 pm on October 25, 2013 Permalink | Log in to Reply

          Apologies if it seemed the opposite was implied.

          No worries, I think it’s just a current technical limitation that widgets are confined to for this iteration. Frustrating :) so I think people like the ideas but are just explaining limitations.

          My comments are not based on a single user’s comments but rather based on the usability issues observed working with clients over the years.

          Awesome! i’m glad to read about your insights and that you’re participating.

    • shaunandrews 2:35 am on October 25, 2013 Permalink | Log in to Reply

      A huge thanks to @designsimply for going through these user tests and pulling out the highlights. Thanks, Sheri!

    • designsimply 9:06 pm on November 5, 2013 Permalink | Log in to Reply

      We reviewed all of the feedback from the comments on this post in our weekly THX chat today. RTL, back button, and escape to exit are going to be worked on. It’s too late in the project to change to an inline overlay layout for the details page, but anyone interested is encouraged to make a plugin showing how it could work if they would like to experiment with something like that. Feedback in yesterday’s core dev chat said that bulk edit wasn’t needed, and it’s not quite ready anyway so we took it out.The team talked about adding a trash icon to the main screen but decided against it because the delete in the modal is probably enough and users were able to find it pretty quickly in user testing.

      Thanks for all the feedback!

  • designsimply 3:15 am on October 23, 2013 Permalink
    Tags: , user-testing   

    THX38 0.7.1 User Testing 

    I put THX38 0.7.1 through some testing. Here are some clips showing the main points of interest. Most are around a minute or so in length, some are shorter.

    Accidentally clicks into the customizer and gets disoriented, first click on the “Customize” button doesn’t work (not sure why), thinks the magnifying glass hover icon indicates search:

    While the themes page is blazingly fast :) the customizer is slow:

    Suggests the “Delete Theme” button should be red all the time:

    Overlooks “Customize” button on the active theme:

    “Delete Theme” is a little hard to find for this user:

    Wishes clicking on the large screenshot in the modal would show a full view:

    Wants more screenshot detail:

    “Delete Theme” process not immediately visible, but found fairly easily:

    Theme description details are a little too technical, i.e. “post format-packed,” HTML, SEO:

    “Delete Theme” is easy:

    Keeps clicking the back button instead of the “x” in the modal and it disorients her a little:

    Tries right-clicking to delete:

    If you’d like to see the full videos, you can download them here: THX38 001THX38 002THX38 003THX38 004THX38 005.

    What would make finding really good themes easier? (survey question responses)

    • Tags or categories for subject matter, colors, or any number of style ideas and needs
    • Marketing maybe? Make it free?
    • If I were given an option of more than 20 themes to choose from, I would like to be able to choose by category, color theme, or complexity.
    • I can’t think of anything to really beat browsing. I mean, you could categorize them and even list them according to common uses (like Church) but I think I’d have to keep looking to find the perfect one. If you categorized based on HTML5 and all that, I think you’d lose a chunk of your user base (although having more technical categories for advanced users isn’t a bad idea at all).
    • The ability to search by genre. e.g. if I was a hairdresser key in the term hairdresser and get back a range of themes that could work for that business model. Complete with a choice of images would be fantastic.

    Key Observations

    • The magnifying glass/search icon was only mentioned by one user. The visibility icon could still be a good alternate because an eye symbol is often used for “view” online.
    • Themes load very fast! This makes it especially noticeable when the customizer loads slowly.
    • Viewing most themes on a brand new WordPress install is lackluster because there isn’t much data to customize per se.
    • Users had no trouble finding “Preview,” “Activate,” and “Delete Theme” in the modal when hover buttons were removed. Consider taking the hover buttons out. I know this adds one click to activating a theme (pushback if you feel strongly about it).
    • Noteworthy user suggestion:  make the “Delete Theme” button red all the time
    • Noteworthy user suggestion:  make it so you can click the screenshots in the modal to enlarge them (carousel style?) or make them zoomable

    All in all, the plugin is in really good shape. It’s fast! The “delete theme” link doesn’t get triggered accidentally but is still relatively easy to find. All of the bugs found via user testing were fixed in plugin version 0.8. If I could vote for two updates for v1, I would recommend: remove preview and activate hover buttons, change hover icon to a “view” symbol.

     
    • Lea Cohen 6:26 am on October 23, 2013 Permalink | Log in to Reply

      I have been following the posts and I love this plugin. If I could request one thing, I would appreciate RTL treatment :)
      I’ve never collaborated before in such things, so am not sure if and how I can help, but if there’s any way I can – I’ll be glad to.

      • Matías Ventura 4:50 pm on October 23, 2013 Permalink | Log in to Reply

        Good call — yes RTL will be added for sure. You are welcome to contribute anyway you can. We have chats on IRC on Tuesdays if you want to join. Otherwise you can keep participating on these threads.

        • Lea Cohen 12:32 pm on October 24, 2013 Permalink | Log in to Reply

          Could I help in implementing the RTL css in the plugin?

          • designsimply 8:15 pm on October 24, 2013 Permalink | Log in to Reply

            You could submit a patch. Do you know how to make one? I’d recommend starting with something very small and checking in with matveb to make sure you’re on the right track before doing more.

            • Lea Cohen 6:28 am on October 28, 2013 Permalink

              No, I’ve never submitted a patch. I agree with you that something very small would be a good start.

            • designsimply 9:04 pm on October 28, 2013 Permalink

              Do it! I found a basic tutorial that could help: http://www.doitwithwp.com/submit-patch-wordpress-core-beginners/ and you can ping me in #wordpress-core-plugins (or anyone in #wordpress) if you have questions.

            • Lea Cohen 6:21 am on November 4, 2013 Permalink

              Thank you so much, @designsimply. That’s a wonderful tutorial.

              I tried installing IRC on my computer, but we have a problem with open ports at my work. I’ll try contacting you from home, since I have a couple of questions. Thanks again for your support and encouragement!

    • biznis86 12:29 pm on October 23, 2013 Permalink | Log in to Reply

      I already suggested a concept without the popup modal, but haven’t received any response. Here is what I was thinking of:

      Also:

      • Move theme name above theme screenshot
      • Move all theme action buttons to the bottom of screenshot
      • designsimply 2:41 pm on October 23, 2013 Permalink | Log in to Reply

        The modal window is performing well in user testing. What problem are you trying to solve by trying to change it? I think one of the reasons for the theme name and action buttons being at the bottom of things is to put emphasis on the visuals, the screenshots specifically. Are your suggestions to move the theme name and action buttons a personal preference? Or how are you proposing that change will benefit users?

      • designsimply 3:30 pm on October 23, 2013 Permalink | Log in to Reply

        Btw, personal preferences are cool to bring up (you’re a user too!), but it would be awesome to include why you’d like to see things moved and how you think it will help along with the suggestions if you can.

        Aside: feedback on design (and reacting to it) is not as easy as it seems :) and my initial response sounded too defensive to me (maybe?) when I re-read it after posting. Anyway, I’m always looking for ways to get better at the feedback thing, both sides of it—just something that’s on my mind.

        • biznis86 3:56 pm on October 23, 2013 Permalink | Log in to Reply

          Sure, here are my arguments:

          Modals are not a good user experience, especially not on mobile devices. Inspiration behind my design comes from redesigned google image search, which was previously modal and is now similar to what I’ve posted. It works very well, at least for me. I really hate when links open in new window and I hate modals for the same reason. They add confusion and navigate user from their focus field.

          Theme name: there’s no reason to place it inside screenshot as it is not immediately visible to user. To make the name more obvious you need to add a background color to it and wrap everything with borders (which you did). But why? It would be simpler and nicer if you just added the name on top of screenshot, so the screenshot is cleaner and more obvious to user.

          Action buttons (activate, preview, delete, customize): It confuses users if they see buttons positioned differently on active theme and inactive themes and would be better to place them all at the bottom of screenshot. If you add theme name on top as previously suggested, you can also add “Active theme” text where theme name is currently positioned.

          I hope what I posted made any sense. As my company makes 90% of revenues from WP, it’s in my best interest to help as much as I can ;)

          • designsimply 4:13 pm on October 23, 2013 Permalink | Log in to Reply

            The modal isn’t used for mobile. The design is responsive. Try testing it out! Also, the modal isn’t done in a separate page load, so it’s super fast, which might help. Also, my role is more testing than design, so maybe one of the designers will chime in about modal vs. inline theme details.

            Ah, but I think the intention is to take away focus from the theme name and put it on the visual, the screenshot.

            It might be cool to experiment with user testing with the buttons in the detail view moved to the top. Are you interested in coding it up and running a test?

            • biznis86 4:26 pm on October 23, 2013 Permalink

              Do you need me to code our version of THX38 plugin or can we send over layered PSD file to save the dev work? Either way, no problem, first one will only take a bit more time.

              Otherwise, here is a fully sized screenshot of my previous image.

              I still believe, no matter how good modal is, it is never better than inline editing. Still, thank you for participating in this conversation – it just proves WP cares about users ;)

            • designsimply 4:31 pm on October 23, 2013 Permalink

              Do you need me to code our version of THX38 plugin or can we send over layered PSD file to save the dev work?

              For the button moving from the bottom to the top inside the modal? I’d suggest making a patch. I imagine you could do it with a bit of CSS. If you patch it, I’ll test it. Oh hey, I noticed the buttons are at the bottom of the description in your inline theme details mockup too! ;)

              it just proves WP cares about users

              True! so. true.

      • Matías Ventura 4:37 pm on October 23, 2013 Permalink | Log in to Reply

        One of the problems with the inline overlay is that we’ll run out of space quickly for the things we’d like to do, namely: more screenshots (gallery), showcase of real blogs using a theme, support forums, etc.

        Also, it doesn’t feel like the best use of space to duplicate the information (theme name + screenshots) on the same visual canvas.

        The overlay on mobile is a full screen page. Nothing indicates it’s a “modal” on that environment.

        Having said that, if anyone wants to code a version with the inline view and test it, I’d be happy to discuss it. I’m hesitant myself of modals, but in this case (with the url updating, arroy keys navigation, and being quite fast) think of it as a really quick “new page”. You can send people to themes.php#theme/twentyfourteen and it will open the theme for them, with all the details and actions at hand in a focused screen.

        Action buttons (activate, preview, delete, customize [...] you can also add “Active theme” text where theme name is currently positioned.

        We had this in one of the versions and eventually decided against it. The theme name is not that important and throws off the layout balance a bit when it is at the top.

        • designsimply 4:39 pm on October 23, 2013 Permalink | Log in to Reply

          themes.php#theme/twentyfourteen and it will open the theme for them

          I love that.

        • biznis86 5:09 pm on October 23, 2013 Permalink | Log in to Reply

          Not sure I agree that we’ll run out of space, since the modal is of the same size as inline overlay I suggested. More screenshots can be added the same way as to the modal. And user is never confused or disoriented. Although, some info is duplicated on the visual canvas, it follows user action, so there is no confusion about its purpose.

          One other benefit to inline overlay is also that you actually don’t need to “Add new” themes in a separate page. You could add new themes the same way with inline overlay at the end of themes list (with same Add New thumbnail you have now), so it would feel like you’re in control of your overall themes list and nothing would distract you from building it.

          You are spot on with URL updating and sending people to selected theme, but I’m not sure how many users will actually use this. This feature would only be usable if I could send my customers to their admin, where they could open/preview my custom theme (not yet installed) – if that is the case here, I’m really impressed and blown away really.

          I guess this is more my preference than a general opinion, so even if I feel really strongly about my case, I’ll stop here and won’t add more confusion to it.

      • Paal Joachim Romdahl 6:41 pm on October 23, 2013 Permalink | Log in to Reply

        Inline.

        • Keeps the connection with the other themes even when previewing.
        • It is easy to select another theme in the open preview area. One could also scroll through using the arrow keys having one theme after another open as a preview. Move the mouse inside the open preview and scroll to see additional screenshots and features. Or perhaps have small thumbnails below the main preview image. Clicking them opens into the main preview area. It seems like a good idea to code. To test out modal and inline methods.
    • Matías Ventura 4:25 pm on October 23, 2013 Permalink | Log in to Reply

      Thanks a lot for doing these.

      “Keeps clicking the back button instead of the “x” in the modal and it disorients her a little”.

      We can probably make it so the browsers back button takes you back to the grid view. I had it working with pushState but had to switch to normal hashes due to some troubles dealing with the .php extension of the themes page.

      • designsimply 4:33 pm on October 23, 2013 Permalink | Log in to Reply

        I thought you might appreciate the short video clip reporting format. :D

        Making the back button trigger the grid view would be awesome.

        Another idea: make the escape key exit the modal.

        • Matías Ventura 4:46 pm on October 23, 2013 Permalink | Log in to Reply

          Escape would be good, too.

        • Bryan Petty 6:56 pm on October 23, 2013 Permalink | Log in to Reply

          Both sound like good ideas, and escape is certainly standard.

          I think in general that all modal windows should probably be pushed onto the history stack honestly. Less of a specific theme issue here, and more of a general observation.

      • Andy Mercer 1:08 am on October 24, 2013 Permalink | Log in to Reply

        Having the back button return to the grid doesn’t necessarily make sense, but users (myself included) have been trained for decades that when you click something that makes the page change, the back button will undo it.

        Though it will probably add complexity to the code, I’d say it will be worth it just for the comfort factor, and for easing people in. It can always be removed down the line someday.

        • designsimply 9:16 pm on October 28, 2013 Permalink | Log in to Reply

          Clicking the back button could be perceived as broken if you’re in theme details view :) and it has to do with how the modal is setup. You can see it in the test above labeled “Keeps clicking the back button” where she clicks back and ends up on the main dashboard then has to navigate back to Appearance > Themes.

    • Archetyped 6:38 pm on October 23, 2013 Permalink | Log in to Reply

      Some thoughts/suggestions based on the videos

      Delete Theme

      • Delete option in grid view would be nice so that users can easily delete themes without having to go into a modal for every single theme. Using a simple trash can icon would be pretty much universally understood
      • Then, replace the “delete” button in the modal with the same trash can icon to standardize the UI in both views. This will make it less obtrusive while still easy to find when desired.

      View Icon

      The “eye” icon is scary and doesn’t really communicate “more detail” as well as it communicates “Eye of Ra”. The magnifying glass icon is a better direction and the functionality could be even better communicated by adding a “+” in the magnifying glass, which would give us the standard “zoom in” icon. It would communicate “zoom in for more detail” better than the eye.

      Modal

      The responsive modal works well. Normally a modal is a problem on mobile because they are generally not responsive so you have wasted space and positioning/scaling issues when the user interacts with the page. Here the modal is quite usable on mobile with large buttons and a simple layout.

      • designsimply 7:57 pm on October 23, 2013 Permalink | Log in to Reply

        Thanks for watching the videos!

        Two of the tests experimented with taking out all of the hover buttons all together. Even though a couple testers commented that the “delete theme” link is a little bit hard to find at first, most of them found it within a few seconds of looking around. Plus, it shouldn’t be too terribly easy to delete a theme, so I think the tests show it’s in line with the designers’ intention.

        It’s good to hear specific feedback about the icon. I never thought of the eye as scary! :)

        Cool feedback on the responsive design. I agree.

        • landwire 9:34 am on October 24, 2013 Permalink | Log in to Reply

          Delete:
          I also think there should be a “Delete” option in the grid view. Especially if there is another window popping up saying “Do you really want to delete XXXX theme”. So it cannot really happen accidental and that is the main concern I would think.

          • Matías Ventura 1:43 pm on October 24, 2013 Permalink | Log in to Reply

            I don’t think the frequency of deleting themes warrants the placement on the front — specially with the goal being to streamline the interface weight.

            We did play with a “bulk edit” mode in which you get an “x” to delete themes quickly. We couldn’t find an elegant way to invoke it, though. Still totally open to improving this in the future. Not sure it’s an area that warrants a lot of attention at this stage.

            https://cloudup.com/cdC2ME6Y1oK

            • Ipstenu (Mika Epstein) 2:04 pm on October 24, 2013 Permalink

              As someone who handles a lot of cases where people download a bajillion themes (seriously I deleted 30 off someone’s site this week) testing them out, and then never upgrade, a bulk delete option really is needed. Its VERY common for newer users to do that, and every time I give a session in how to use WP to the non-Tech people, I get asked ‘Why can’t I bulk delete?’

              I get why it’s not there now, but it needs to stay on the radar please :)

            • Matías Ventura 2:53 pm on October 24, 2013 Permalink

              Hah, thanks for the feedback from the front! It’s definitely on the radar, and we may very well get to it soon. Open to ideas about how to invoke the edit mode, if possible without another button next to “add new”. :)

        • Archetyped 8:09 pm on October 24, 2013 Permalink | Log in to Reply

          Why wouldn’t we want theme deletion to be easy? It’s an action that a user would feasibly want to do, so why discourage the it? There’s nothing inherently “bad” about deleting a theme.

          As theme deletion is destructive in nature, a confirmation of the action is important, which the popup handles. There’s no other reason to make it any more difficult than that.

          A trash can icon could be unobtrusive enough to not distract the user (like an always-on hot red button might), but would still be recognizable enough. In fact, after ages of UI conditioning, you could reason that a trash can icon is what users would actually be looking for when they want to delete a theme.

          Having to read the text of every button slows the user down from performing the desired action. While the a red-colored button may help to draw the eye to the button, it does nothing in terms of accessibility (e.g. color-blind users).

          • designsimply 8:17 pm on October 24, 2013 Permalink | Log in to Reply

            Looks like we’re going to try some bulk delete options, and I’m sure we’ll discuss a trash icon option in our next office hours. Thanks for the feedback!

  • lessbloat 8:40 pm on October 2, 2012 Permalink
    Tags: , user-testing   

    Decided to run another user through this afternoon:

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems.

    Step Four notes – Change your background color

    3:22 – She tried typing “blue” into the hex text field – didn’t work.
    3:26 – No issues figuring out the color selector. Yay!

    Step Five notes – Change your site title

    No problems.

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    7:55 – Our first successful test of the new media flow. Double yay!

    Recap

    Smashing success. Came in at 8:32 for all tasks. Quickest time yet I do believe…

     
  • lessbloat 5:37 pm on October 2, 2012 Permalink
    Tags: , user-testing   

    Ran another user through Discovery Cycle 1 this morning.

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems.

    Step Four notes – Change your background color

    • 3:36 – He had to play around a bit, but he figured out how to change the color to blue fairly quickly
    • 3:52 – He clicked the “default” button resetting the work he had done to choose a color.

    Step Five notes – Change your site title

    No problems.

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    • 7:20 – Cool, he clicks into new media workflow. But unfortunately he’s looking to paste a URL, which isn’t currently in the new media workflow just yet.
    • 9:05 – He discovers the “Upload/insert” link, and has no trouble adding the URL.

    Recap

    • Overall, I’m super happy with this test. He completed everything in 10 min which is a great time (compared to where we started off at the beginning of the 3.5 cycle).
    • Happy to see the first user try the new media flow, even if it didn’t work out for him. We’ll continue to to test new users as that section fills out.
    • 7:30 – When he entered a URL into the media uploader, it appeared to succeed for a brief second, you saw a file upload happening briefly and then it disappeared (just a heads up @koopersmith).
    • Do we need to do anything about the “default” button in the color picker? Would be a shame if someone took a bunch of time to find just the right color, then lost it with a click.
     
  • lessbloat 4:03 pm on September 3, 2012 Permalink
    Tags: , user-testing   

    I sent two additional users through discovery cycle 1 on Friday afternoon. The four initial tests can be found here.

    User #5 through these scenarios

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    Found it after a bit of trial and error

    Step Four notes – Change your background color

    5:20 – Yay! She was able to use the color picker.

    Step Five notes – Change your site title

    5:55 – No problems.

    Step Six notes – Add your first post

    6:42 – No problems. Used the “+ new” menu

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    9:40 – No problems – Another instance of a user literally copying and pasting the image into the visual editor.

    Recap

    Quickest user test so far. By far. Yay!!!!

    User #6 through these scenarios

    Here’s the video, and my notes:

    Step two notes – Log in

    • Takes 2 times to log in, but no problems.
    • She see’s the big IE is out of date notice.
    • 2:20 – “It’s a little distracting”, referring to dashboard in general.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems. Clicked link in welcome screen.

    Step Four notes – Change your background color

    • 3:30 – hunting and pecking through left nav
    • 5:00 – tries using color picker (which semi-works in IE7/8?), she selects a color from the right hue slider, but it doesn’t update the color. :-(
    • 5:50 – comes back a second time, and only see’s hex text box – user gives up.

    Step Five notes – Change your site title

    • 6:30 – User ends up in Appearance -> header (this is about the 3rd user out of 6 that have tried it here). Note: we need to add a link on this page.
    • 7:40 – Still trying to find it in appearance -> header
    • 8:24 – Finds it in settings -> general

    Step Six notes – Add your first post

    9:30 – IE7/8? woes adding title in “Add new post”

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    No problems.

    Recap

    • So we’ve still got problems with the color picker. I think it’s slightly unfair to judge this too critically since the user is using IE7, or 8?
    • I’m going to work up a quick patch to add a link to edit the users site title from the appearance -> header page.
     
    • toscho 11:10 am on September 4, 2012 Permalink | Log in to Reply

      I think I will add the ability to edit the blog title on the header appearance page to my themes from now on. These videos are real eye-openers. :)

    • snaushads 7:15 pm on September 5, 2012 Permalink | Log in to Reply

      Im new to Make WordPress, though ive been using wordpress for 2 yrs (not as a developer), I got to know how the UI is being worked upon by Matts “State of the Word 2012″.
      And yes the video made me realize that its not the way “i” think users use WordPress.
      How can i contribute as a non-programmer ?

      I understand the correct url would be: http://make.wordpress.org/ui/tag/discovery-cycle-1/

      Thanks.

  • lessbloat 4:48 pm on August 27, 2012 Permalink
    Tags: , user-testing   

    Now that we’ve got a couple of rough patches (welcome screen, color picker, “view” my site) it’s time go back and test a few users in our first discovery cycle with these patches applied, and see if we’ve made any progress.

    Here’s the video, and here are my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    • 4:00 – She’s looking for a link
    • 4:14 – “Oh here it is” – She clicks “Visit my blog” in toolbar (Win for “view” my site patch).

    Step Four notes – Change your background color

    • 5:45 – When asked to change the background color, she immediately (instinctively) clicked the big blue “customize my site” button (woot!).
    • 5:55 – she easily found the BG color option in the customizer.
    • 6:00 – Oh noooes, she found a bug in Iris (so painful to watch…)! clicking the right column of Iris does’t change the color in the text-field (After a bit of testing, it looks like if you click the right column first, before clicking the left gradient box, nothing happens. If you click, if you click the left box first, then the right column, it works as expected.) cc// @mattwiebe
    • 8:07 – “Each time I view my blog it gives me a slightly different picture”.
    • 9:10 – still trying to figure out why BG color isn’t changing – “I’m so confused”
    • 10:30 – Moved on without being able to figure out how to change BG color. :-(

    Step Five notes – Change your site title

    • 10:50 – Looking in Appearance -> Header for a place to change her site title (which really makes sense, we need to add a link there).
    • 11:10 – didn’t find it in Appearance -> Header, no she’s checking all up and down the left nav
    • 12:20 – Found it in the customizer
    • 12:30 – What the? She couldn’t add spaces? That is weird. There is a bug in one of the patches? cc// koopersmith can you spot any reason why this would be happening in one of the patches listed above?
    • 13:20 – “I’m very confused – I don’t know how I would insert spaces”
    • 14:22 – gave up without being able to add spaces…

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    She found how to upload the image – not going to worry about anything after that for now, since media manager is being revamped. We’ll do plenty of user testing on the new media manager once it’s complete.

    Recap

    • Bug in Iris prevented her from changing her background color
    • Can we please NOT have rotating header images in twenty twelve? None of the users we’ve tested have really comprehended what’s going on there.
    • We really should add a link on “Appearance -> Header” to “Settings -> General” to change their site title or tagline.
    • There was a bug in the customizer when she tried to update her site title which wouldn’t allow her to add spaces.

    I’d like to get the color picker bug fixed before I run another user through these changes, but overall, I’d say these patches have already really improved things for first-time users. Thoughts?

     
    • Amy Hendrix (sabreuse) 4:58 pm on August 27, 2012 Permalink | Log in to Reply

      I’m really excited about some of the improvements!

      (And Twenty Twelve doesn’t ship with any header image at all — if anything, we’ll be documenting that yes, you can add a header, and you don’t need to go to some scary code-snippet back alley to do so.)

    • Isaac Keyet 6:06 pm on August 27, 2012 Permalink | Log in to Reply

      Honestly it seemed like the main problem she had with changing colors was not the bug but that type of color picker altogether – a round color picker with the slider setting the black key (like the default apple picker) would probably be easier to comprehend for beginners. e.g.: http://cl.ly/image/061K1E1k2S2u

      • etoxin 7:42 am on August 28, 2012 Permalink | Log in to Reply

        It looks like the actual picker (in the square) is far too transparent. Maybe a small subtle animation on click could solve this or a darker colour.

        • etoxin 7:50 am on August 28, 2012 Permalink | Log in to Reply

          At 8:25 she gets the colour picker to work. Maybe simplify it to colour swatches and offer an advanced colour picker.

    • toscho 6:42 pm on August 27, 2012 Permalink | Log in to Reply

      What a pleasant voice. :)

      It should be possible to change the site title in Appearance/Header without going to settings.

      And I think the pointer for media uploads is positioned a little bit too obtrusive. The Upload/Insert text is clear enough.

      • lessbloat 6:47 pm on August 27, 2012 Permalink | Log in to Reply

        The fact that she saw that is a bug. That pointer shouldn’t even be showing unless they click the “Add images/media” link under the right column “Learn how to” section. I’ll look into suppressing that pointer unless they click that link. This implementation is just a stop-gap until we figure out how we want to connect the two related pointers. I didn’t want that to stop me from starting to test users, so I made a quick work-around (which unfortunately was buggy). ;-)

    • Peter Chester 6:49 pm on August 27, 2012 Permalink | Log in to Reply

      I get so happy reading this! Rock on!!!

    • John Blackbourn (johnbillion) 7:07 pm on August 27, 2012 Permalink | Log in to Reply

      Where did the “Install a new theme” pointer that popped up on step 5 come from?

      • lessbloat 7:12 pm on August 27, 2012 Permalink | Log in to Reply

        Same bug as @toscho mentioned. I had forgot a “!” in the pointer code. This one should have only been triggered when the user clicked the “Change your theme” link in the right column of the welcome screen. Instead of only showing those pointers when the user clicked those links, I was showing the pointers all of the time, unless the user clicked those links… ;-) Ah… what fun a missing “!” can cause.

        Anyhoo. This is fixed now on my testing install, and should be good to go for the next user test.

      • Konstantin Kovshenin 4:26 am on August 28, 2012 Permalink | Log in to Reply

        Why was she taken to Appearante – Themes in the first place? She was on her Dashboard, she clicked Customize, she saved and closed, and should be returned to where she initially was, her Dashboard. I pronounced the same “whoops” she did when watching the video :)

        @koopersmith, I think we fixed “redirecting to where we came from” in the Customizer in 3.4? Did it break again or does it work for front-end only?

        • Daryl Koopersmith 5:20 am on August 28, 2012 Permalink | Log in to Reply

          I think we fixed “redirecting to where we came from” in the Customizer in 3.4? Did it break again or does it work for front-end only?

          Neither — default behavior is to redirect to the manage themes page, and it doesn’t appear that these patches are using either of the methods that will cause the “return” action to redirect to another page.

          We have two different solutions to do so: one is a query arg (see the admin bar for an example) and the other is customize-loader.js, which uses HTML5 history to maintain page state (used in the theme browser).

        • lessbloat 9:00 am on August 28, 2012 Permalink | Log in to Reply

          Good catch. Ya, it would be great if clicking close would take her back to the dashboard, instead of themes.php.

    • Diana 11:01 pm on September 9, 2012 Permalink | Log in to Reply

      Greate material! I would like to reinforce some aspects :) :
      This color picker is really confusing for most people, the tinyMCE text color is way more helpful providing some colors right on click .
      In Customizing panel, the itens should be separated by area such Header, Content, Background etc , though this is more a theme author approach?!
      Title and Description should stay within Header options, (not link to Settings).
      The “Insert into post” button should be blue.
      Most people access Dashboard instead using the left menu itens, then I think this panel should display icons/links for those panels with some short description. (People are afraid of going somewhere they can’t go back :D)

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel