Make WordPress UI

Tagged: user-testing Toggle Comment Threads | Keyboard Shortcuts

  • lessbloat 8:40 pm on October 2, 2012 Permalink
    Tags: , user-testing   

    Decided to run another user through this afternoon:

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems.

    Step Four notes – Change your background color

    3:22 – She tried typing “blue” into the hex text field – didn’t work.
    3:26 – No issues figuring out the color selector. Yay!

    Step Five notes – Change your site title

    No problems.

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    7:55 – Our first successful test of the new media flow. Double yay!

    Recap

    Smashing success. Came in at 8:32 for all tasks. Quickest time yet I do believe…

     

    • Isaac Keyet 11:20 pm on October 2, 2012 Permalink | Log in to Reply

      re: step 4: probably way overkill but it’d be cool if the field did support the basic colors as specified in html and css. I.e.: aqua, black, blue, fuchsia, gray, grey, green, lime, maroon, navy, olive, purple, red, silver, teal, white, and yellow.

    • toscho 12:04 am on October 3, 2012 Permalink | Log in to Reply

      Or add a script that converts these words automatically to hex values when they are typed.

    • bradparbs 4:34 am on October 7, 2012 Permalink | Log in to Reply

      Based on this and the other videos, no one seems to know what QuickPress is. I threw a ticket up on Trac, #22119, suggesting we rename it. I think it might help.

  • lessbloat 5:37 pm on October 2, 2012 Permalink
    Tags: , user-testing   

    Ran another user through Discovery Cycle 1 this morning.

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems.

    Step Four notes – Change your background color

    • 3:36 – He had to play around a bit, but he figured out how to change the color to blue fairly quickly
    • 3:52 – He clicked the “default” button resetting the work he had done to choose a color.

    Step Five notes – Change your site title

    No problems.

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    • 7:20 – Cool, he clicks into new media workflow. But unfortunately he’s looking to paste a URL, which isn’t currently in the new media workflow just yet.
    • 9:05 – He discovers the “Upload/insert” link, and has no trouble adding the URL.

    Recap

    • Overall, I’m super happy with this test. He completed everything in 10 min which is a great time (compared to where we started off at the beginning of the 3.5 cycle).
    • Happy to see the first user try the new media flow, even if it didn’t work out for him. We’ll continue to to test new users as that section fills out.
    • 7:30 – When he entered a URL into the media uploader, it appeared to succeed for a brief second, you saw a file upload happening briefly and then it disappeared (just a heads up @koopersmith).
    • Do we need to do anything about the “default” button in the color picker? Would be a shame if someone took a bunch of time to find just the right color, then lost it with a click.
     

  • lessbloat 4:03 pm on September 3, 2012 Permalink
    Tags: , user-testing   

    I sent two additional users through discovery cycle 1 on Friday afternoon. The four initial tests can be found here.

    User #5 through these scenarios

    Here’s the video, and my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    Found it after a bit of trial and error

    Step Four notes – Change your background color

    5:20 – Yay! She was able to use the color picker.

    Step Five notes – Change your site title

    5:55 – No problems.

    Step Six notes – Add your first post

    6:42 – No problems. Used the “+ new” menu

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    9:40 – No problems – Another instance of a user literally copying and pasting the image into the visual editor.

    Recap

    Quickest user test so far. By far. Yay!!!!

    User #6 through these scenarios

    Here’s the video, and my notes:

    Step two notes – Log in

    • Takes 2 times to log in, but no problems.
    • She see’s the big IE is out of date notice.
    • 2:20 – “It’s a little distracting”, referring to dashboard in general.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    No problems. Clicked link in welcome screen.

    Step Four notes – Change your background color

    • 3:30 – hunting and pecking through left nav
    • 5:00 – tries using color picker (which semi-works in IE7/8?), she selects a color from the right hue slider, but it doesn’t update the color. :-(
    • 5:50 – comes back a second time, and only see’s hex text box – user gives up.

    Step Five notes – Change your site title

    • 6:30 – User ends up in Appearance -> header (this is about the 3rd user out of 6 that have tried it here). Note: we need to add a link on this page.
    • 7:40 – Still trying to find it in appearance -> header
    • 8:24 – Finds it in settings -> general

    Step Six notes – Add your first post

    9:30 – IE7/8? woes adding title in “Add new post”

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    No problems.

    Recap

    • So we’ve still got problems with the color picker. I think it’s slightly unfair to judge this too critically since the user is using IE7, or 8?
    • I’m going to work up a quick patch to add a link to edit the users site title from the appearance -> header page.
     

    • toscho 11:10 am on September 4, 2012 Permalink | Log in to Reply

      I think I will add the ability to edit the blog title on the header appearance page to my themes from now on. These videos are real eye-openers. :)

    • snaushads 7:15 pm on September 5, 2012 Permalink | Log in to Reply

      Im new to Make WordPress, though ive been using wordpress for 2 yrs (not as a developer), I got to know how the UI is being worked upon by Matts “State of the Word 2012″.
      And yes the video made me realize that its not the way “i” think users use WordPress.
      How can i contribute as a non-programmer ?

      I understand the correct url would be: http://make.wordpress.org/ui/tag/discovery-cycle-1/

      Thanks.

  • lessbloat 4:48 pm on August 27, 2012 Permalink
    Tags: , user-testing   

    Now that we’ve got a couple of rough patches (welcome screen, color picker, “view” my site) it’s time go back and test a few users in our first discovery cycle with these patches applied, and see if we’ve made any progress.

    Here’s the video, and here are my notes:

    Step two notes – Log in

    No problems.

    Step two notes – Explain what you see

    No problems.

    Step Three notes – Preview your blog

    • 4:00 – She’s looking for a link
    • 4:14 – “Oh here it is” – She clicks “Visit my blog” in toolbar (Win for “view” my site patch).

    Step Four notes – Change your background color

    • 5:45 – When asked to change the background color, she immediately (instinctively) clicked the big blue “customize my site” button (woot!).
    • 5:55 – she easily found the BG color option in the customizer.
    • 6:00 – Oh noooes, she found a bug in Iris (so painful to watch…)! clicking the right column of Iris does’t change the color in the text-field (After a bit of testing, it looks like if you click the right column first, before clicking the left gradient box, nothing happens. If you click, if you click the left box first, then the right column, it works as expected.) cc// @mattwiebe
    • 8:07 – “Each time I view my blog it gives me a slightly different picture”.
    • 9:10 – still trying to figure out why BG color isn’t changing – “I’m so confused”
    • 10:30 – Moved on without being able to figure out how to change BG color. :-(

    Step Five notes – Change your site title

    • 10:50 – Looking in Appearance -> Header for a place to change her site title (which really makes sense, we need to add a link there).
    • 11:10 – didn’t find it in Appearance -> Header, no she’s checking all up and down the left nav
    • 12:20 – Found it in the customizer
    • 12:30 – What the? She couldn’t add spaces? That is weird. There is a bug in one of the patches? cc// koopersmith can you spot any reason why this would be happening in one of the patches listed above?
    • 13:20 – “I’m very confused – I don’t know how I would insert spaces”
    • 14:22 – gave up without being able to add spaces…

    Step Six notes – Add your first post

    No problems.

    Step Seven notes – Preview your new post

    No problems.

    Step Eight notes – Publish an image

    She found how to upload the image – not going to worry about anything after that for now, since media manager is being revamped. We’ll do plenty of user testing on the new media manager once it’s complete.

    Recap

    • Bug in Iris prevented her from changing her background color
    • Can we please NOT have rotating header images in twenty twelve? None of the users we’ve tested have really comprehended what’s going on there.
    • We really should add a link on “Appearance -> Header” to “Settings -> General” to change their site title or tagline.
    • There was a bug in the customizer when she tried to update her site title which wouldn’t allow her to add spaces.

    I’d like to get the color picker bug fixed before I run another user through these changes, but overall, I’d say these patches have already really improved things for first-time users. Thoughts?

     

    • Amy Hendrix (sabreuse) 4:58 pm on August 27, 2012 Permalink | Log in to Reply

      I’m really excited about some of the improvements!

      (And Twenty Twelve doesn’t ship with any header image at all — if anything, we’ll be documenting that yes, you can add a header, and you don’t need to go to some scary code-snippet back alley to do so.)

    • Isaac Keyet 6:06 pm on August 27, 2012 Permalink | Log in to Reply

      Honestly it seemed like the main problem she had with changing colors was not the bug but that type of color picker altogether – a round color picker with the slider setting the black key (like the default apple picker) would probably be easier to comprehend for beginners. e.g.: http://cl.ly/image/061K1E1k2S2u

      • etoxin 7:42 am on August 28, 2012 Permalink | Log in to Reply

        It looks like the actual picker (in the square) is far too transparent. Maybe a small subtle animation on click could solve this or a darker colour.

        • etoxin 7:50 am on August 28, 2012 Permalink | Log in to Reply

          At 8:25 she gets the colour picker to work. Maybe simplify it to colour swatches and offer an advanced colour picker.

    • toscho 6:42 pm on August 27, 2012 Permalink | Log in to Reply

      What a pleasant voice. :)

      It should be possible to change the site title in Appearance/Header without going to settings.

      And I think the pointer for media uploads is positioned a little bit too obtrusive. The Upload/Insert text is clear enough.

      • lessbloat 6:47 pm on August 27, 2012 Permalink | Log in to Reply

        The fact that she saw that is a bug. That pointer shouldn’t even be showing unless they click the “Add images/media” link under the right column “Learn how to” section. I’ll look into suppressing that pointer unless they click that link. This implementation is just a stop-gap until we figure out how we want to connect the two related pointers. I didn’t want that to stop me from starting to test users, so I made a quick work-around (which unfortunately was buggy). ;-)

    • Peter Chester 6:49 pm on August 27, 2012 Permalink | Log in to Reply

      I get so happy reading this! Rock on!!!

    • John Blackbourn (johnbillion) 7:07 pm on August 27, 2012 Permalink | Log in to Reply

      Where did the “Install a new theme” pointer that popped up on step 5 come from?

      • lessbloat 7:12 pm on August 27, 2012 Permalink | Log in to Reply

        Same bug as @toscho mentioned. I had forgot a “!” in the pointer code. This one should have only been triggered when the user clicked the “Change your theme” link in the right column of the welcome screen. Instead of only showing those pointers when the user clicked those links, I was showing the pointers all of the time, unless the user clicked those links… ;-) Ah… what fun a missing “!” can cause.

        Anyhoo. This is fixed now on my testing install, and should be good to go for the next user test.

      • Konstantin Kovshenin 4:26 am on August 28, 2012 Permalink | Log in to Reply

        Why was she taken to Appearante – Themes in the first place? She was on her Dashboard, she clicked Customize, she saved and closed, and should be returned to where she initially was, her Dashboard. I pronounced the same “whoops” she did when watching the video :)

        @koopersmith, I think we fixed “redirecting to where we came from” in the Customizer in 3.4? Did it break again or does it work for front-end only?

        • Daryl Koopersmith 5:20 am on August 28, 2012 Permalink | Log in to Reply

          I think we fixed “redirecting to where we came from” in the Customizer in 3.4? Did it break again or does it work for front-end only?

          Neither — default behavior is to redirect to the manage themes page, and it doesn’t appear that these patches are using either of the methods that will cause the “return” action to redirect to another page.

          We have two different solutions to do so: one is a query arg (see the admin bar for an example) and the other is customize-loader.js, which uses HTML5 history to maintain page state (used in the theme browser).

        • lessbloat 9:00 am on August 28, 2012 Permalink | Log in to Reply

          Good catch. Ya, it would be great if clicking close would take her back to the dashboard, instead of themes.php.

    • Diana 11:01 pm on September 9, 2012 Permalink | Log in to Reply

      Greate material! I would like to reinforce some aspects :) :
      This color picker is really confusing for most people, the tinyMCE text color is way more helpful providing some colors right on click .
      In Customizing panel, the itens should be separated by area such Header, Content, Background etc , though this is more a theme author approach?!
      Title and Description should stay within Header options, (not link to Settings).
      The “Insert into post” button should be blue.
      Most people access Dashboard instead using the left menu itens, then I think this panel should display icons/links for those panels with some short description. (People are afraid of going somewhere they can’t go back :D )

  • lessbloat 5:49 pm on July 11, 2012 Permalink
    Tags: user-testing   

    In reviewing the first 6 user tests, one thing really stands out to me (it’s a glaring problem in each of the tests). Can you guess what it is?

    It might not be what you’d expect…

    (More …)

     

    • Brent Logan 6:23 pm on July 11, 2012 Permalink | Log in to Reply

      And good luck reorganizing the menu structure without upsetting everyone who is used the way it is… :-/

      • lessbloat 4:07 pm on July 12, 2012 Permalink | Log in to Reply

        Haha. Ya, I’m not positive that we’ll need to go there. :-)

    • Shane 8:44 pm on July 11, 2012 Permalink | Log in to Reply

      I wonder how much of the time spent searching was on high traffic vs low traffic activities. Having to search to figure out how to set my time zone might be acceptable. After all, how often do I really change it. Having to search to create a new piece of content, which I do regularly would be indeed a total fail.

      • lessbloat 4:12 pm on July 12, 2012 Permalink | Log in to Reply

        That’s a good point Shane. I agree, some actions should take precedent over others.

    • Andrea Rennick 12:28 pm on July 12, 2012 Permalink | Log in to Reply

      What if there was a teeny tiny help section that a user could search to find what they needed?

      If I’m in a program and can’t find a menu item, I head for the Help -> Search areas.

      • Ipstenu (Mika Epstein) 3:22 pm on July 12, 2012 Permalink | Log in to Reply

        Oh that would be a nice thing. Do enough people poke at Contextual Help to make that a good place? Maybe the tool bar could have a ‘search’ that flipped to backend when on admin pages?

        • Amy Hendrix (sabreuse) 9:01 pm on July 12, 2012 Permalink | Log in to Reply

          I don’t think I’ve ever seen a user notice the help tab until it’s pointed out to them. Visible search in the admin might be more visible…

          • JerrySarcastic 11:07 pm on July 14, 2012 Permalink | Log in to Reply

            I have to agree, the help tab is strangely invisible to most new (and even advanced) users I run in to. Perhaps finding a way to add the search tool within the help tab will get more eyeballs on it. Seems logical to have it there, since the functions can be so closely related.

      • lessbloat 4:15 pm on July 12, 2012 Permalink | Log in to Reply

        I like this idea a lot. :-) I’d love to give users the option of A) browsing, or B) quickly searching.

    • Amy Hendrix (sabreuse) 8:58 pm on July 12, 2012 Permalink | Log in to Reply

      Sadly, that’s exactly what I expected – because I end up hunting around for things myself an embarrassing amount of the time.

    • Luke 9:35 pm on July 12, 2012 Permalink | Log in to Reply

      I like the search idea a lot. I could almost see it becoming a bit like launching applications from spotlight in Mac OS. Start typing “post” and you could get immediate feedback about what section of WordPress are available for that search term. Spotlight is one of the fastest ways around Mac OS. A similar feature in the WordPress admin area could be quite accessible for new users and incredibly useful for power users.

    • Vitor Carvalho 10:58 pm on July 14, 2012 Permalink | Log in to Reply

      Why not simply implement something like Ubuntu HUD? A global ajax search bar for all the admin tasks.
      http://youtu.be/w_WW-DHqR3c

    • Cyndy Otty 10:30 am on July 17, 2012 Permalink | Log in to Reply

      Mika Epstein sent me over here after I posted this in the ideas section. All of which is probably still valid for 3.5, which makes me want to cry a bit.

      But yes, the reordering and/or moving menu has been an issue I’ve brought up for several versions. If it’s difficult for those of you with sight, imagine how maddening it is for those of us playing hunt-and-peck with a screen reader. (And the fly-out menu thing does not help!)

    • C0BALT 7:44 pm on July 26, 2012 Permalink | Log in to Reply

      I think a huge plus would be the ability to re-organize the post or page editing page, especially for people who have had wordpress setup for them.

      Think of it like this.
      Currently, people fill in the title, then start typing. This is obviously very simple, but falls apart when the person has finished typing and hits publish.

      There needs to be a “must fill in these things” option, and the ability to re-organize the order things get filled in… to match the training that you’ve already left with them.

      1: Select Category (this is most important so it shows up in the right place)
      2: Type or select Tags
      3: Type fill out custom fields / taxonomies
      4: upload image(s)
      5: Type Title
      6: modify URL title (optional)
      7: enter content
      etc. Plus removal of other fields on the page which are not used

      Click publish…
      You say this is already all there. I say, it’s not in the right order and it’s all scattered all over the page.
      I would want those 7 things listed in order from top to bottom and when you hit “test” or “publish” it tells you, everything is valid OR something needs to be filled out still, and that thing that isn’t filled out turns red.

      There is so much extraneous information (and un-ordered) on post / page pages it is extremely difficult for new people to wordpress to figure this out.

  • lessbloat 9:39 pm on July 10, 2012 Permalink
    Tags: , user-testing   

    Round 3 of the “WP as a CMS” cycle is ready for your viewing enjoyment. :-) Previous user tests for this discovery cycle can be found here and here.

    This one’s audio is a bit fuzzy at times. I changed up a few of the tasks to be a bit more clear on what I was wanting this guy to do. Have a look, and please highlight anything you spot (that could be fixed) in the comments.

    Experiencing technical difficulties with the video – Sorry – I’ll get it posted as soon as I get that sorted…

     

    • lessbloat 11:42 pm on July 10, 2012 Permalink | Log in to Reply

      Sorry, for whatever reason I can’t get the video to stream from anywhere… :-(

      Here’s the file: http://wpusertesting.com/videos/CMS-user-3.mp4

      Here are my notes:

      • 1:41 – He was confused as to where to find the “Front page display” setting
      • 3:00 – He found it and made the change – “Obviously what I had to do what go to ‘reading’, that made no sense at all to me”
      • 3:33 – “It’s a nice looking page – I’l admit, I like it – It’s just confusing getting to that point”
      • 4:00 – He was asked to disable comments on his site. First he went to “Edit Page”, and didn’t see anything. Next he went to “Comments” (which seems to make logical sense).
      • 5:38 – He couldn’t find his way back to settings, so he logged out, and logged back in – “This is the only way I can think to go back to settings”
      • 6:03 – Finally found settings menu in left nav
      • 6:10 – He browsed through each setting one by one on the “general settings” page
      • 6:16 – Clicked “discussion” link in main left nav
      • 6:54 – He assumed that his change to setting was changed without having to hit the “Save Changes” button (which granted I didn’t specifically instruct him to do, and is below the fold).
      • 7:08 – Had no idea where he might find the widgets – “Tools maybe… plugins… Media…) – just hunted and pecked through the left nav
      • 8:50 – Found widgets (by clicking on a link in the “Right Now” section of the dashboard ;-) ) – but is having trouble figuring out how to expand the widgets in the right side “Main Sidebar” section.
      • 9:42 – As he’s deleting the widgets one by one – “It seems like there should be a global way to delete all of these at the same time”
      • 12:00 – He went back to settings when I asked him to remove the header image

      Observations

      • More and more, I’m seeing these users think their settings are saved without actually hitting the save button. Should we test putting a save button at the top of the page as well?
      • Users need a home-base – a place they can go when they aren’t finding what they are looking for, and count on being able to find it.
      • When widgets have been dragged to a sidebar, the only way to see the widget details is to click the tiny arrow on the right side of the widget (You should probably be able to expand the widget by clicking anywhere)

  • lessbloat 8:49 pm on July 9, 2012 Permalink
    Tags: , user-testing   

    Here’s round 2 of the “WordPress as a CMS” user test. You can view round one here.

    As much as I love spending 2+ hours making notes on each user test video, I’m going to try a different approach with this one. As you watch this one, please make a note in the comments if you spot anything that could be fixed. I’ve found the “2:23 – Something happened here” format to work pretty well.

     

    • Adam W. Warner 9:03 pm on July 9, 2012 Permalink | Log in to Reply

      1:41 – the user wasn’t directed to “Save Changes” and no static page was seen on the forntend, but she didn’t realize that.

      • lessbloat 12:52 pm on July 10, 2012 Permalink | Log in to Reply

        The last guy did that as well. And neither one of them bothered with selecting a page from the “Front page” and “post page” drop downs. I’ll modify the script slightly for the last user test to be more specific.

    • Andrew Ozz 10:04 pm on July 9, 2012 Permalink | Log in to Reply

      Yes, the user didn’t save a single setting. We can improve that by adding AYS on page unload when any settings fields have changed (like on the Edit Post screen when post_content has changed). Considering that settings aren’t changed often that seems acceptable interruption. Eventually can have a “Disable confirmation” in Screen Options to turn it off.

      Another idea: have a step-by-step help tab that can be opened from a well visible link at the top of each Appearance and Settings screen. This would be shown for new users only and dismissed on per-screen basis similarly to the Welcome postbox. In that tab, have steps for 2-3 common tasks that can be performed on the screen (UI like in the existing help slide-down).

    • Andrew Ozz 10:26 pm on July 9, 2012 Permalink | Log in to Reply

      Also, perhaps changing the visual style in Available Widgets to be more like a list of items, not similar to the widgets that are in the sidebar would be an improvement? Showing some sort of a popup menu on click with “Add to [main sidebar]” would probably help new users too.

    • lessbloat 3:06 pm on July 10, 2012 Permalink | Log in to Reply

      My notes:

      • 1:42 – Clicked static front-page (but didn’t save setting)
      • 2:58 – Had no problem adding a page (through “+ New” drop down menu)
      • 3:41 – She never published the contact us page (man, I’ve got to be super explicit with those instructions)
      • 4:40 – “Im definitely finding this to be confusing” referring to not knowing where to go to remove widgets.
      • 4:50 – Referring to the dashboard, “There is a lot of text, and a lot for me to read
      • 5:40 – Starts looking through left nav – link by link – to find where she can edit widgets
      • 6:17 – Looks under appearance – clicks “menus” link (did not see widgets link just above it)
      • 6:59 – On the widgets screen, she thought the widgets in the “Avbailable Widgets” section were the widgets active on her site, and tried dragging the “search” widget under “Active Widgets” to the “Inactive Widgets” area (She did not notice the “Main Sidebar” full of widgets)
      • 7:25 – She repeated dragging from “Active” to “Inactive” for all of the widgets (thinking she was successful)
      • 8:29 – “Those don’t really want to stay as I’m dragging them, I don’t understand why” – she mentioned as she would drag a widget down, and drop it in the “Inactive Widgets” area without being over a drop zone (thus the widget would disappear, and she’d have to start over).
      • 8:50 – The widgets were formatted really weird as she dragged them into the “Inactive Widgets” box and got them to stay. They weren’t aligned right at all.
      • 9:52 – She then deleted all widgets in the “Inactive Widgets” area, and they went away, so she thought she was golden.
      • 11:00 – She still thinks “Active Widgets” are her main sidebar – tries clicking on the “Text” widget, but nothing happened.
      • 11:21 – “How do I actually open this widget though, it’s saying that it’s available”
      • 11:41 – She figured out how to drag it to the “Showcase Sidebar” (and not the “Main side” which I thought was interesting)
      • 14:40 – “It really does have everything to build a website – It’s awesome” (Good to hear ;-) )

      Thoughts:

      • So, first off, I need to be much specific in my instructions. :-) I’m going to do my best to get this last user to actually set the “Front page displays” setting properly.
      • The dashboard (as we know), put her into a bit of sensory overload.
      • Watching this user use the widgets screen provided some unique insights (starts at 6:40).
      • Since the “Active Widgets” were front and center, she assumed they were the ones currently activated on her site.
      • Drag and drop of widgets posed some momentary challenges
      • There were some brief formatting issues in the “Inactive Widgets” box at about 8:40.

  • lessbloat 4:32 am on July 5, 2012 Permalink
    Tags: , user-testing   

    Here’s the start of another discovery cycle (#2). This one is focused around WordPress as a CMS (using it to build a website vs. for a blog). I’ll need to make some edits to this script before I run any other users through it.

    You can download the video and follow along if you’d like (however, this one is super painful to watch – you’ve been warned!)

    (More …)

     

    • helenyhou 3:14 pm on July 5, 2012 Permalink | Log in to Reply

      Thoughts are… a mix of “ugh”, “sigh”, and “WHAT HAVE WE WROUGHT?!”

      Seems to me quite a bit of the above could be helped by some of the things we’ve discussed so far – easy access to immediately doing things from the dashboard, better sorting of Appearance vs. Options vs. Settings, perhaps more focus on/discoverability with the Customizer.

      There are also some existing tickets and discussion about making the static front page workflow and logic easier to understand overall, both on UI and dev fronts (there is a lot of confusion about is_home() vs. is_front_page() in WordPress development land). I’ll dig those up and probably put some under low hanging fruit, because I think we can tackle them sooner rather than later.

    • saltcod 12:07 am on July 12, 2012 Permalink | Log in to Reply

      1:56 to set a static page! Wow.

  • lessbloat 5:52 pm on June 27, 2012 Permalink
    Tags: , user-testing   

    It’s summary time.

    Recap

    We’ve run 3 users through the same set of scenarios (via usertesting.com). You can review the results here, here, and here.

    Disclaimer

    Now… What exactly are we trying to accomplish with all of this?

    It’s important to note that we have to think about these discoveries in the context of the typical WP user. But to be honest, I’m not entirely sure I know what the “typical WP user” looks like (but I hope to dig into that over the next couple of months ;-) ). Sure, I have assumptions, as I’m sure you do, but my assumptions aren’t backed by any sort of data.

    The important thing to keep in mind here is that these 3 people were paid to perform a list of tasks. While we can benefit from watching their interactions (and looking for patterns of complexity/confusion), we need to recognize that the flow they follow is prescribed, and likely different in some ways from an actual user.

    Overview of discoveries

    Here’s an unfiltered list of observations across all of the tests:

    Background

    • Had trouble figuring out how the color selector worked (x3)

    Dashboard

    • Unsure what “QuickPress” was
    • Unsure where “1 post”, “1 page” came from under “Right Now”
    • Had lots of trouble finding a link to view their site

    Media Upload

    • User clicked “Select files” when they actually had a URL to paste (and should have used the “From URL” tab). (x3)

    Post Add New

    • Lost all changes when she clicked a link taking her away from the new post page (after already making some changes)
    • Didn’t notice “Insert into post” button when adding an image in the media modal
    • Clicked “Preview” button, but didn’t realize a new tab had opened (causing confusion)
    • Doesn’t appear to be a clear path for users that just wish to post a photo

    Settings

    • Had trouble knowing where to go to change her site title (x3)

    Toolbar

    • User didn’t notice links in toolbar (x2)
    • User never noticed “+ new” dropdown (x2)
    • The blog dropdown menu is completely different when in the admin vs. on site (should it be?)

    Twenty Eleven

    • Confusion over why the header image was changing (x2)

    Next steps

    This is where I need your insight/input.

    1) First, is there anything that you noticed that’s not on this list?
    2) Next, which of these do you think we should attempt to address (are there any quick wins that would benefit all WP users)?
    3) If you had to prioritize your list, what would you tackle first?

    I’ll hold off for a day before I post my thoughts. Don’t be shy! ;-) I’d love to hear what you think.

     

    • Drew Jaynes 12:53 am on June 28, 2012 Permalink | Log in to Reply

      1) Something I think that is starkly missing is mention if the seeming importance of the Welcome Screen and the way in which these 3 users relied so heavily on it. The users weren’t dismissing the screen and using the Dashboard so much as returning to it over and over for guidance.

      2) I think one quick win would be to revisit the “Add Media” button. I know I’ve seen some discussion on trace of a labeled button or a larger icon. Another option might be something like an idea suggested by @kovshenin for dragging media directly into the editor. On the topic of the ‘site title’ issues, it seems like we’re moving more and more toward consolidating settings and previews into the Customizer. Perhaps some clearer verbiage on the Welcome Screen about what the Customizer is for (and some pointers once you get there) would be more helpful.

      3) Not to rehash too much, but I’d say 1) Add Media, 2) Clearer connection between Welcome Screen & Customizer.

    • Helen Hou-Sandi 10:15 pm on June 28, 2012 Permalink | Log in to Reply

      1. Maybe I missed it, but I didn’t see any mention of the featured image/post thumbnail functionality – there are quite a few tickets already about how weird and difficult that workflow is.

      2. More prominence for the Customizer and a consolidation of all of those items into the same admin menu rather than spread out between Appearance and Settings seems like a relatively quick win. Or even duplicating menu items. Would also *love* to find a better color picker – Koop and I have discussed this in the past.

      3. Besides the featured image thing, which I’ve talked about several times, I’ve been contemplating a reworking of the dashboard to become more useful – I think I called it “more mission control, less info center”. Full editor + post formats (would help with quick posting photos, much like the iOS app does) for Quick Press, better/prettier information about basic site/post metrics (including all other public post types), etc. This would also hopefully roll in the nice things the welcome panel does – a guide to getting things done for first-timers, as well as quickly enabling power users to get where they usually need to go. For instance, I could see items like “Edit the home page” for a site that has a static front page assigned.

    • Chelsea 12:08 am on June 29, 2012 Permalink | Log in to Reply

      1) I think the “Settings” issue can be expanded to an overall organization/delineation between “Settings” and “Appearance” (as @lessbloat mentioned in an earlier user test writeup). It might be worth thinking about doing user tests specific to figuring out which part of the admin users generally look for things like “Site Title” “Header” “Language” etc. Its something I see a lot of people struggle with when setting up a new WordPress site.

      2) I second @helenyhou on making the Customiser more prominent. It could also go a long way with giving insight on how to address the problem I mentioned in #1.

      3) I would prioritize rehashing the media flow/making it easier to create media-heavy content above everything else. Its what we get criticised for the most and I think its the biggest thing other CMSs have over WordPress.

    • lessbloat 4:29 am on June 29, 2012 Permalink | Log in to Reply

      1) As I alluded to in my disclaimer, the area that I’m most interested in focusing (at the moment) is in gathering more data around who’s using WordPress, and what they’re doing with it. For example, I’d like to figure out the percentage of people who download WP that:

      • use it for business vs. a personal site
      • use it for a blog/website/blog & website/other
      • consider themselves designers/developers/both/neither
      • are first time users/repeat users
      • are building a site themselves/or for someone else
      • consider themselves novice/intermediate/pros

      I’m also very interested in figuring out a way to create a benchmark of sorts and a way to chart our peaks and valleys within the app (moments of joy/pain) over time. If we can figure out a way to do that, we can then prioritize the elimination of the biggest valleys, and work on flows that get users to moments of joy quicker and more frequently.

      2) A couple of things:

      • Welcome screen – I think we all agree that a second pass may be beneficial here
      • Media insert – Giving the “Insert into post” button a class of button-primary (would make it stand out a bit better)
      • Toolbar links – I may be opening up a can of worms here (not knowing any back story on the current design), but I’d love to see A) some visual indication that each of the 4 drop downs are in fact a drop downs, without having to hover over them, and B) some ideas around ways to make the “+ new” drop down stand out a bit more (it’s a really handy resource, and 2 of the 3 users never noticed it).
      • Color selector – I’m with @helenyhou, I’d love to see this get some attention (paging @koopersmith ;-) )
      • Media upload – It might be nice to at least start brainstorming alternative flows here

      3) Honestly, these all sound exciting to me. I’ve tried to list them in order (based on impact/time) specifically with the goal of improving this discovery cycle. After we’ve got a few patches pulled together, I’ll apply them to my little testing environment, and rerun this scenario-set with 2 additional users.

      I’ll also be working towards getting my hands on some more data. I’ll post more about that to this P2 in the coming weeks.

    • Shane Pearlman 3:43 pm on June 29, 2012 Permalink | Log in to Reply

      For #1, I’m deeply fascinated with that question. While .com may have some success in beginning to build those analytics, I have no clue how to proceed on .org driven sites besides surveys. With such a vast ecosystem, how do you get strong data about user roles and behaviors. For example, I’m deeply curious how many users on average have admin level access (even if they don’t need it) vs people in editorial / authorship roles. It truly affects their experience on the back end.

      @lessbloat, how were you thinking of exploring?

      • lessbloat 4:23 pm on June 29, 2012 Permalink | Log in to Reply

        For the first part of #1, I’d love to start off by running a series of small (one-question) KissInsights polls after people click the download button on http://wordpress.org/download/. It would look something like this: http://cl.ly/0n0y1v3N3Y1m1L0D1X1S

        After we get some data, I’ll summarize the results on this P2.

        The second part (figuring out a way to measure peaks and valleys over time) is going to take some noodling. I’m all ears if you’ve got any ideas.

        • Shane Pearlman 6:43 pm on July 3, 2012 Permalink | Log in to Reply

          +1 for the kiss insight poll. That is a great idea.

          • lessbloat 7:34 pm on July 3, 2012 Permalink | Log in to Reply

            With that said, it’s still self-selecting. I have no idea at this point what percentage of users get their WP install from WP.org. I’d also venture to say that those users fall on the outskirts of WP’s 30 million users (on the more advanced side). But, it’s a start, and any data is better than no data IMO, as long as you acknowledge that it’s a bit tainted.

            Though @Jane did have a great idea about adding the UI Advisory signup form on the forums, maybe we could get a few KissInsignts tests in there?

  • lessbloat 6:01 pm on June 26, 2012 Permalink
    Tags: , user-testing   

    Round 3 of this “Discovery Cycle” is ready to preview. Check out round one & round two, if you’ve not already.

    Here’s the video for this round.

    (More …)

     

    • Mel Choyce 6:40 pm on June 26, 2012 Permalink | Log in to Reply

      It’s really interesting to see where everyone tripped up. Watching everyone try to user the color picker was pretty painful. Maybe something like this would be a little more intuitive? http://www.colorpicker.com/ As you change the hue on the right, it adjusts the color accordingly, while on the current WP color picker, the inside color box doesn’t change at all when you adjust the outer ring (which is pretty counter-intuitive).

      I also liked your suggestion in the first round of adding “View My Site” into the dashboard menu.

      These were great to watch, thanks for doing them!

      • lessbloat 4:44 pm on June 27, 2012 Permalink | Log in to Reply

        Thanks for taking the time to watch them. I appreciate the feedback. :-)

    • Shane Pearlman 6:55 pm on June 26, 2012 Permalink | Log in to Reply

      These are simply wonderful. Any chance in the future you might be willing to set it up so that core team might watch them live and discuss? We ran our last usability test for our plugin (http://wordpress.org/extend/plugins/the-events-calendar/) on join.me and invited all contributors to watch real time and chat on Skype. The discussion that sprung from all participating concurrently was highly valuable.

      • lessbloat 4:42 pm on June 27, 2012 Permalink | Log in to Reply

        Thanks Shane.

        That sounds fun. Usertesting.com doesn’t offer live streaming (that I’m aware of). I’m sure you could take one of the videos and rig it up to a live streaming service of some sort though.

← Older posts

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel