3.3 Pre-Beta User Feedback (aka Testing Results)

Last week I did a bunch of testing, 20 participants (and had done some in Portland as well during WCPDX). I ran people through the new stuff, some screenshots, and watched them using their current sites as well. Did a combination of using my test site (or their site if they had beta tester plugin activated) on my laptop and going to participant homes. Mostly the former, for logistical reasons. I’d wanted to experiment with some long-distance testing using skype and screenshare as well, but ran out of time due to a bout with food poisoning, so am still planning to test out that technique with some of the people who volunteered (on this post on my blog about testing) in the future, maybe before beta2.

Flyout Menus

Universally liked, and no one was really bothered by the loss of ability to leave sections open once they got used to it (a few lamented loss of permanently open specific sections at first), which took an average of 3 screens. Most common complaint was the styling, that it wasn’t clear enough which section was in focus. I had talked about this with MT when they first went in, so will circle back with him to give it more delineation and contrast. I want to get rid of that fade for sure, it’s not a design element we use anywhere else, and it didn’t make anyone say, “ooh, that’s pretty,” which would be the one thing that might have swayed me (I like it when people think the UI is pretty). A few people (6) mentioned the animation without prompting and thought it was too slow. The rest were asked afterward if they thought the animation was too fast, too slow, or just right, and the responses were 0 for too fast, 12 for too slow, and 2 for just right.

Recommendations:

  • Ditch the animation and make flyouts appear instantly before beta 1.
  • Change styling during beta 1 to be higher contrast and more in line with current UI.

Uploader

The uploader changed a little during testing (early results -> try a fix -> more sessions, etc) so I don’t have numbers that are consistent here for opinions. Overall, everyone liked the new uploader, though some were confused by it at first. The initially really big dropbox was reduced in height to see if that would help, and it did. Even in the shorter dropbox, though, the small text centered in the middle made a few people wonder if there was a bug. “[] Scale images to max width 1024px or max height 1024px” was overlooked by most, and the few who noticed it wondered where the number came from. All guessed it was probably set by the theme, but weren’t sure where it would be used at that size.

Moving to a single ‘add media’ icon (from 4) confused people during testing because the icon in place was for video, not the multimedia icon from the Media Library as had been requested, so people wondered if the image upload was broken. When the intended icon + making ‘Upload/insert’ clickable was explained, participants were in favor of the change.

Gallery and post-upload image management still considered painful by 8 of 20 participants. 7 participants didn’t have a strong opinion as they don’t typically edit images/metadata after uploading or use galleries for multiple images in posts. 5 thought it was fine, though my guess is that they’re just so used to it they’ve stopped caring.

“From URL” upload was changed only for the last handful of participants, but all responded positively (only one even noticed the change).

Recommendations:

  • Adjust styling/layout of uploader popup (bigger text in dropbox).
  • Provide explanatory text near “[] Scale images to max width 1024px or max height 1024px” and add to help text.
  • Make ‘Upload/insert’ text clickable and replace video icon with camera+note icon from left menu.
  • On From URL tab, move “Insert media from another web site” to where it currently says “Add media file from URL” but make website one word per our convention.
  • On From URL tab, make “Images” be “Image” and make “Audio, Video, or Files” be “Audio, Video, or Other File” since they can only do one file at a time.

One Header to Rule Them All

The combination of the wp-admin header and the admin bar in the dashboard is something I was expecting more pushback on (I remember some people complaining the header font got too small in 3.2, and in this new approach, it’s not there at all), but people seemed to like it and think it was cleaner. No one mentioned the gain in vertical real estate, but 17 mentioned it looking cleaner and appreciating the reduction of duplicated info/links. I tested a few different colors (used comps from UI group and Chelsea) and a few different orders/labels/items for the bar itself based on the discussion on the Trac ticket.

Size and placement: It’s fine!

Colors: In the default dark bar currently on trunk, the font needs to have more contrast. In the lighter comps, the fonts needed to be darker for same reason. Thought people might equate the lightest version with browser chrome, but no one did. People did not have a strong preference for the darker or lighter versions, and when forced to choose one, it was split evenly (except for 3 people who suggested making it an option in Writing Settings (where they seemed to think Profile Personal Options would be located, to a man (and one woman)). We can just choose whichever wins the core team + Matt vote (guessing it will be dark).

Content: Addition of W menu liked by 16, not cared about by 4 (“I doubt I’d ever use it.”). Links included were fine with all, adding the other footer links was seen as a good idea. About evenly split as to if About [version] and credits should be combined (proposed as tabs in one screen with a sketch) or should have it’s own link in the menu.

The change in behavior for title being used as link between front and back confused 4 people at first. 2 people figured it out on their own after clicking bak and forth a couple of times, 1 person asked for confirmation that it was intended behavior (and was cool with the answer), and 1 person thought that in the dashboard there should be a dropdown that says “View site” to be consistent with the dropdown to Dashboard on front end. Though I want the title to remain clickable to the other side, making that dropdown isn’t a bad idea, could also include shortlink to the homepage when shortlinks in use (wp.com has shortlinks here).

When on front end, tested variations for title dropdown that included a full wp-admin nav (left side in dash) and an abbreviated version with the links in current admin bar. There was no strong preference based on design, as most people said they mostly just wanted access to the screens they used most, which were variable.

The Updates circle being the brightest thing was seen as weird, most thought it should be on par with comments number, and several suggested we give it an icon like comments has. Worth considering, would make it prettier and more consistent.

Multisite variation. Tested with 6 people who currently use multisite, so were familiar with Network Admin, had others look at options also. Tested variations of where the Network Admin link and the list of sites should go. Tried all under name link, tried all under W (i.e. “your WP install” instead of WP resources), tried as one combined link (and tried both Network Admin and My Sites as labels), and tried as two separate links. Tried the top-level link on left and on right. (Thank you Chelsea for whipping out a ton of comps for this.) In general, this was too many choices for testing. In future should narrow it down to just 2 or 3 variations. That said, 3 people chose having sites under name (2 of these were .com users) but network admin as own link, 2 chose having each as a top level link, 1 chose having both under name link, 2 chose under W (1 of these was conflicted because she also liked the resources links), and 12 chose to have both under one top level link — 10 said under My Sites and 2 said Network. Placement between W and current site name got the most votes.

Search. No one minded it being missing on dash side.

Help. New layout underwent some construction and style updating during testing, so I don’t have numbers for voting etc. Current state is based on iterative adjustment during test period. The breakdown into shorter section with persistent links on right was universally liked. Several people suggested embedding images and/or video now that there felt like there was more room. Also tested placing help in left section (related to blog name) and right section (related to person name), 17 out of 20 voted for left section (but of these 3 said it would be tidier on right with a ? instead of word Help a la search/magnifying glass on front end)

Recommendations:

  • Choose light or dark (prob dark) color scheme and make font/icons lighter for better contrast.
  • Keep W menu, add the other footer links into it and ditch them from footer (Credits, Freedoms, Documentation is already there, Feedback).
  • Add My Sites top-level between W and current site name, make Network Admin the first site and give it a different background shade, then list all sites in network.
  • In dash, add View Site and Shortlink submenu items to site title.
  • Need to choose between full list of nav link or snack menu under site name in front end. Choose one to get in before beta 1, adjust to optimize ux before beta 2 if we go with snack menu.
  • Get icons for Add New (+), Updates (up arrow?), Help (?) to provide consistency and make more scannable. Could alternately reduce to icons (+numbers) and show labels on hover if there are concerns about space (must ensure keyboard accessibility of all dropdowns/flouts, btw).

New Feature Pointers

Universally liked. Also tested yellow alert version from John O’Nolan; 9 said it stood out more but of those 9, 7 didn’t think it was very friendly because they associated the color with a dashboard alert.

Recommendation:

  • Do beta 1 as is and improve styling before beta 2. The new Pandora has some snazzy new feature pointer styling, let’s take a cue from that but give it our own palette etc.

Post-update Version Screen

Universally liked. Requests for links to more info — and including video walkthroughs when applicable — common among participants.

Recommendation:

  • Start putting in more structured info and linking up to Codex. Make prettier.

New Install Welcome Box

Tested this with sketches for concept, then asked participants to make list of what they would want to see in this box. Concept universally liked, though several thought it might be more useful if paired with maintenance mode during initial setup with a ‘go live now’ button or something. Preferred advice/links varied based on experience level, with most including some settings (tagline, display name, discussion options most common), deleting default post/page/comment, editing profile, and creation of one of most things (a text widget, a post with an image in it, an about page).

Recommendation:

  • Get the box in there with some default links, take a poll during beta 1 on wordpress.org blog of what things people would find most useful for 1st time installs, and launch final version in beta 2.