4.7 Beta 1 User Testing in the Customizer and with Twenty Seventeen

I ran four users through some tasks in the customizer with 4.7 beta 1 and Twenty Seventeen. The format was pretty free, but due to time considerations I gave them some level of direction at times. Arim, Kai, and Triet participated in the previous testing for customizer themes, Teresa was using WordPress for the first time.

Arim

Notes:

  • Bug: add menu items not closed when clicking in preview
  • Can’t see whether an external link in a menu actually works
  • No easy path to cropping a currently selected header image, but she found a way.
  • Doesn’t understand a lot of the theme feature filter options (she lists specifics!)
  • Header video (from YouTube) didn’t work for her at first, this was actually a ticket with a patch somewhere regarding broken selective refresh I think (it showed up after a full refresh).
  • Clicks the customizer close button a lot, before realizing that that’s not how you go back.
  • “How would I put something in the second column if I use the two column layout”
  • Bug: she hit an infinite loop in the customizer preview 🙁 Not sure why.
  • Previewing video on device preview is broken in Twenty Seventeen. Also, can we hide the video when entering device preview modes to reflect that it’s not displayed? That could probably be done with CSS.
  • Thought that the logo being “bigger” (relative to the size of the screen/video) on smaller screens was weird.
  • Expected more to change with custom colors in Twenty Seventeen. But thinks it’s more aesthetically pleasing the way it works.
  • After recording, she said she likes the idea of having a video header, and also being able to use a YouTube video was really helpful. She likes that there’s an option to use an .mp4 directly, but feels like she’d have access to a YouTube video more often.
  • Thinks the placeholder is actual text in the custom css textarea.
  • I knew that she knows what CSS is, so I had her try changing something. She wasn’t successful, but felt like she could do it if she cared enough and appreciated that there was a way to do it.

Kai

Notes:

  • Calling a menu “top” implies that it’s shown at the top of the screen and was confusing, since it’s at the bottom of the screen. This led to menu locations confusion.
  • Edit shortcuts: “that’s cool”. Wants them to just be there without clicking.
  • Weird that you can’t edit the titles of dummy content pages. This conflicts with the usefulness of being able to create new pages within the customizer – you have to create a new one, but then also have to figure out how to delete the old one.
  • Also tries static front page well before theme options to get to the front page section options.
  • Bug: something went wrong with save/publishing – but the content came back when she re-opened the customizer. It looks like the front page options got cleared.
  • Add new page under each option is confusing, thinks it’ll add another front page content section.
  • Menu locations are confusing, but “display location” helps a little bit.
  • Prefers edit shortcuts over finding options in the section/panel navigation.
  • Even on this very large screen, the feature filter has far too many options.
  • “They’re all like shopping themes”. She was browsing in the “portfolio” subject.
  • Didn’t see any major issues with the theme installer.
  • Has concerns about not being able to really see a demo of themes with content.

Triet

Notes:

  • Conceptually, icons and shift-click make sense, but the details get confusing. For widgets with a list of categories, it’s unclear that the individual categories are coming from the widget instead of being able to be edited directly. The difference between widgets and menus led to the confusion here, I think.
  • Can’t find static front page section, even when prompted with specific terminology. Granted, this was testing on a random test site with a bunch of imported content that may have added to the confusion.
  • Doesn’t know where to click to edit different content. Preview vs. controls areas are very unclear to him.
  • Was generally confused by the fact that not everything could be edited with the icons, leading to confusion about how to edit other things on the site.
  • The front page sections in Twenty Seventeen made sense once he was directed to them. Again, the content on the test site led to some confusion.
  • Found selecting text as the way to show/hide icons, as opposed to clicking. Wasn’t able to figure it out even with some hints.
  • After he stopped the recording he suggested having outlines on the site for what areas can be edited, with edit icons there and being able to edit in the preview instead of having to switch between the site and the controls “bar”.
  • Bug: Front page section placeholders are shown before changing the sections in Twenty Seventeen.

Teresa

(video is hopefully coming soon, there were some issues with her computer saving the recording)

Notes:

  • I started her from a clean install to test starter content.
  • Her computer/internet was really slow, so keep that in mind. (it was her computer, not WP)
  • She also hit an infinite reload.
  • Can’t find a way to remove content from a front page section in Twenty Seventeen.
  • Looks in static front page for front page sections, eventually find it in theme options but says it was confusing. We should reconsider the placement and naming of these.
  • No issues with browsing/installing themes.

Action Items

Some initial changes we need to make that I noted during the tests:

  • We need a better link for custom CSS help.
  • For CSS, we can add validation if someone pastes in a `<font>` or some sort of markup with a `style=` attribute, and provide a more useful user feedback message that whatever they pasted isn’t actually CSS.
  • The “randomize” options for custom headers shouldn’t show up unless there’s more than one image.
  • Menu item screen options are all enabled by default. They should all be disabled by default. This is a regression, possibly dating to 4.3.
  • Dummy content doesn’t show up in available nav menu items. That fees like a bug.
  • Based on these results, it would make more sense to put the front page section controls in the static front page section than in theme options in Twenty Seventeen.

#4-7, #custom, #customize, #theme, #theme-install, #theme-switcher, #twenty-seventeen