THX38 0.7.1 User Testing

I put THX38 0.7.1 through some testing. Here are some clips showing the main points of interest. Most are around a minute or so in length, some are shorter.

Accidentally clicks into the customizer and gets disoriented, first click on the “Customize” button doesn’t work (not sure why), thinks the magnifying glass hover icon indicates search:

While the themes page is blazingly fast 🙂 the customizer is slow:

Suggests the “Delete Theme” button should be red all the time:

Overlooks “Customize” button on the active theme:

“Delete Theme” is a little hard to find for this user:

Wishes clicking on the large screenshot in the modal would show a full view:

Wants more screenshot detail:

“Delete Theme” process not immediately visible, but found fairly easily:

Theme description details are a little too technical, i.e. “post format-packed,” HTML, SEO:

“Delete Theme” is easy:

Keeps clicking the back button instead of the “x” in the modal and it disorients her a little:

Tries right-clicking to delete:

If you’d like to see the full videos, you can download them here: THX38 001THX38 002THX38 003THX38 004THX38 005.

What would make finding really good themes easier? (survey question responses)

  • Tags or categories for subject matter, colors, or any number of style ideas and needs
  • Marketing maybe? Make it free?
  • If I were given an option of more than 20 themes to choose from, I would like to be able to choose by category, color theme, or complexity.
  • I can’t think of anything to really beat browsing. I mean, you could categorize them and even list them according to common uses (like Church) but I think I’d have to keep looking to find the perfect one. If you categorized based on HTML5 and all that, I think you’d lose a chunk of your user base (although having more technical categories for advanced users isn’t a bad idea at all).
  • The ability to search by genre. e.g. if I was a hairdresser key in the term hairdresser and get back a range of themes that could work for that business model. Complete with a choice of images would be fantastic.

Key Observations

  • The magnifying glass/search icon was only mentioned by one user. The visibility icon could still be a good alternate because an eye symbol is often used for “view” online.
  • Themes load very fast! This makes it especially noticeable when the customizer loads slowly.
  • Viewing most themes on a brand new WordPress install is lackluster because there isn’t much data to customize per se.
  • Users had no trouble finding “Preview,” “Activate,” and “Delete Theme” in the modal when hover buttons were removed. Consider taking the hover buttons out. I know this adds one click to activating a theme (pushback if you feel strongly about it).
  • Noteworthy user suggestion:  make the “Delete Theme” button red all the time
  • Noteworthy user suggestion:  make it so you can click the screenshots in the modal to enlarge them (carousel style?) or make them zoomable

All in all, the plugin is in really good shape. It’s fast! The “delete theme” link doesn’t get triggered accidentally but is still relatively easy to find. All of the bugs found via user testing were fixed in plugin version 0.8. If I could vote for two updates for v1, I would recommend: remove preview and activate hover buttons, change hover icon to a “view” symbol.

#thx38, #user-testing

THX (Themes) Update

The past week we’ve been committing significant updates to the plugin.

Screen Shot 2013-10-12 at 3.19.55 PM

Here’s a list of the changes that went in for version 0.7 (major props to @shaunandrews):

  • A lot of work went into rethinking how modal-overlay works and looks for showing the expanded theme view.
  • Adds keyboard navigation (with arrow keys) to quickly browse through themes while on details/modal view.
  • Significant JavaScript refactoring and cleanup.
  • Adds “delete” theme functionality.
  • Implements theme updates notices on theme grid, and update info on modal view.
  • Adds a theme count to the section title that updates immediately with search.
  • Several style improvements: theme blocks on grid view, add new theme, hover indicators, active theme more prominent, screenshot gallery with thumbnails, responsive styles, and a bunch more.

The keyboard navigation is becoming a pretty nice way to quickly flip through themes.

Screen Shot 2013-10-12 at 3.20.05 PM

What’s left?

Functionality wise it’s getting there, but we still have to keep polishing. (I’m being told modals are broken in Firefox.) We are a small team, so any reviews and suggestions are welcome — code or otherwise. Next up for JS rendering is looking into scaling issues (when, say, you have 200+ themes).

#core-plugins, #thx38

THX38 Update

We’ve been busy building and improving the plugin. We are approaching the point of a full feature prototype, which will allow us to start testing smaller incremental changes.

I committed the first pass at the browse themes screen based on the mockups we’ve discussed before:

Screen Shot 2013-09-24 at 4.30.13 PM

By the way, we are running a bit short on development resources, so if you are proficient in PHP and/or JS and want to contribute to this project, please, either chime in to one of our weekly chats or ping me directly. We are using Backbone to build all the theme views. We started a short document with the current tasks.

Screen Shot 2013-09-24 at 4.29.43 PM

We have two tasks going on: make theme switching work with the current prototype, and finish implementing theme fetching to the install-themes screen.

#core-plugins, #thx38

Update on THX38

Things have been in motion for the themes project. I’ve been working on making the plugin prototype ready for some initial testing. It’s looking like this:

@shaunandrews ran a test with it, and here’s the video.

Compared to how bad last test went, it’s pretty cool to see the dramatic effect the “add new theme” block had. (Also helped by the removal of the tabbed interface and extra information.) The user grasped immediately that she was seeing some themes that were already available to her blog, and how she could add more. Fairly straightforward, which is all this test was about.

Then, of course, as soon as she got to the filters page everything went down again, but we already expected that — still good to have one more test showing the same fundamental problem there.


On last Tuesday’s meeting we discussed the different mockups that were shared so far, apart from the plugin ones.

  • Shaun:
  • Emil:
  • Paal Joachim:×704.jpg

It seems we have mostly two directions to try for themes.php. The one the plugin is building (simplify the screens, remove the tabs, but keep a distinction between “your themes” screen and “installing new ones”), and one that merges installing new themes with your currently installed themes on the same screen. As a quick analogy, themes.php as your apps folder, or themes.php as an app store. Another possible outcome is that this screen could render a specific experience for new users alone.

Shaun is polishing his prototype so we can test it. We argued that it would be hard to know which of these two was the better one, since we would also need to test with people that already have a bunch of themes installed and don’t care much about the discovering new ones part. But having some insight is still good.

Finally, we talked briefly about multi-screenshot support. The plugin has a proof of concept implementation, showing the screenshots on the expanded view of a theme as a gallery. To test it, add a screenshot-2.png image to the root of a theme (up to five). Also relevant core ticket: #19816.

#3-8, #core-plugins, #thx38

Team THX38 had its first formal meeting last…

Team THX38 had its first formal meeting yesterday. See logs.

We talked about the things we see as more problematic, and spent some time discussing Filters/Search, as we identified that as an area with huge room for improvement. If you have read Dave’s post on design processes, this in area where we would benefit from exploring a bunch of quick directions that we can later test, so feel free to submit ideas.

Shaun also shared the usertest video he ran. Some interesting points from the video:

  • It was pretty clear that discovering new themes was not easy at all. The amount of time between arriving at themes.php and actually seeing new theme screenshots was huge. (The direction of the initial concepts I posted before address this issue by putting a new empty theme-block at the end of the grid of currently installed themes that says “Add new theme”, and gets rid of the tabbed interface. That would be something to test.)
  • “I was thinking I would have a screen with a bunch of themes to look at”. This is a nice way to put it. 🙂 She says this when clicking on “install themes” and seeing the entire screen covered with checkboxes. It seems we have this backwards with “search” being the first tab selected, you should be able to get into screenshot-browsing right away and maybe refine with searches and filters along the way. (Another thing we discussed were “categories” like photography, food, etc.)
  • Filters names don’t mean much visually — or rather, they mean different things to different people — so it’s a rough place to start with. The user later mentions she just wants to browse through images and see what jumps at her.
  • While browsing, it would be a nice experience if installing a theme was not disruptive — you click install and maybe see a progress bar on top of the screenshot, but you can still keep browsing and installing other themes.
  • Once you installed a theme, the message and screen you are sent to is a bit underwhelming.

Comparisons to existing theme/template showcases were brought up: themes, Squarespace, Tumblr, Weebly, to name a few. The plan is to report back the findings at next meeting, so those interested, please, grab one and research what you like and don’t like (with images!) for the discussion. Put your name in the comments so we know we have things covered.


I set up an initial plugin for the project that you can grab at Note: it currently makes themes.php unusable, it’s only meant for us to play with as we go. It setups your theme collection in Backbone.

Change of meeting time

We are switching the day of our chats to Tuesdays. So next gathering will be on August 27th at 18:00utc.

#core-plugins, #thx38

THX38 project and weekly meetings

Hello all. Team THX38 will be working on the Theme Experience project, spanning the theme admin screens as well as improving the .org directory. Check our initial overview for information on our goals and the pain points we want to address. We also have some visual concepts there to start the conversation.

Weekly office hours

We’ll be meeting in #wordpress-core-plugins on Monday, August 19 18:00 UTC. If this works out we’ll keep this time going forward.

Current Team

@matveb (lead), @samuelsidler, @melchoyce, @shaunandrews, @jacobdubail. The following people have also expressed interest in helping when time permits: @helen, @bpetty, @sabreuse

If you’re interested in contributing to the project, leave a comment here and let us know!

Next steps

For next week, Shaun will be working on running some tests around themes.php, and we’ll also be setting up a plugin soon so we can start playing with these ideas. See you around!

#3-8, #core-plugins, #thx38