User Testing, Round Three – Theme Test

Setup Setup

You have launched a new blog about a Major League Soccer team. You have written some content already, but are not ready to officially launch the site yet. The next step is to customize the look of the site so it reflects the team or Major League Soccer, in general.

Top ↑

Task Task

RSL Diary is a blog about Major League Soccer team, Real Salt Lake. Your task is to change the look and feel of the site however you choose for a soccer blog about a professional team. The team’s colors are dark red (claret), dark blue (cobalt) and gold — but don’t be afraid to go outside these guidelines if you have a more interesting idea.

This is what the site looked like to begin with.

Top ↑

First test user First test user

Download the full clip here

It takes a really long time to figure out how to perform the task. Finally finds the customize screen and is thwarted by a lack of theme options to do what he wants to do.

https://www.usertesting.com/videos/B8j6ugI1q8rkfFJNDYefEg/clips/477945

Eventually, he finds Themes page, but doesn’t use them…

https://www.usertesting.com/videos/B8j6ugI1q8rkfFJNDYefEg/clips/477950

About 15 minutes in he revisits the Themes page and tries a new theme…but then is more frustrated because the theme he chose (Twenty Thirteen) has even fewer color options.

https://www.usertesting.com/videos/B8j6ugI1q8rkfFJNDYefEg/clips/477963

This is what the site looked like when he was done.

Questions Questions

What was the most frustrating part of this task?
Not having many options to change items I would have wanted to change.

Did you feel confident in completing the task? Did you run into problems?
Yes I felt confident, but didn’t feel I was given adequate tool to make the changes that needed to be made.

Did you feel like you needed help at any time in completing the task? If so, where did you go for help?
I didn’t feel like I needed help I felt like there was something missing. Like the tools were not really available to use.

What was your favorite part of the task? Talk about some of your successes.
Being able to review the changes quickly and make the needed decisions @ that time. I was also able to easily scroll thru the options to change the blog.

Top ↑

Notes Notes

He never tries to add a new theme, even though he mentions it at the end of the (full) clip. Hypothesis: Adding themes is scary.

It takes him 3 minutes to even figure out how to do what the task is asking (change the look and feel of the site), and his first stop is Customize, but Customize isn’t the best place to change the look and feel of the site. Hypothesis: Our terminology/jargon is misleading.

Not once does he even consider using (or looking for help). In fact, in his responses, he says “I didn’t feel like I needed help”. Hypothesis: It doesn’t matter how visible or well-written help is (or isn’t) if people aren’t inclined to ask for help anyway.

Top ↑

Second test user Second test user

Download the full clip here.

This user states at the beginning that he likes soccer so SCORE! 🙂

Very quickly figures out where he needs to go to change the look and feel, even tests out the help tab, but, like the other user, defaults to just customizing the current theme rather than “changing the look and feel” – e.g. changing the theme, adding a new theme.

https://www.usertesting.com/videos/yAmg7C2S6B_NTy1YFjUgRg/clips/478095

He runs into the same problem – lack of theme support to change different colors.

https://www.usertesting.com/videos/yAmg7C2S6B_NTy1YFjUgRg/clips/478108

He does discuss the possibility of adding a theme, again, and what he might do if he had his own images, but he doesn’t try any of the other themes.

https://www.usertesting.com/videos/yAmg7C2S6B_NTy1YFjUgRg/clips/478126

This is what the site looked like when he was done.

Questions Questions

What was the most frustrating part of this task?
Could only add two of the three colours Couldnt change any other colours i wasnt really frustrated with this though, i thought it was straight forward and easy to do

Did you feel confident in completing the task? Did you run into problems?
Yes i saw ‘appearance’ straight away and knew to engage with that area I also found some help in the help section

Did you feel like you needed help at any time in completing the task? If so, where did you go for help?
no But I did find a help section nonetheless

What was your favorite part of the task? Talk about some of your successes.
Previewing the site was rewarding and good to check how your changes look

Top ↑

Notes Notes

Though he finds the themes, even goes to the add new theme page, he never tries adding one. Hypothesis: Adding themes is still scary? Or was it the time restriction? (They are supposed to complete the task(s) within 15 minutes.)

First stop is Customize. Last stop is Customize. Various stops in between are Customize. He never tries other themes and is frustrated by the lack of color controls. Hypothesis: It’s not clear (enough) that you may have more/different options with different themes.

Top ↑

Conclusions Conclusions

I’m not sure if these confirm my hypotheses or not. It’s hard to tell how much of the users being unwilling to try new themes is because of the time constraint or because of the UX. There’s definitely something going on there, though, with people being unwilling to check out new themes.

Is it worth it to add more themes to the test site with more options?

Should we consider adding things like “you may use images from the media library” or “you may try adding new themes” to the task, or would that be guiding them too much?