Background (Site building study #1)

These are the results of a user research study investigating mental models related to building and customising a website. Results are split across five posts:

Background | Segments: Bloggers · Small Businesses · Site Builders | Conclusions

Results have been compiled from the sitebuilding research conducted at the end of December, and a report is ready. Make a cup of tea, it’s a long one! 🍵

Huge thanks to @jarahsames, @alexislloyd, @bengrace, @benrearick, @bph, @cathibosco, @chrisvanpatten, @designsimply, @evawong, @johngough, @joshuawold, @karmatosed, @lilibet, @lonelyvegan, @mapk, @melchoyce, @mkaz, @msdesign21, @nao, @paaljoachim, @pento, @thedezzie, @tmmbecker, @tobiasziegler, @xarisgn, and Melissa Vander Wilt for helping to make this happen. Research like this takes a village, and it was fantastic to have so many people jumping in to lead sessions, take notes, share insights, and sift through all the data. Thank you for all your hard work! 🌟

If you have any questions about these results or would like to conduct your own research, please drop into the #research channel in Slack and say hello.

With that said, let’s dive into the full report! There’s a lot of information to digest, so this will be split into five sections (see discussion), to be shared here over this week and next.

Executive summary

This report provides an analysis of WordPress users in three key segments as well as their emotional journeys and mental models related to building and customising a website. Findings were compiled via a series of interviews with bloggers, small businesses, and people who build sites for others.

Key findings

  • Technological change presents cognitive challenges.
  • Most users want to get a site up as quickly as possible so they can focus on their core interests.
  • The way WordPress’ theming system work doesn’t match the way people think about building a site.
  • People don’t have pre-defined visions of what a site should look like.

Recommendations

  • Split themes into “styles” and “templates”.
  • Provide different paths for power users and for beginners.
  • Be very intentional with how changes are rolled out.
  • Allow for building by experimentation and play.
  • Documentation and help should be baked into the product.
  • Adopt a more data-informed approach to the introduction of new features.
  • Avoid adding features that aren’t integral to the core experience.
  • Clearly communicate future product vision and roadmap.

More context and explanation of these findings and recommendations will be included in the final post of this series.

Background information

At the end of December, a group of intrepid designers, developers, and other curious sorts embarked on an exploratory research study with the intent of learning more about how people think about site-building

Identified goals

  • Understand how people currently build websites using either WordPress or other tools.
  • Learn more about the mental model users have about building a website.
  • Assess pain points and areas of friction when it comes to building and customising a website.
  • Identify terms used in relation to the visual structures of a website.

After establishing a research plan, the research team started by talking to stakeholders and members of the community to get a better understanding of the goals of Gutenberg’s phase 2 and how best to direct research efforts. From this, the group learnt that there was some confusion over the vision and overall project mission during phase 1, and that a challenge for phase 2 would be to build a product that would meet the needs of a wide range of different WordPress users. The overall consensus was that research efforts should focus primarily on talking to WordPress’ end users, who tend to be less technical and aren’t actively involved in the open-source community.

In total, the research team talked to seventeen people over seven days, loosely following this script and adapting as needed to the individual conversations. Researchers learnt about people’s goals, their experience building websites, and their struggles. The team also looked over users’ shoulders as they worked on their sites and observed their process.

A quick note about research methods: Generative research generally happens at the beginning of a project, to explore a problem through the lens of a user’s experience. Evaluative research happens once you have prototypes built, to find out if the design approach solves the problem effectively.

For this study, the research team was conducting exploratory, generative user research to learn about people’s experience building websites. This wasn’t usability testing or evaluative research, and the aim wasn’t to collect feedback or usability results from the current iteration of Gutenberg. Evaluative research will come later in the process—stay tuned!

How does research work in an open-source environment?

The process was as open to the community as possible whilst ensuring the highest level of participant privacy. In total, over twenty-five people were involved in interviewing, note-taking, and data-mining. Huge thank-yous to everyone who participated in the process! 🌟

Exploratory research can be challenging to plan because you’re not exactly sure what you’re looking for when you set out. On top of that, both the domain (building websites) and the market (literally everyone who makes websites) are vast. These complexities meant the research team has a big task ahead of them.

To gather high-quality data as quickly as possible, the team worked with a contract recruiter. Participants were sourced from an existing database of WordPress users that had been recruited for research about the Jetpack plugin, specifically looking for a mix of beginners and more advanced users. The aim was to get as wide a range of different needs and perspectives as possible represented, but time and budget restrictions meant the team wasn’t able to meet all of these targets. The team spoke to seventeen people in total, all of whom spoke English as a first language and lived in North America.

With this in mind, it’s important to note that the results here reflect only a small portion of WordPress’ total userbase. This merits further research in order to deepen our understanding of our users. This research provides a baseline upon which future research efforts can build.

Want to look through the raw data yourself? Anonymised data is available in this Airtable.

If you’re interested in doing your own research study or in building on these results, please drop into the #research channel in Slack and say hello! The research group is working on a toolkit to help you get started and can also provide lots of support, feedback, and encouragement. 

Stay tuned! Next will be all about learning about the blogger segment.

#gutenberg, #research