Gutenberg Usability Testing at WordCamp Europe

This year, there will be a usability testing table at WordCamp Europe for Gutenberg, similar to the past one at WordCamp US. I’m looking for as many volunteers as I can find to help run tests at WCEU or process the tests remotely after the conference.

Get involved running tests

Tests will run for 30-minute blocks over the course of the WordCamp. To be involved, you must be attending WCEU. You can sign up here.

What do you need to do?

Facilitators will be in charge of the testing time. If you wish to just run tests and not co-ordinate, please sign up as a tester.

Facilitators should ideally have some usability testing experience, but if you’d like to try facilitating regardless, there will be information that will help you run sessions.

Facilitators will:

  • Welcome people interested in testing Gutenberg.
  • Set up testers with the Gutenberg test survey and the Gutenberg testing site, then start the provided screen recording app.
  • After the test is complete, save the test recording and reset everything.
  • Chat with testers about their experiences and their thoughts on Gutenberg, taking notes where possible.

If you are not attending WCEU, there is still a way you can help out: after the event, there will be a post created asking for volunteers to help process the tests.

Hope to see you there!

Call for Testing: Gutenberg 5.7.0 rc.1

This release includes

Features

Enhancement

You can find a full list to test against here.

The goal of Gutenberg is to simplify the creation of rich pages and posts in WordPress by replacing old custom HTML, CSS, and shortcodes with native Blocks. The Gutenberg plugin is currently the main focus for the testing group and Gutenberg is the new editing experience in WordPressDownload Gutenberg 5.7 rc 1

To help test, please download gutenberg.zip from the releases page or use the button above, install/activate the plugin, and try testing the new features and also if existing flows work.

You can also see other options for getting set up for testing in the handbook. When testing, use the latest stable release of WordPress (5.1) and the Gutenberg 5.6 plugin (download using the links above). All testing is welcome even if it’s just one or two items!

If you find a new bug, please file it in gutenberg on GitHub. Thank you! Please join us in #core-test on WordPress Slack any time if you have questions about testing!

#5.7, #call-for-testing#gutenberg

Call for Testing: Gutenberg 5.6 rc.1

This release includes:

You can find a full list to test against here.

The goal of Gutenberg is to simplify the creation of rich pages and posts in WordPress by replacing old custom HTML, CSS, and shortcodes with native Blocks. The Gutenberg plugin is currently the main focus for the testing group and Gutenberg is the new editing experience in WordPress. Download Gutenberg 5.6 rc 1

To help test, please download gutenberg.zip from the releases page or use the button above, install/activate the plugin, and try testing the new features and also if existing flows work.

You can also see other options for getting set up for testing in the handbook. When testing, use the latest stable release of WordPress (5.1) and the Gutenberg 5.6 plugin (download using the links above). All testing is welcome even if it’s just one or two items!

If you find a new bug, please file it in gutenberg on GitHub. Thank you! Please join us in #core-test on WordPress Slack any time if you have questions about testing!

#5.5, #call-for-testing#gutenberg

Call for Testing: Gutenberg 5.5 rc.1

This release includes:

  • Try using a Group block by adding some inner blocks to it and changing the Group block background color.
  • Add vertical alignment support to the Media & Text block.
  • Add the image fill option to the Media & Text block.
  • Lots of bug fixes and enhancements.

You can find a full list to test against here.

The goal of Gutenberg is to simplify the creation of rich pages and posts in WordPress by replacing old custom HTML, CSS, and shortcodes with native Blocks. The Gutenberg plugin is currently the main focus for the testing group and Gutenberg is the new editing experience in WordPress.

To help test, please download gutenberg.zip from the releases page or use the button above, install/activate the plugin, and try testing the new features and also if existing flows work.

You can also see other options for getting set up for testing in the handbook. When testing, use the latest stable release of WordPress (5.1) and the Gutenberg 5.5 plugin (download using the links above). All testing is welcome even if it’s just one or two items!

If you find a new bug, please file it in gutenberg on GitHub. Thank you! Please join us in #core-test on WordPress Slack any time if you have questions about testing!

#5.5, #call-for-testing#gutenberg

Would you like to help test Gutenberg?

We would love to have you! Gutenberg releases are happening every few weeks right now.

To help with general testing after opting in to the “Try Gutenberg” callout in the WordPress 4.9.8 dashboard, please comment here or say hello in #core-test on WordPress Slack.

To help with pre-release testing, or regression testing, first get a testing environment setup.

If you find a new bug, file it in gutenberg on GitHub.

Anyone can jump in to help with issues labeled Needs Testing and if you are a developer then issues labeled Needs Technical Feedback would be a great place to lend a hand. See the bug testing guidelines in the handbook for some tips.

If you want to help and aren’t sure where to start, just ask!

#editor

Help us Analyse the WCUS Gutenberg Usability Videos!

Huge thanks to everyone who participated in usability testing at WCUS. Our next step is to analyze the video footage – and we need your help!

The rest of this post tells you how you can help us with the analysis, even if you are new to usability testing and have never done this before.

If you have any questions comment below or better yet ping us in #core-flow on WordPress Slack.

How to analyze video footage:

  1. Go to the Volunteer Reviewers tab to fill out your name, Slack username and your WordPress.org username.
  2. Go to the Survey Results tab to assign yourself at least 5 videos to review. Enter your name in column B (or column D if column B is NOT empty) of the videos you are reviewing.
  3. Go to the session tab using the ID link listed in column A of the Survey Results tab.
  4. Review the tester’s survey results before watching the video.
  5. Follow the video link to watch the session and enter the following information in the highlighted cells:
  • Start time – when the tester started using Gutenberg.
  • End time – when the tester completed the task in Gutenberg.
  • Summarize the participant’s WordPress experience, if they mentioned it. (Some participants were asked or may have mentioned it during their session.)
  • Take notes about the user experience in the note section below. Include the timestamp from the video for the note, select note type (Bug, Pain point, or Insight) and your name with each note.
  1. Return to the Volunteer Reviewers tab to summarize the sessions in column E. Include any notable or/and recurring themes. If you feel a video should be reviewed by the Gutenberg team, specify this in column F.

For your reference, these are the two sample posts users were asked to replicate using Gutenberg: Test 1, Test 2.

Tips on Reviewing:

  • When taking notes on the session, please be sure to keep your feedback separate from the participant feedback. Notes on the individual session tabs should be from the participant. If you have suggestions or conclusions of your own, keep those on the reviewer’s tab, and be sure to flag any videos you think worthwhile for additional review.
  • Watch for and note any instances where user confuses functionality, i.e. looks for caption when should be attempting to add a block quote, etc. or moments of delight, was there anything they mentioned that they liked?
  • Note whether the participant succeed in completing the task or if they gave up. Try to note where or why they stopped.
  • In attempting to complete the task, does the participant pause for any length of time? Do they hover the mouse or appear to randomly click around? Does the participant get “lost”? Try to note what they were attempting to find or trying to do during this time.
  • Watch for visual cues from the participant like a furrowed brow of concentration, or audible cues like “hmmm…”? If these occur, note what the participant was trying to do at the time. If they solved it, are you able to note how?
  • If available, watch through the participant completing the Part 3 end survey. Do their responses provide any additional insight on pain points they experienced during the task? Sometimes participants will provide additional explanation of why they seemed to struggle. Is there anything they say but don’t type into the form?

Special thanks to @betsela, @fuyuko, @annaharrison, and @lynneux for developing and refining these plans.

Gutenberg usability testing meeting three

Last week we had a usability meeting. The meeting time wasn’t good for everyone, so let’s change the time this week.

When? Tuesday 31st October 19:00 UTC.
Where? wordpress.org Slack #core-flow: the testing channel.
Who should come? Anyone interested in helping test Gutenberg, all skill levels welcome.

Last week we talked about plans for testing. We will continue that conversation this week.

#gutenberg

Gutenberg usability testing meeting two

Some work has already been doing after our first meeting, which you can find out about here. It’s time to build on that work, let’s have another meeting.

When? Tuesday 24th October 18:00 UTC
Where? wordpress.org Slack #core-flow: the testing channel
Who should come? Anyone interested in helping test Gutenberg, all skill levels welcome.

#gutenberg

Writing my first Gutenberg Block

Inspired by a comment I read a week or so ago, I wrote my first Gutenblock. Here are my impressions of that process.

I initially planned on copying an existing block, but quickly found the example in the blocks directory README.

This is the ultimate result, you should open the example and the result to follow along how things changed.

Getting from one to the other was not too tricky, but did have a handful of surprises and confusing bits.

I started with changing the block name, which caused the first problem – I got an error that it needed to be namespaced. That was easily rectified.

Changing the title was easy enough, but I didn’t know what I should change the category to. I got an error when I left it blank, so I set it to common.

Next up, changing the attribute definition. Changing the type to ‘int’ was a lucky guess, there was nothing to suggest what the valid types are. source confused me for some time, as I didn’t really know what it needed to be. I figured I needed to store the value as a data attribute on the HTML that Star returned, so I wrote a basic version of the Star element, and tried to save it there. It… didn’t work. I tried several variations of this before noticing in the docs that source is optional. So, I tried deleting it, and it worked! That bit of magic made me smile, but I was a bit annoyed I spent so much time on a non-problem.

Writing the setStars() method was easy enough, even without really knowing what the props.setAttributes() call did.

Trying to add the Stars render to the children array gave me a weird error, complaining that the element needed an identifier. I tried adding an id attribute to the HTML that Stars rendered, but that didn’t help. Googling the error message showed me that I needed to set the key attribute, which was kind of weird, and I didn’t understand why it was necessary now, when the example didn’t need it. But it worked, so I moved on.

Adding the input element to children was also easy. I got the same error as before about needing an identifier, so I added a key attribute to that, too. I added all the extra attributes for fun, but realised when defining the min / max attributes that there was no nice way to validate the data being put into the stars attribute.

It was around this time that I discovered the Stars element wasn’t actually rendering, the problem was that I didn’t know how to put the text inside the div element. Did I need to append it? Was there a createTextNode() method I needed to call? As it turns out, it just magically accepted the string, but I only found that out by guessing. (Excuse the hacky math to add the “½”, I got a little carried away.)

One annoyance throughout was that every time I changed the attribute definition, or the output of Stars, Gutenberg threw a warning about the format changing. It seems like this warning would show whenever a plugin upgrade included a block change, though I didn’t test that.

And that’s about the sum of it – I found it to be a generally pleasant experience, it was nice that I could do this all without needing to compile from JSX, but I knew that option was there for if I wanted it. It’s still a bit of a raw experience, but it shows a lot of promise for creating a delightful development experience.

Next steps for Gutenberg testing

There was an open Gutenberg testing chat held on Wednesday, you can find out more about this and catch the video and notes here. There were a few outcomes from this and some next steps.

Firstly, what has happened so far?

Organised testing for Gutenberg has been in the form of 2 unmoderated usability tests. You can find out more about those here. These were a freeform and a task based test.

The test so far worked whilst Gutenberg was in early stages. However, it was limited in user type and response. The goal now is to scale up testing and make sure there are usability tests with each user type defined and prioritized, as well as non-WordPress users.

What are we going to do now?

In the group chat it was decided, the next step was to define user types, goals for testing, and the tasks for testing with each of the various user types defined.

To begin this, let’s use a collaborative spreadsheet to gather ideas. The scope of what we’re going to add is:

  • User types: In this tab let’s braindump a list of user types. For a starting point, the ones outlined in Matt’s post here have been added. What more should we add? Then let’s define:
    • Goals: What are the main goals of this user type? What do they want to achieve with an online publishing process?
    • Notes: Do you have anything else you think needs noting about this user type?
  • Tasks: In this tab the focus is tasks that people will do with Gutenberg. What can be broken down into tasks and test?

Add your thoughts on the above to the existing spreadsheet over the course of this week. If it’s easier you can leave a comment on this post with your thoughts and it will get added to the spreadsheet. Do you have some past research you can link to help add to this?

Are you a member of an agency, University, or are you in contact with a user type that can help the project reach? If so, let us know in the comments below, saying what you are able to help with for testing outreach.

Once the week is up, we can have a meeting again in #core-flow in wordpress.org Slack (time to be announced). We will go over the types and look at how we can begin testing.

If there is anything else you think should be considering in this testing focus, please let add a comment.

Get involved!

If you’d like to be involved in helping test Gutenberg, please leave a comment below and you can be added to the list of amazing people helping test this project.

This post was composed with the help of @msdesign21.