Tag Archives: core-flow

Maria 6:52 pm on December 15, 2017
Tags: core-flow   

Help us Analyse the WCUS Gutenberg Usability Videos!

Huge thanks to everyone who participated in usability testing at WCUS. Our next step is to analyze the video footage – and we need your help!

The rest of this post tells you how you can help us with the analysis, even if you are new to usability testing and have never done this before.

If you have any questions comment below or better yet pingPing The act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.” us in #core-flow on WordPress Slack.

How to analyze video footage:

  1. Go to the Volunteer Reviewers tab to fill out your name, SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. username and your WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ username.
  2. Go to the Survey Results tab to assign yourself at least 5 videos to review. Enter your name in column B (or column D if column B is NOT empty) of the videos you are reviewing.
  3. Go to the session tab using the ID link listed in column A of the Survey Results tab.
  4. Review the tester’s survey results before watching the video.
  5. Follow the video link to watch the session and enter the following information in the highlighted cells:
  • Start time – when the tester started using GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/.
  • End time – when the tester completed the task in Gutenberg.
  • Summarize the participant’s WordPress experience, if they mentioned it. (Some participants were asked or may have mentioned it during their session.)
  • Take notes about the user experience in the note section below. Include the timestamp from the video for the note, select note type (Bug, Pain pointPain point Pain points are “places where you know from research or analytics that users are currently getting hung up and have to ask questions, or are likely to abandon the site or app.” — Design for Real Life, or Insight) and your name with each note.
  1. Return to the Volunteer Reviewers tab to summarize the sessions in column E. Include any notable or/and recurring themes. If you feel a video should be reviewed by the Gutenberg team, specify this in column F.

For your reference, these are the two sample posts users were asked to replicate using Gutenberg: Test 1, Test 2.

Tips on Reviewing:

  • When taking notes on the session, please be sure to keep your feedback separate from the participant feedback. Notes on the individual session tabs should be from the participant. If you have suggestions or conclusions of your own, keep those on the reviewer’s tab, and be sure to flag any videos you think worthwhile for additional review.
  • Watch for and note any instances where user confuses functionality, i.e. looks for caption when should be attempting to add a blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. quote, etc. or moments of delight, was there anything they mentioned that they liked?
  • Note whether the participant succeed in completing the task or if they gave up. Try to note where or why they stopped.
  • In attempting to complete the task, does the participant pause for any length of time? Do they hover the mouse or appear to randomly click around? Does the participant get “lost”? Try to note what they were attempting to find or trying to do during this time.
  • Watch for visual cues from the participant like a furrowed brow of concentration, or audible cues like “hmmm…”? If these occur, note what the participant was trying to do at the time. If they solved it, are you able to note how?
  • If available, watch through the participant completing the Part 3 end survey. Do their responses provide any additional insight on pain points they experienced during the task? Sometimes participants will provide additional explanation of why they seemed to struggle. Is there anything they say but don’t type into the form?

Special thanks to @betsela, @fuyuko, @annaharrison, and @lynneux for developing and refining these plans.

Anna Harrison 9:40 pm on November 6, 2017
Tags: core-flow, , ,   

Gutenberg Usability Testing Plan – Feedback Needed

We are getting ready to roll out a new round of usability tests for GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ mid-November. In this next round, we will focus on testing writing flowFlow Flow is the path of screens and interactions taken to accomplish a task. It’s an experience vector. Flow is also a feeling. It’s being unselfconscious and in the zone. Flow is what happens when difficulties are removed and you are freed to pursue an activity without forming intentions. You just do it.

Flow is the actual user experience, in many ways. If you like, you can think of flow as a really comprehensive set of user stories. When you think about user flow, you’re thinking about exactly how a user will perform the tasks allowed by your product.Flow and Context
. We are also very keen to widen the net for participant feedback, including testing with participants who are not current WordPress users.

In order to make this happen we have drafted a usability testing plan – and we need your feedback and suggestions (in the comments below) before the next meeting in #core-flow on Thursday 9 Nov at 18:00 UTC. The next steps will be to test the user test on a small sample set (week of Nov 10th), refine at the #core-meeting on  Tuesday 14 Nov 17:00 UTC, and roll out to a wider audience starting from 15th Nov.

We will also have a usability testing section at WCUS, so if you are attending please drop by the and join in!

Proposed Usability Tests

To test the flow of writing, we propose to construct a three part usability test:

  1. General demographics:  including prior WordPress experience, age and device used. This information will help us to segment findings
  2. The main task: participants will be asked to re-create the post shown in an image. There will be three images to select from, mapping roughly to a beginner, intermediate and advanced level
  3. Follow up questions: a few questions about the experience of re-creating the post

Participants will be optionally invited to upload their screen recording, and answer a few questions about their video footage.

Testing Script

Question 1: Do you currently use WordPress?

  • Yes
  • No

Question 2: Would you describe yourself mostly as a…

  • Developer
  • Designer
  • Blogger
  • Business Owner
  • Other: ________

Question 3 (optional): How old are you?

  • Under 18
  • 18 – 30
  • 31 – 40
  • 41 – 50
  • 50 – 60
  • Over 60

Question 4: What device are you using?

  • Mobile phone
  • Tablet
  • Laptop
  • Desktop

Question 5: Let’s get set up!

Check that you have the following items ready as you will need them to complete the task

  • Open Gutenberg editor in a new browser window
  • Ensure that you have the Twenty Seventeen theme selected
  • Open the task image in a new window [ beginner | intermediate | advanced ]
  • Start a screen recording
  • Remember to talk out loud as you complete the task

Your task is to re-create the page that you see in the image using the Gutenberg editor. Remember to start your screen recorder, and talk out loud as you complete the task! When you are finished, continue on to answer a few questions about your editing experience.

Question 6:  Did the task take long or shorter than you expected?

  • It took longer than I expected
  • It took about the amount of time that I expected
  • It took less time than I expected

Question 7: Can you tell us why?

Question 8: Was the task easier or harder than you expected?

  • It was harder than I expected
  • It was about what I expected
  • It was easier than I expected

Question 9: Can you tell us why?

Question 10: Are you more or less likely to use the Gutenberg editor in the future?

  • I am not likely to use Gutenberg in the future
  • I am unsure
  • I am likely to use Gutenberg in the future

Optional section: screen recording analysis

It would help us out a lot if you could upload your screen recording and answer a few questions about your recording

Question 11:  Save your screen recording, and upload your file here…

Question 12: How long did it take to complete the task?

Question 13: Was the title added correctly?

  • Yes
  • No

Question 14: Was the quote added correctly?

  • Yes
  • No

Question 15: Was the image added correctly?

  • Yes
  • No

Question 16: Where were the main sources of friction?

Thank you so much for your help!

If you would like to be kept in the loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. with the progress of Gutenberg testing, please leave us your email below and we will add you to the Make.WordPress user testing mailing list

Test Setup

The test can be completed by a participant, or used as a run sheet for an observational research session.

In order to complete the test, participants will need to:

  1. Get their hands on a device (laptop, tablet, desktop or mobile device)
  2. Ensure that they know how to do a screen recording on the device
  3. Load up the user test
  4. Follow the instructions in the test
  5. Upload the screen recording to the cloud
  6. Optionally, code the results from the screen recording
  7. Optionally, write up a blog post and tell us what you found

Reporting usability test results

There are three ways in which you can report back your user test results:

  1. You can simply answer all the questions in the test instructions. Remember to upload your screen recording at the end
  2. You can optionally analyse your screen recording footage by answering the optional questions in the final section of the test instructions
  3. In addition, you are welcome to write up your test results in a blog post

Get Involved

Have an idea on how to improve the usability testing plan? Have your say in the comments below before the next meeting in #core-flow on Thursday 9 Nov at 18:00 UTC. Once we have collected all feedback we will post a link to the test script and open the call for user testing!

 

#gutenberg, #usability-testing

Tammie Lister 5:25 pm on October 30, 2017
Tags: core-flow,   

Gutenberg usability testing meeting three

Last week we had a usability meeting. The meeting time wasn’t good for everyone, so let’s change the time this week.

When? Tuesday 31st October 19:00 UTC.
Where? wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. #core-flow: the testing channel.
Who should come? Anyone interested in helping test GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/, all skill levels welcome.

Last week we talked about plans for testing. We will continue that conversation this week.

#gutenberg

Tammie Lister 11:32 am on October 12, 2017
Tags: core-flow,   

Gutenberg usability testing meeting two

Some work has already been doing after our first meeting, which you can find out about here. It’s time to build on that work, let’s have another meeting.

When? Tuesday 24th October 18:00 UTC
Where? wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. #core-flow: the testing channel
Who should come? Anyone interested in helping test GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/, all skill levels welcome.

#gutenberg

Tammie Lister 5:41 pm on September 8, 2017
Tags: core-flow   

Next steps for Gutenberg testing

There was an open GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ testing chat held on Wednesday, you can find out more about this and catch the video and notes here. There were a few outcomes from this and some next steps.

Firstly, what has happened so far?

Organised testing for Gutenberg has been in the form of 2 unmoderated usability tests. You can find out more about those here. These were a freeform and a task based test.

The test so far worked whilst Gutenberg was in early stages. However, it was limited in user type and response. The goal now is to scale up testing and make sure there are usability tests with each user type defined and prioritized, as well as non-WordPress users.

What are we going to do now?

In the group chat it was decided, the next step was to define user types, goals for testing, and the tasks for testing with each of the various user types defined.

To begin this, let’s use a collaborative spreadsheet to gather ideas. The scope of what we’re going to add is:

  • User types: In this tab let’s braindump a list of user types. For a starting point, the ones outlined in Matt’s post here have been added. What more should we add? Then let’s define:
    • Goals: What are the main goals of this user type? What do they want to achieve with an online publishing process?
    • Notes: Do you have anything else you think needs noting about this user type?
  • Tasks: In this tab the focus is tasks that people will do with Gutenberg. What can be broken down into tasks and test?

Add your thoughts on the above to the existing spreadsheet over the course of this week. If it’s easier you can leave a comment on this post with your thoughts and it will get added to the spreadsheet. Do you have some past research you can link to help add to this?

Are you a member of an agency, University, or are you in contact with a user type that can help the project reach? If so, let us know in the comments below, saying what you are able to help with for testing outreach.

Once the week is up, we can have a meeting again in #core-flow in wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. (time to be announced). We will go over the types and look at how we can begin testing.

If there is anything else you think should be considering in this testing focus, please let add a comment.

Get involved!

If you’d like to be involved in helping test Gutenberg, please leave a comment below and you can be added to the list of amazing people helping test this project.

This post was composed with the help of @msdesign21.

Tammie Lister 4:25 pm on September 4, 2017
Tags: core-flow   

Gutenberg testing planning group chat

As the GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ project develops, it’s now time to think about a testing plan. A way to get this tested on as many users as possible, on as many environments as possible.

As a first step, come and join the first Gutenberg testing planning group chat at 19:00 UTC on Wednesday. Join #core-flow to get the link to the group chat just before it starts – it will also be added as a comment here.

Why a chat to plan? Coming up with a scalable plan on how to progress testing is important. A few points for discussion:

  • What are our initial testing goals?
  • What format should the testing be in?
  • How do we reach outside WordPress to test with as many people as possible?
  • How do we scale testing?
  • How can we improve gathering feedback

… If there are any other points you think we should add please put them as a comment.

This chat will be recorded and a video put up after to ensure that everyone gets a chance to keep up. Notes will also be taken.

From this chat a plan will be formed that will be posted for iteration from feedback. This will then form the structure for testing Gutenberg.

+make.wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org//design +make.wordpress.org/coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.