Accessibility usertest: Twenty Sixteen

The new default theme Twenty Sixteen is close to release, so we gave it to our accessibility testers to check it.

Testes who joined: Michelle DeYoung, Gabriela Nino de Rivera Torres, Jeffrey de Wit, Sirisha Gubba, Heather Migliorisi, Tobias Clemens Häcker, Kim van Iersel, Geof Collis, Shaun Everiss, and Rian Rietveld.

Some remarks where not related to Twenty Sixteen but related to WordPress core. Those remarks where not included into this report, like the comment form issues. We will open tickets on them later.

Summary is at the bottom of this post.

Test results

Jeffrey de Wit

Firefox and NVDA

  • Focusing the featured image makes NVDA go “blank”
  • It may just be me, but the author, category, and tag links in the post meta simply state just that. That is, “themedemos”. Or “Uncategorized”. Nothing else, no context. (Same happens with the widgets, but that’s probably a core issue :))
  • I like how the search button is hidden with screen reader text, but it should probably appear when focused.
  • When focusing links, it sometimes happens that the page scrolls and makes the link go behind the black border around the page. This usually happens when the link that is to be focused is not in sight when tab is pressed.

That last one seems to be specific to Firefox and IE Edge, I don’t notice it with Chrome because it seems to make an effort to center focused links.

It can be reproduced with the second post – there’s a link under the h6 where you can see this issue, and then from there it goes back up to the author link in the post meta, which is also “hidden”. I’ll have some videos ready shortly.

Firefox:

Chrome:

Michelle DeYoung

Windows 8.1 / Firefox 40.0.3 / Window-Eyes 9.2

Landmarks: Landmarks are voiced. The W3c validator does indicate that the additional role=”specificlandmarkname’ is not needed on the html5 elements (header, form, nav, main, aside, and footer). The questions – has screen reader and browser support grown enough to utilize HTML5 semantics, sans aria landmark roles, to convey the various regions on the screen? In my opinion, I feel aria landmark roles should still be used with the new HTML5 semantics, and I continue to add them to my markup until I hear otherwise. 🙂

Menu navigation: Users should be able to select ‘ESC’ to close a drop-down menu, and then when tabbing or arrowing again they are taken to the next main menu item. This will make it easier to navigate the menu without it being necessary to tab through many items to get to the needed item.

I would recommend making the asterisk on the form labels the same size that it is in the instructional text so it will be easier for low vision users to detect.

Input fields: The color contrast for the border or background colors could be increased. They are difficult to discern. This is part of the minimalistic flat design, however, the light gray can impact users with low vision and they can also look disabled. Note: The input fields look great when they get keyboard focus.

Gabriela Nino

I have taken a look to the front-end with VoiceOver. In overall it was easy to navigate and the elements were well recognized. Opening the sublevels of the main menu was simple too. I was able to access the links and buttons and I did not get stuck.
Something that caught my attention was that images like the post “this is a sticky post” are ignored by the screen reader. I step directly from thetitle of the post to post content.
The images of the Markup: Image alignment worked well with VoicOver.

Rian

Safari, keyboard only

Menu in responsive mode: If the menu is expanded, it is not clear when the menu button gets focus, for example if you want to close it again. The dropdown toggle arrows do not have a clear focus (only change of color, no dotted outline)

Is it a feature, that the menu repeats itself in the footer in responsive mode (I actually quite like that).

Normal menu: Only color change on hover. Active menu item is bold : ok

Page navigation: No indicator current page number for a screen reader. Color is not enough to indicate this is the current page number.

page navigation twenty sixteen

Sirisha Gubba

NVDA/FF

  1. Roles are used without aria label (more than one navigation is used)
  2. Search button is not visible on the page.
  3. Two search fields with role=search are there on the 404 page
  4. Screen reader user have to go through all submenu items in order to get to another tab item
  5. Also I see color contrast issues with the input fields which would be issue for low vision users.

Note Rian: except for the 5th item these are all extra’s but not required for WCAG 2 AA.

Heather Migliorisi

This theme tested pretty well!

Tested with VO and Chrome.

  1. the main nav with sub items could be easier to navigate. Currently, if you want to navigate to “Page B,” you have to tab through the entire main nav, sub items included. I like how Adobe’s Accessible Mega Menu tabs through the top level list items. It also provides the expanded/collapsed state to the VO user.
  2. Search field – VO reads “search for” twice
  3. Might be nice to be able to skip the list of keywords added to an article

Summary

The main opinion: This theme tested pretty well!
No drastic errors. Yay!

First: can someone reproduce Jeff’s error?

Missing context for sighted users (maybe also a usability issue): The author, category, and tag links in the post meta simply state just that. That is, “themedemos”. Or “Uncategorized”. Nothing else, no context. For screen reader usres there is extra text to give context.

Landmarks: A lot of the remarks where about landmarks. W3C and VAWE now mark elements with the same role as warnings (not errors). Like: Element nav does not need a role attribute role=nav.

Note Rian: We discussed this in the team chat and decided that the roles should stay for now, for backwards compatibility with old assistive technology.

Navigation: three the same remarks:

  • Users should be able to select ‘ESC’ to close a drop-down menu, and then when tabbing or arrowing again they are taken to the next main menu item.
  • Screen reader user have to go through all submenu items in order to get to another tab item
  • Currently, if you want to navigate to “Page B,” you have to tab through the entire main nav, sub items included.

Note Rian: this is not an accessibility error, but a keyboard user usability issue. Might be worth to find a solution for this.

Featured image: There was confusion about the featured image with the link being skipped. This is on purpose to prevent extr noise for screen readers by adding an aria-hidden=”true” to the link.

Input fields: The color contrast for the border or background colors could be increased.

Focus errors:

  • search button should show up on focus
  • inconsistent focus for responsive Menu toggle button
  • submenu toggle responsive menu should have an outline

Page navigation

  • no ARIA indicator current page number for a screen reader
  • color is not enough to indicate this is the current page number

 

 

#a11y-audit, #accessibility-usertest, #twenty-sixteen

Testing Twenty Sixteen

By now, I’m sure you’ve noticed Twenty Sixteen has been released on Github and the theme repository to spur development and testing. If you haven’t tested the theme for accessibility, now’s the time! I mentioned it in a recent chat, and a members from the community have already tested it, but the more the merrier!

What do I test?

Test the theme according to the accessibility-ready requirements. Since this is a default theme, you should test for the recommendations too. Also, it’s a good idea to look at the applicable WCAG 2.0 requirements (Link is not the official WCAG requirements, but a more readable version), and test the theme according to those, within reason. This way, the theme can be its best accessibility-wise.

How do I test?

The accessibility-ready requirements give some information on how to test. We also have a list of tools that will help when testing.

How do I report bugs, issues or make a enhancement request?

What information do you need to know when creating an issue?

Here’s a good format to follow when creating a bug report:

Title: Accessibility: ISSUE SUMMARY HERE

What I Did:

What I Expected:

What Actually Happened:

I’m not a developer, how can I help?

No problem! You don’t have to be a developer to help out. Test the theme on your favorite operating system with assistive technology. This way, we get a better idea of how the theme works with different configurations. Try to break the theme! Note anything that is confusing, and makes the theme harder to use on the front end.

#twenty-sixteen

Accessibility Usertest: Select2

In the last 2 weeks @lindsaymac and @helen requested if the Accessibility Team could take a look at Select2.  Select2 is a jQuery replacement for select boxes and gives you a customizable select box with support for searching, tagging, remote data sets, infinite scrolling, and many other highly used options.

But how accessible is it, and is it accessible enough to include into WordPress core? Some research has been done by Drupal, but there are still issues open on GitHub.

On the Select2 Github there are examples, styled with bootstrap. To insure we only test the select2 functionality, we made a series of test pages with the minified JQuery and select2.css and select2.js only.

The questions we asked our testers:

  • Can you select an option
  • Can you remove a selected option
  • Is it obvious to you how to select an option
  • If you use a screen reader: did you get sufficient feedback on what was happening
  • What problems did you have trying to select an option

The testers who joined: Michelle DeYoung, Gabriela Nino de Rivera Torres, Bram Duvigneau, Cyndy Otty, Philip Chalker, Geof Collis, Bart Simons and Andrea Fercia.

Test results

1. Basic dropdown

Example of choices Select2

Example of choices that don’t get read out, the entered value gets read out instead

Geof Collis: JAWS 14.5 IE 11.
Not accessible. I tried to open the box but just got dead air and I found the edit field down the page but was confused on what to do so typed in the letter a and arrowed down to get a list of states only to realize that if I arrowed up and down I got the list I expected in the drop down box.

Bart Simons: JAWS15 and IE11,  JAWS15 and Firefox, JAWS16 and IE11, JAWS16 and Firefox
JAWS says “combo box collapsed”.
I press enter to go to forms mode.
Since I am told that this is a combobox I try using arrow down to scroll through the list. This does nothing. I can’t go beyond Alaska. Also arrows up, left and right do nothing.
Then I try to navigate by typing the first letter. This does not work either.

The test suggestions suggest to use space bar. This is something I would never have tried by myself.
After pressing space bar, I am told that I am in an edit box. I type “new” and I hear “New Mexico”. Again there is no possibility to use down arrow to go to New York for instance.
Even New Mexico does not seem to be selected.

Besides, it is really not intuitive to require a space bar in a combo box.

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
The combo box does not respond to Control+Option+Space bar VoiceOver command to open the list of choices. It does respond to the space bar only.

When the list of choices it’s open, the first element getting the focus is the link to go back to the list of examples. The second element is the search field text and lastly the table with the choices. I’m using Control-Option+right arrow to navigate inside the combo box element.

The texts for Time zone are ignored by VoiceOver.

VoiceOver offers me to navigate the table with the choices using Control-Option-right/left arrow or up/down arrow. Both commands worked fine.

There seems to be something different with the choice Hawaii. There is an intermittent behaviour that requires me to use the Control-Option-right/down arrow command several times before moving to California. It seems like the focus gets stuck on Hawaii choice for some reason.

It is not possible to move inside the list of choices using only up/down arrows.

Using only the Enter key will open/close the combo box choices. In this case the first element getting the focus is the search field text. When making a search, the user does not get feedback if the state that she is looking for it is found or not on the list of choices.

I could not find how to select an option of the list of choices. Normally I’ll use Control+Option+Space bar command, but I don’t get a response. Using Control+Option+Enter does not work either. Using only the Enter key will close the list without selecting the current choice (the one with the focus on it). But if I search for the state and click on Enter, the list will close but the current choice will be selected.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only: Works fine.

FF/NVDA:
Forms mode –  The state name is announced. This is considered the label via the aria-labelledby.  The label should be the ‘Select a US state”.
The text input in the expanded list is not labelled.
Virtual mode – Can arrow through the items in the combobox, but all that is voiced by NVDA is “blank”.
Note: Can expand and collapse the list.

Philip Chalker:  iOS + VoiceOver.
It all reads fine on the screen

Cyndy Otty:  iOS + VoiceOver.
But VO is reading the list, but selection itself is definitely wonky.

 

2. Multiple select boxes

Geof Collis: JAWS 14.5 IE 11.
Not accessible. Entered the box but heard nothing.

Gabriela Nino de Rivera Torres: VoiceOver+Safari.

The combo box does not respond to Control+Option+Space bar VoiceOver command to open the list of choices. It does respond to the space bar only.

Using only the Enter key will open/close the combo box choices. In this case the first element getting the focus is the search field text. The second element getting the focus is the link to go back to the list of examples and lastly the table of choices.

VoiceOver offers me to navigate the table with the choices using Control-Option-right/left arrow or up/down arrow. Both commands worked fine.

I could not find how to select a choice directly from the list of choices. Control+Option+Space bar command does not work. Using only the Enter key will send me to the instructions page. But if I search for the state, I’m able to select it using the Enter key after entering the word (although I don’t get any feedback to confirm me if the state I’m looking for was found or not).

I could not found how to remove a choice. Any of the common VoiceOver commands don’t work to get focus on the selected choices.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only:

  • No visual indicator that it is a select box and offers choices.
  • Yes, selections can be accessed and added.
  • Yes, can remove selections by using the ‘Backspace’. Selection will be removed a letter at a time.

FF/NVDA:
Combobox is missing a label. NVDA does announce that it is a combobox.
List is announced that it is expanded, but the as user arrows through the list the items are announced as “blank”.

If user randomly selects and item (they will not know what they selected), the items are added to the input field, but they are not announced.
Unable to access added items in the input to have them announced.  If I assume items exist in that form field, then I can select “Backspace” to remove them a letter at a time.
Note: Unable to type a selection name in the input field.

3. Placeholders

Geof Collis: JAWS 14.5 IE 11.
Semi accessible. I expected that I would hear the placeholder before going into forms mode but had to enter the field before I did.

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
No, VoiceOver did not announce the placeholder inside the first option. The combo box does not respond to Control+Option+Space bar VoiceOver command to open the list of choices. It does respond to the space bar and Enter key.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
FF/NVDA: Yes, Placeholder is announced. (No labels for input fields)

4. Disabled mode

Geof Collis: JAWS 14.5 IE 11.
Semi accessible. I expected a checkbox for the modes but realized to click on the button and it worked but I’m starting to wonder if I should be reading 2 drop down menus because that is what I am getting in all examples so far.

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
I was able to enable and disable the two select boxes. It does work with Control-Option-espace bar and with the Enter key.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only: Yes, Enable/Disable buttons work

FF/NVDA:

  • No labels.
  • Yes, Enable/Disable buttons work.
  • Tabbing backwards through the inputs after ‘Disable’ has been selected skips over the inputs initially, but if user continues one more tab backwards the 2nd input will be announced, “Combobox collapsed has autocomplete”.

5. Disabled results

Geof Collis: JAWS 14.5 IE 11.
Not accessible. Sometimes it reads first and other times it doesn’t it isn’t consistent and I only get one box. Used the back button and it froze my browser and had to close it down, grrr!

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
I’m not able to move from the first option. I’m using Control-Option-right arrow to navigate between the options on the table. Using only right/left/up/down arrow to move from one option to the other won’t work.

If I interact with the table choices (Control-Option-shift-down arrow) I’m able to move to the second option and it will be announced by VoiceOver. From the second option it is impossible to move to the third option.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only:

  • No labels for inputs.
  • Yes, disabled item is skipped over when arrowing.
FF/NVDA:
  • List items are announced as “blank”.
  • Yes, disabled item is skipped over when arrowing.

6. Select with a button

Geof Collis: JAWS 14.5 IE 11.
Semi accessible. But I find all of these boxes confusing and verbose so I’m quitting, I dont like it in the least, time to go back to the drawing board. :O)

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
Yes I can set the option in the select drop using the two buttons for each example. In both examples, VoiceOver does not give feedback after the setting function. I’m able to press the buttons using Control-Option-space bar, only the space bar key and the Enter key.

Also it is possible to open/close the select choices using Control-Option-space bar, only the space bar key and the Enter key. I think those two examples are easiest to use with VoiceOver.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only:

  • 1st Example: ‘Set’ & ‘Init’ work, but the others do not.
  • 2nd Example: Yes, it works.

FF/NVDA:

  • 1st Example: ‘Set’ & ‘Init’ work, but the others do not.
  • 2nd Example: Yes, it works.   Note: Selected items are not announced in the input field they are added to.

7. Limiting the number of selections

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
I was not able to select any of the choices. Once in the table choices I tried selecting with the Enter key (it sent me back to the instructions page), using space bar key only and Control-Option-espace bar did not work either.

The combo box does not respond to Control+Option+Space bar VoiceOver command to open the list of choices. It does respond to the space bar key and the Enter key.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only: Yes, it works.

FF/NVDA:

  • Yes, can only select two items.
  • No, the feedback is not announced.
  • No, the item choices are not announced.
  • No, selected items are not announced.

8. Hiding the search box

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
The search box it is very well hidden, VoiceOver did not notice it.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only: Yes, it works.
FF/NVDA: Yes, Search box is hidden and not voiced by screen reader.

9. Add choices automatically

Gabriela Nino de Rivera Torres: VoiceOver+Safari.
I was able to add choices using the coma and the space. The combo box does not respond to Control+Option+Space bar VoiceOver command to open the list of choices. It does respond to the space bar key and the Enter key.

Michelle DeYoung: Windows 8.1 / FF 40.0.3 / NVDA 2015.2
Keyboard only: Yes, it works.
Note:
Using space – Cannot add more than two rows of items.
Using comma – Cannot fill past one row of items.

FF/NVDA:
Yes, can add items with space or comma, and they will appear in the list.
If too many items are added to the list they are dynamically removed except for the first item.
>Note:
Using space – Cannot add more than two rows of items, but NVDA will still voice like items are being added.
Using comma – Cannot fill past one row of items, but NVDA will still voice like items are being added.
Items – Items in list are announced as “blank”.  Items added to the input are also not announced.

Notes from Bram Duvigneau

I had a quick look at Select 2 using Firefox + latest NVDA nightly build.

The Select 2 widget is announced as combo box, however it doesn’t respect the common keyboard navigation conventions on Windows. I expect to be able to use the arrow keys to select an item when the select box is in closed state, that doesn’t work. The alt+down arrow hotkey, which is the default to open a dropdown in Windows, works. However, when the select box is open, the active item is not announced by a screenreader. I can select an option by pressing enter and after closing the box the selected item is readable.

Conclusions: It seems that some aria support has been implemented. This causes a screenreader to pick up the role (select box) and the state (open/closed).
However, much more work is needed to support the focus in an open Select 2 widget. All the special Select 2 features such as grouping items and typing to filter/search the list has not been tested, but this also needs additional aria markup to be read by screenreaders.

Another issue is the expected keyboard shortcuts. These are different on every platform and Select 2 should feel native on all of these platforms, so this has to be implemented and tested prety well. E.g. on Windows I don’t expect a dropdown to open when I press space bar, but on Mac this is the common way to open such an element.

Notes on the test results by Andrea Fercia

  • Difficult to understand how to open the combo box. Users expect to open the combo box using just the down arrow key. Previous version worked that way.
  • Missing label for the input field (The text input in the expanded list is not labelled). It gets announced with the name of the first option, e.g.: “Alaska  combo box  collapsed  has auto complete”. Investigate on current usage of “aria-labelledby”.
  •  Searching:
    • leaving the field empty, it is possible to arrow through the items in the list but every option gets announced as “blank”
    • entering some charavters e.g. “mi” and arrowing through the results e.g. “Minnesota”, “Mississippi”, “Missouri” will read out just “mi” which is the entered value in the input
    • no feedback about how many search results – the previous version announced “5 results are available, use up and down arrow keys to navigate” or “No matches found”.
  • “I could not find how to select an option of the list of choices”. I guess that’s because all you hear is “blank” or the characters you entered in the search field.
  • No feedback when selecting an item: If user randomly selects and item (they will not know what they selected), the items are added to the input field, but they are not announced.
  • Multiple select boxes: “Unable to access added items in the input to have them announced.  If I assume items exist in that form field, then I can select “Backspace” to remove them a letter at a time.”Once you’ve added options, they’re not announced so users really have to guess they’re in the input field.

Conclusion

The main problems are:
  • There is no or not enough feedback for screen readers on what is selected or what can be selected (list items are announced as “blank”.)
  • No feedback about how many search results.
  • The select dropdown does not behave like a user expects, quoting Bart: “It is really not intuitive to require a space bar in a combo box.This is something I would never have tried by myself”
  • Testers report slightly better results on iOS + VoiceOver
  • Select2 has at the moment too many accessibility issues to be included into WordPress core

#a11y-audit