- Tests done in June 4-9, 2015
- Update June 14, results from Michelle DeYoung
- Update June 18, results from Kim van Iersel
- Update June 20, results from Michelle DeYoung (NVDA/ChromeVox, after integration into core)
- Update June 23, results form Tobias Clemens Häcker (ChromVov, after integration into core)
Questions for the test team
- can you add a menu
- can you fill it with menu items
- can you remove a menu item
- can you reorder a menu item
- can you access the menu options
- can you access the menu help
Testing using NVDA latest version, Firefox and Internet Explorer latest builds and windows 7
Just did some tests on the menus.
And I found them hard to navigate stuff all over the place.
I think I reordered a menu successfully but no feadback.
I can edit various items but I couldn’t find where to add one.
I was able to find a menu button at the bottem of the page to add but didn’t manage to edit and add it.
The structure was quite confusing.
Testing using Windows 8.1, IE 11, and JAWS 16.
First, to actually get into editing the menu is a lot of clicks and
reading buttons that need to be expanded. by the time I was able to
locate where to edit the actual menu in terms of the menu items
themselves I felt kind of lost.
The text on the button reads Add Links. It might be better to read
something like Add Menue Item/Links or Add menu Item.
I was able to add a menu and add links. However, I was *not* able to
move any of the menu items. The move options are not clickable by
From a usability/accessibility standpoint the menu customizer is slow
and cluncky to use. It took me a good 15 minutes to add a menu which
using the current menu tool would have taken me about 2 minutes.
Granted I had to figure my way around the UI, so there’s that.
I know the current proposal is to remove this feature from the menu in
4.3 and place the menu creation function inside the customizer. I
think I like the concept, but think we should punt it to 4.5 to ensure
accessibility of the feature is strong. I think other parts of the
customizer need a lot of accessibility work as well.
Testing using JAWS 14.5 IE 11
When I click on a sub heading I’m not sure where I am, took me a bit to realize so I would like the focus to be on the heading I clicked on and not the back button after the heading.
Focus works well when I edit a menu item but I get dead air when I arrow through edit boxes , navigation and title attribute for example
There are way too many sections here, I would like to see it set up the way the other menu section:
1 area for all options
list pages, categories etc without having to click on a heading.
be able to change all attributes of a menu item in the same place, with this setup I have to go to another section to reorder, this is just to time consuming and annoying way they have it.
Conclusion: If they are going to do away with the Menus section then just move it to the Customize section just the way it is.
On the whole I was able to do basic tasks, like select which menu is active for which location, add/remove links, reorder links and add new menus. It took a little while to figure out how it all worked but once I understood this, was able to use it. The issues are listed below, some are down to limitations of ARIA support in Supernova.
Testing with Supernova V14 on Win7 64bit & IE11
When using the reorder feature, there is no screen reader feedback to indicate the menus level, perhaps a level indicator can be added after the page title that says something like “top”, “level 1”, “level 2”, etc to show how far down the tree each menu item is. As this is only really to benefit screen reader users, it could be visually hidden using the CSS clip method.
When adding new menu items, the overlay appears at the end of the page, which is not imediately obvious, perhaps moving focus to the top of this overlay on the click event of the add links button will help alternatively insert the overlay in the dom tree at the point where the add links button was clicked or where the link will be inserted.
The level 4 headings in the add new menu items overlay are not declared as links, so is not obvious that they need to be clicked to show/hide content. Either use regular html anchors or add ARIA attributes to indicate these headings have a click action.
Testing with NVDA V2015.1 on Win7 64bit & IE11
The click events to select the menus tab or any section/menu within the tab, with NVDA you need to press enter twice, the first press appears to put NVDA into a form edit mode and the second press makes the selection and exits the form edit mode. Not sure why this is.
When reordering menu items, NVDA anounces “menu is now a submenu item” or “menu item moved out of submenu”, which is useful, However, when reviewing the list while reordering, there is no indication at what level any given menu item is currently set to.
When adding new menu items, the overlay is placed at the end of the dom tree, it will be better if placed either after the add links button or at the point where the links are inserted.
The level 4 headings in NVDA also anounce “toggle”, to indicate the click action is required to show/hide options. This probably don’t work in Supernova V14, due to very limited ARIA support.
Also the toggle state for collapse/expand menus in the preview panel, work in NVDA, were as Supernova doesn’t work so well, giving the same output for both states.
Gabriela Nino de Rivera Torres
VoiceOver and Safari.
Adding a new menu
Add a menu button: when the user select the button she hears “add a menu button press return or enter to open”. When clicking on on the button, a new menu name field and button appears, but the user is not aware of it.
When the user clicks on “Create menu” the user hears that the menu was created. The new menu has a blue color that differentiates it of the rest, but when I try moving to another element in the page it gets berserk.
I not sure if this weird behavior is due to using Safari, but when the interface gets crazy it is very difficult to find the way back to anything. At some point, a lot of elements that are hidden gets focus and it is very confusing to know were I am. After it gets berserk I need to refresh the page and start anew.
It was difficult too to add an item to the menu. There is a lot happening on the screen that is not necessary told to the user.
I will agree with previous comments, adding menus or items requires a lot of clicking and the way that page elements are organized makes very hard to understand the elements order with the screen reader.
I prepared a video to show the strange behavior after clicking on the “Create menu” button.
Michelle deYoung (June 14)
Creating Menus, Keyboard only testing
Tested in Windows 8.1 using FireFox, Chrome and Internet Explorer 9 with Keyboard only.
Issue: Once a new menu is created, the keyboard focus is moved to the content area.
Suggested: Keyboard focus should remain in the left sidebar menu section so the user can add items, reorder, delete menu, etc.
Issue: The Reorder link receives focus, but does not reorder menu items when selected. The “Add Links” button might be more understandable if it is renamed.
Suggested: Rename “Add Links” button to “Add Menu Item”.
Issue: When adding a Menu item, the item is added twice to the Menu sidebar.
Issue: When tabbing from the ‘X’ to the “Add Links” button, there are 12 to 14 tabs that have no visible focus indicators. The number of tabs vary. I assume this is from hidden items that it is hitting.
If ‘Enter is selected, it unfolds a blank area above the Customizing > Menus item.
‘X’ and “Save and Publish” only appear to get focus when the user tabs in reverse tab order.
Suggested: Remove extraneous tabbing to reach focusable elements on the screen, so user can easily identify where they are at on the screen.
Issue: No visible focus indicator for the help icon.
Suggested: Make it gray like the Properties (gear) icon, then blue when it receives focus. Or add a border around it when it receives focus.
Firefox: No visible focus indicator for the help icon.
This is how it looks in IE: Visual focus on the help button
Issue: Access to menu item – reverse tab order: Once user has added a menu item, if they choose to navigate/move backwards in the tab order to customize the added item, the items in to expandable. The user has to navigate back to top of the browser and start from there.
Issue: In Internet Explorer 9. When the “Add Links” button is selected and the right side section is expanded, the user focus is dropped into the Search field which is fine, however, the user is not able to see the placeholder text unless they don’t enter a search criteria and tab out of the field.
Suggested: Allow the placeholder text to be visible when the field receives focus.
Place tab on the expanding section, and allow the user to tab into the Search field.
Kim van Iersel (June 18)
On Jaws 16 Pro.
I did all the tests and could answer everything with: Yes I could do that (add a menu. fill it with menu items, remove a menu, reorder a menu, access menu options, access menu help).
Michelle deYoung (June 20, after integration into core)
Tested in Windows 8.1 using Chrome + ChromeVox and FireFox + NVDA
- can you add a menu (YES)
- can you fill it with menu items (Yes – as far as functionally with the screen reader and keyboard. No – The extraneous text voice by ChromeVox and NVDA will be confusing to user and could very well impact their ability to accomplish their task.)
- can you remove a menu item (YES)
- can you reorder a menu item items (Yes – as far as functionally with the screen reader and keyboard. No – it is not very understandable for a non-sighted user to hear “Reorder” and “Done” without knowing they have to navigate back to the menu itself, and without instruction of what and how to reorder. Once back in the menu the reorder buttons are nicely done and announced clearly as to what the action is.
- can you access the menu options (YES)
- can you access the menu help (YES)
Issue: Add Menu button:
Once the user has selected ‘Add a Menu’ the focus stays on that item.
Before selecting ‘Enter’ or ‘Return’:
ChromeVox and NVDA:“Add a menu press return or enter to open.
After selecting ‘Enter’ or ‘Return’:
ChromeVox:“Add a menu press return or enter to open”.
NVDA: only announced after user as moved away from the button and then back to it:“Add a menu press return or enter to open.”>
Remediation suggested: Once the user has selected ‘Add a Menu’ it might make it easier to have the cursor focus dropped into the ‘New menu name’ field.
When the button changes to the ‘X Add a Menu’ it should say something like
“Add a menu press return or enter to collapse or close.”
<button class="button-secondary add-new-menu-item add-menu-toggle" type="button">
Add a Menu
<span class="screen-reader-text">Press return or enter to collapse or close </span>
Issue ChromeVox: >Add Menu button
Once the menu name is added and the ‘Create Menu’ button is selected,
Screen reader voices:
- “Menu create”
- “Status create menu button”
- “List items widgets press return or enter to open this panel heading 3”
- “List item static front page press return or enter to open heading 3”
- “Collapse internal link”
- “Search widget within search widgets edit test search entry”
- “Archives heading 4”
- “A monthly archive of your sites posts”
- “Calendar heading 4”
- … (and so on)
Issue NVDA: Add Menu Field
Form field is voiced as “List with two items edit has auto complete”
The input element needs a label relationship defined.
Placeholder is only read with NVDA if user accesses the field via forms mode. If the user is arrowing to the item in virtual mode no label will be announced.
<label><input class="menu-name-field" type="text" data-customize-setting-link="new_menu_name" value="" placeholder="New menu name"></label>
Place holders should not be used for labeling. Use explicit or implicit labeling. When using implicit labeling ensure that the for and id attributes are used to create a data relationship between the label and form field since some AT’s don’t support implicit labeling well.
If you do not want the label to show hide it from view using css, but allow AT’s to still access the label name.
Note: If aria-label is used on the input for a label, it will only be read by screen readers when tabbed to.
<span class="hidefromview>Enter new menu name</span> (hide label text from view with css, but allow it to be read by screen readers.)
<input id="menuname" class="menu-name-field" type="text" data-customize-setting-link="new_menu_name" value="" placeholder="New menu name">
Issue: Adding a Menu Item
Add button is just announced as “Add”. The item name is read prior to the button, but it might be more understandable for it to be voiced with the item name.
Ex: “Add a Gallery Page menu item”
Issue: When user selects ‘Enter’ to add a menu item it is duplicated in the left panel.
Issue NVDA/FireFox: Reorder/Done button
Once the user as selected “Reorder” the focus remains on the button that then has changed to “Done”.
Note: “Done” is only announced is the user navigates away from the element and then back to it.
Reorder should have some more information associated with it for non-visual users. Perhaps taking user focus to the top of the list of menu items along with additional instruction on what to do now the reorder button has been selected would be beneficial and more understandable
Issue NVDA: Removing Menu Items
When the add menu item window pane is open, the “X” for removing the item is available, but once the user closes the pane and moves to the window pane where they can edit the menu item, the remove “X” is moved down toward the bottom of the expanding collapsing accordion items for editing.
Tobias Clemens Häcker (June 23, after integration into core)
Windows 7, Chrome, ChromeVox
Except for menu items being added twice everything works well. Was surprised how easy it was to use.
Video (might need some minutes to render)