Matrix Test Results

In the AccessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) chat for Friday, August 4th, the team ran the entire meeting within Matrix, using a variety of different clients, to assess the current state of the environment for users with disabilities. If you’re not already familiar with Matrix, it’s an open-source federated chat platform that is being evaluated as a possible replacement for SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Right now, it is possible to use Matrix or Slack clients to interact in a Slack meeting, so while we held our meeting using Matrix, it was recorded in the Slack accessibility channel. Review the conversation in Slack.

As long as we can use both clients to interact, there are no significant concerns. However, we need to set accessibility criteria that the Matrix experience needs to meet before it’s feasible for the project to migrate solely to that platform. (Other teams may have other criteria, of course.)

Summary

Overall, Matrix was marginally usable for screen reader users and keyboard navigation. Most buttons were labeled and possible to interact with, and it was possible for screen reader users to post messages, read messages, and change channels. (Or rooms, in Matrix terminology.)

However, there were significant barriers to usability that would make the experience slow and frustrating for both these groups of users. As a productivity tool, it would significantly decrease the productivity and participation rate of users with disabilities.

Issues Noted

Please note that this was not a formal testing scenario, and should not be considered complete. This was a live interactive session in which we explored the environment trying to discover as much as we could in a short time.

  1. @alexstine (NVDA, Windows, Chrome, expert screen reader user) So far, I’m finding the web version of Element not verbose at all, causing me to really explore the page to get anything useful.

    Issue: Using a screen reader, the application didn’t provide much in the way of orienting tools that Alex could use to efficiently locate key functions and information.

  2. @Travel_Girl (Maja Benke) (Element web app, Firefox, Dark Mode, prefers reduced motion) I find it hard with keyboard to use it, but I’m not a native keyboard user.

    Issue: Two fundamental issues here. First, the keyboard shortcuts are difficult to locate. Second, the keyboard shortcuts are inadequate, and don’t provide tools for moving between application regions or navigating messages.

  3. @alexstine Landmarks are lacking and I can’t figure out how to predictably access the message list.

    Issue: As above, this is a lack of orienting tools and navigation mechanisms for reading messages.

  4. @alexstine Can you find a keyboard shortcuts guide? I’m coming up empty on basic docs.

    Issue: Keyboard shortcuts list was difficult to find in the app, and online docs were extremely minimal.

  5. @alexstine Got to be honest, this experienced is pretty terrible in comparison (to Slack). There is just no way to navigate the app globally. Slack uses F6 or Ctrl+F6 in the browser, and none of these work in Element web.

    Issue: Inability to navigate around the app is a major problem for screen reader users.

  6. @alexstine …There is no global navigation shortcut. This is going to be an absolute nightmare to use outside of fairly advanced users. It’s like Discord, passes useability standards but not by much.

    Issue: This is a common problem in applications that have done a good job with micro accessibility issues, but haven’t given significant consideration to macro issues. E.g., buttons are labeled and it’s possible to move from control to control, but the larger scope of navigation and orientation isn’t sufficient.

  7. @joedolson (Android Element app, Talkback) Worked OK in Android/Talkback. I didn’t explore extensively, and found one unlabeled button, but message navigation and authoring was fairly easy.

    Issue: The unlabeled button was the ‘Threads’ button, if I remember correctly.

  8. @Travel_girl I don’t know how to access the rooms via keyboard (if the room is not already open) as the tabindex is not consistent.

    Issue: While Ctrl+K can be used to open a command center to search rooms, navigating via various standard keyboard commands doesn’t move you around the page in a very predictable manner. This would be less of an issue if there was better exposure of keyboard commands.

  9. @alh0319 (Amber Hinds) When you open the more information button in the web app it doesn’t shift focus into that panel and so it’s essentially as if it’s not there. I haven’t even figured out how to tab into it.

    Issue: Loss of focus prevents users from using important interfaces.

  10. @joesimpsonjr Had difficult uploading an image (random I know) but no ability to add ALT text.

    Issue: Users cannot add alternative text to images. This makes images inherently inaccessible to screen reader users.

  11. @alh0319 I’m Mac, Chrome. I can get to the button and it reads[sic] as a button, but the space bar and Return key don’t trigger it. I think it’s acting as though my focus is still in the message list.

    Issue: Seems like a problem with custom key events not firing correctly. I’m not 100% clear what the context of this was; in the conversation thread, it may have been referring to the upload image button.

  12. @alh0319 Also if you search for a person to send them a DM, there is no way to get to the results for the person. The “button” to select the person is a div.

    Issue: This refers to the People > Start Chat button in the navigation sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme.. There is no way to navigate using the keyboard to choose a person to chat with.

  13. Numerous people commented on the lack of formatting controls. While formatting can be done in markdown (or at least markdown-like syntax), not all users are familiar with these options. There was no documentation of how to do this in the app that we could locate.

  14. @joedolson I also don’t like that there’s no border that indicates where the message text area is located. I keep clicking above it.

    Issue: Especially for low vision users, having a well-defined indication where you need to click to initiate an action is very helpful.

Reviewed by @alexstine for technical accuracy and clarity.