Follow the standard theme review instructions on setting up a test environment. You should perform all your theme reviews in this environment.
You need to have an account at WordPress.org in order to log into the Theme Review trac. Make certain that your theme-trac user profile has a valid email address, to ensure that you’ll be notified about updates and comments on themes that you are reviewing. You do not need to be a theme reviewer to perform an accessibility-ready review, but it’s helpful for you to know as much as possible about the whole process.
Claiming a Review
When a new theme shows up on the Accessibility Ready report, check it and see if a comment has already been added to the theme indicating that somebody will do an accessibility review; if so, don’t worry about it, this theme is already claimed. If there isn’t one, simply add a comment on the theme stating that you’ll do the accessibility theme review once the theme is ready to be approved.
Don’t begin the accessibility-ready review until the theme is ready to be approved as a theme or near to it. Many submitted themes never reach the point where they are actually ready to be made live, and with limited time, we want to focus our efforts on themes that are already ready to be published.
Once you’ve claimed a theme, it’s your responsibility to keep track of its progress. If the theme submitter chooses to remove the accessibility-ready tag, the tag also needs to be removed from the ticket; you need to be a Theme Reviewer to do this, but you can always ping me or an admin to take care of it for you, or ask the theme reviewer.
Doing the Review
When reviewing a theme for accessibility, go through each point of the Theme Review Guidelines first, in order. Make the review clear so that each guideline is represented separately as a line item in the review.
You are only reviewing on the strict text of the guidelines as written; you can freely suggest improvements that aren’t part of the guidelines, but you should not fail a theme because it has an accessibility issue that is not documented in the guidelines.
Remember that these are guidelines, not rules. There are always possible solutions that don’t conform to the letter of the guidelines but still satisfy the needs of accessibility.
Be sure to inspect the site in both normal browser view and in any responsive views; menu toggles are common problems for keyboard accessibility.
If you have any accessibility questions during a review, cc
davidakennedy on the ticket.
After handling the required accessibility guidelines, check for ARIA landmark roles. If the theme uses them, inspect the theme to ensure that ARIA roles are used correctly and that all content of the site is wrapped in at least one role. ARIA landmarks are not required, but if they are used, they must be used correctly.
If you find issues with accessibility, publish the ticket comment and wait for follow-up from the theme author. After the theme author uploads a new version, you’ll need to check it again, and repeat until their theme meets the requirements, at which point you should make a comment that explicitly states that the accessibility-ready review has been passed, and that the theme is approved for the accessibility-ready tag.
It’s not your job to figure out how to resolve a problem with a theme; but if you have specific suggestions, do provide them.
Accessibility Guidelines: Notes for Reviewers
This is not a dogmatic review in which we require any specific heading structure; the usage of the phrase “reasonable” should be considered flexible. Each sub section is required to use headings, and those headings should be hierarchically organized; but use your judgement. The requirement is that a headings map of the page will return usable data, not necessarily a perfect structure.
There are many examples of potentially repetitive link text on a page; these can be difficult to spot in the test data, since where they are used is variable. There’s a specific theme unit test post that should display a More link, and archives pages frequently use “Continue Reading” links. Also keep your eyes opens for other repetitive strings.
The most common occurrence of this issue is with responsive menu toggles, but it could apply in many other environments, as well, if the theme has implemented other types of controls. Use your browser’s inspector to check what element is used. Buttons and links are acceptable; any other element is almost certainly not.
Tab through the page to check three things:
- Tab order is natural and intuitive
- All links and controls are visually apparent when they receive :focus
- No links or controls are skipped during keyboard navigation
You’ll need to make sure that you’ve set up a menu in the theme that implements a drop down menu. Several appropriate menus are automatically included with the theme test unit data, but may need to be activated in the theme you’re testing.
Testing contrast is tricky. There are many tools for testing contrast, and you’ll need to pick the tool that fits your workflow best. We require conformance to WCAG AA. Many tools that globally test color contrast are only color aware, and do not also pick up on font size, so you’ll need to make sure to check the actual size of the fonts displayed. You can get this information in Firefox by using the element inspector and switching to the Computed tab, where the calculated font size is shown in pixels.
You also need to be aware of alpha transparency; where alpha transparency is applied, many tools will not pick up the correct color (including Gez Lemon’s Color Contrast Analyzer).
As you gain experience, you’ll find it easy to notice color combinations that look suspect and need to be individually checked, but this is an area where it’s difficult to notice every possible pairing. Be as thorough as you can be.
Note in particular that contrast changes also apply to change states: if a region changes color or background on :focus or :hover, that change must meet contrast requirements, unless there’s a secondary change.
Skip links are easy to test. Tab from the browser toolbar into the site. If you do not see a skip link before you tab into the site content, that’s a failure. If the skip link does not move you into the content area when you activate it, that’s a failure.
Most themes will implement only two forms; a search form and a comment form. Check both for appropriate labeling, indications of required fields, and test submission and response behaviors. Standard WordPress behavior is acceptable; if the development has implemented anything fancy (live search, AJAX responses, etc.), that should be checked carefully.
Most themes implement very few images. Be careful to differentiate between images coming from the theme and images coming from the theme unit test data; if it’s inside the content of a post, it’s from the theme unit test data. Featured images must include the alt attribute. Featured images are generally clearly labeled in the theme unit data.
For the purpose of testing, “images” includes usage of font icons like dashicons, which are graphic in nature even though they don’t employ the
img element. For purely ornamental images, a blank alt attribute is conforming, for ornamental font icons, I recommend use of aria-hidden.
Reference the W3C’s alt text decision tree for assistance on a specific image, if required.
This has not yet come up in any of the themes I’ve reviewed; but it should be pretty self-explanatory.