Call for Testing: wp-theme-auditor

Building on initial efforts at WCUS 2018 Contributor Day in Nashville, the Accessibility Team has continued to move forward with the development of automated accessibility testing tools for WordPress themes. The work so far is available in the wp-theme-auditor GitHub repository, and the Accessibility Team is now seeking feedback from the broader WordPress community in order to make this tool as useful as possible.

wp-theme-auditor is a Javascript package which includes end-to-end testing tools from the Gutenberg project and Deque Labs’ axe accessibility testing framework. It is meant to be installed as a development dependency in a WordPress theme, and includes a default set of test cases based on the Accessibility Team’s a11y-theme-unit-test-data. wp-theme-auditor can be run against a local development site (remote sites are theoretically supported but untested).

You can install the tool by running the following command (if the theme to be tested does not already include a package.json file, you will need to create one first by running npm init -y):

npm install --save-dev wpaccessibility/wp-theme-auditor

Installing the dependency also runs a postinstall script which copies the required Babel and Jest configuration files to the theme directory and copies required test files (including the default test cases) to a test folder within the theme directory.

Two scripts must then be added to the theme’s package.json file:

  "scripts": {
    "create-test-cases": "create-test-cases",
    "test:axe": "wp-scripts test-e2e"
  }

If you want to test your theme with these default test cases, you need to import a11y-theme-unit-test-data into your development site first. Then, running WP_BASE_URL=https://mytheme.test npm run test:axe (where the local development site URL is https://mytheme.test) will audit all the imported posts and pages using axe and provide detailed feedback about any violations. (The imported posts and pages from a11y-theme-unit-test-data have been crafted to ensure that content does not include any accessibility issues, so tests will catch accessibility issues in the theme itself rather than content.)

Running npm run test:axe will load any files which match the pattern test/*.test.js, so if you need to exclude any of the default test cases, you can move them out of the test folder.

The other script, create-test-cases, provides an interactive command line prompt for creating new test cases for custom post or page templates. For example, if your theme has a custom page template for a contact page and you want to make sure that your contact form passes axe tests, you can use the interactive prompt to specify the post type (post or page), slug, and label for that test:

npm run create-test-cases
Creating test cases...
? What is the post type? page
? What is the slug of the post or page? contact
? What is the title of the post or page? Contact
Test case created at /var/www/wp-content/themes/mytheme/test/contact.test.js.

Theme developers are encouraged to try wp-theme-auditor and share feedback via the project’s GitHub repository. Future goals for the tool include the ability to add it to themes using the wp-cli scaffold command and the ability to run tests in a Docker container as part of a continuous integration (CI) environment.

While all automated accessibility testing has limitations and there are many accessibility issues it cannot catch, wp-theme-auditor could be an exciting new tool to guide WordPress theme developers towards accessibility best practices.