The idea behind the hosting test results page is to run WordPress’ automated test suite across a variety of hosting environments, to help verify that changes in WordPress don’t unintentionally affect hosts, and to help hosts be aware of how changes in WordPress or their environments affects how WordPress runs on their services.

Hosts that report receive automated emails when the tests are passing on WordPress.org, but failing on their environment so that they can investigate.

Want to run the WordPress PHPUnit test suite on your own hosting infrastructure? Thank you; we look forward to seeing your results reported!

If you have questions about the process or run into test failures along the way, please open an issue in the project repository and we’ll help diagnose/get the documentation updated. Alternatively, you can also pop into the #hosting-community channel on WordPress.org Slack for help.

Setting Up

There are two primary steps in the setup process.

First, you’ll need to configure the phpunit-test-runner to run in a representative server environment. Read through the GitHub README for full configuration details. Notably:

  • The test runner can be driven from Travis CI or a cron job on the machine itself.
    • Eventually, we’d like to trigger the test runner for every commit. At this time, running it every 3-6 hours should be sufficient.
  • There are a variety of environment variables you can set to control the test runner’s behavior.

Second, once the test runner is passing all tests in your server environment, you can begin reporting results to WordPress.org. To do so:

  1. Create a bot WordPress.org account. If your company is Wonderful Hosting, Inc., this bot account username might be wonderfulbot. Make sure to set its email address to something monitored by a human — we’ll send an email when there’s a test failure.
  2. Create a new issue in the phpunit-test-runner GitHub repo requesting your bot user to be added to this WordPress.org site as a “Test Reporter”. We’ll need the email address associated with the user in order to add them. Under the hood, we’re using a custom post type to log and display test results.
  3. After your bot user has been added to this WordPress.org site, you’ll be able to create an application password for your test runner. Sign into the WordPress.org site and navigate to Users -> Your Profile to generate the application password. Then, set the application password as an environment variable: export WPT_REPORT_API_KEY='wonderfulbot:Osho NHgM xYSY UWF9 qNUn YdjV'. Replace the characters after the colon with the application password for your bot user.

If all goes well, you’ll see your test results submitted at the end of php report.php:

The last line indicates your results have been successfully uploaded to the reporter website.