Dive Into Distributed Unit Tests

Recently, I have spent some time looking into using pre-built tools for distributed unit tests. What I found was that all of them were built to fetch the test results. What we need, for this project, is to have the results sent to us. At this point it seems like a custom WordPress solution will be the best route.

There was some discussion in #hosting-community about how the tests would be triggered. Some hosts have scripts that run nightly, so slipping tests into that process would make implementation easy. However, using a nightly solution would make it more difficult to determine exactly which commit caused an issue.

If we move forward with running tests on a per-commit basis, do hosts watch for commits and then report the results? Or do we ping hosts? It seems like pinging hosts would be ideal, but it might not be an option for all hosts. It would be really nice to get some feedback here, so your thoughts would be appreciated.

The first step is to start on the method used to report results. This will be a WordPress based application, with a REST API. Hosts will submit the results with the commit SHA to be aggregated. I’m going to dig into this over the next several weeks and will share a GitHub link when I have something up and running.