Theme Review Incentive Program

Hi theme reviewers! It seems like there’s been some disagreement about the implementation of the theme review incentive program lately. Disagreement draws attention, and once it was looked at more closely, turns out the theme review incentive program (as it exists right now) is actually kind of a problem. In this post, I’ll lay out the issues that jump out, and ask for your feedback; we’ll gather up the team opinion and take it to a conversation with Matt about what should be done moving forward.

When this team suggested the program, I think there may have been a miscommunication about the intent/outcome, or perhaps it just sounded like such a cool idea that some of the more questionable aspects didn’t get the scrutiny they could have, but when Matt approved trying the idea as an experiment it sounded more like it would keep themes rotating through the Featured section for the benefit of users, not that people would be featuring their own themes in exchange for reviewing themes for the directory.

Issues:

  • People choosing their own themes to feature. Basically, what this amounts to is a form of pay-for-play, where someone gives something of value (in this case reviewing time and expertise) in exchange for something of value (advertising in the form of a featured theme listing). That’s pretty much the opposite of how contributing to the WordPress project is supposed to work. No other team has that kind of tit for tat, nor should they. Recognizing contributors needs to be absolutely separate from formalized promotion of their products, businesses, etc.
  • Themes being chosen repeatedly. The featuring was intended to be a way to move more themes through that lineup (and not by putting more themes on that page), and though that page wasn’t getting the attention it deserved before the TRT program, we should still be working toward that goal. Any theme that is always on the Featured page might as well just be called a default theme; if someone thinks their theme is so awesome that it should basically be a default theme, then they should consider donating it to core as a potential base for the next year’s default theme.

The goal of Featured Themes is to present people with a variety of quality themes they might not see otherwise. So featuring the most popular themes (which are already listed under Popular Themes) or the same themes month after month doesn’t really serve that goal. And telling people that if they volunteer their time they’ll be rewarded with advertising is pretty unfair to people volunteering in other areas of the project. Recognition is something we need to get better with, but we shouldn’t be using promotion of products — even if those products are free — as a reward. So! What does that mean for this program?

Basically, it has to change. In talking to Matt, we came up with a few possibilities for the featured themes page:

  1. Abandon the program altogether, and use an algorithm to select featured themes to auto-rotate so there’s always something different there rather than a static selection for any period of time. Note: this is Matt’s preference.
  2. Switch to an algorithm to choose featured themes, but build in a feature for theme reviewers to suggest themes for inclusion that they reviewed and think are really good, so that it pulls from a pool recommended by TRT members (incentive or just in general, whatever).
  3. Keep the incentive program but put two rules in place: no one can feature their own theme, and no theme can be selected to be featured more than once in a six month period. That addresses both the project pay-for-play issue and the stagnation issue, but still retains the possibility of gaming the system (you choose my theme, I’ll choose yours, etc).
  4. Something we haven’t thought of but that addresses these issues.

So. In the comments, I ask that you weigh in, with these caveats:

  1. Be constructive. No accusations, no name-calling, no anger. Just working toward a common goal. Your comment should move this conversation forward, not rehash old ones.
  2. Focus on the goals of the project and how this impacts users, the team, and the overall project, not your own personal goals. If you care about your own goals and not the goals of the WordPress project, that’s fine, but that’s not what project-wide decisions are based on, so it shouldn’t be the subject of a novel-length comment here.
  3. Think about other ways we can recognize (rather than reward) contributors on this team. Putting recognition on the pages of themes they’ve reviewed? Doing something like the about page they do for core but for the theme review team? Any and all suggestions encouraged, as long as they don’t veer into pay-for-play territory.

Sorry if this bums you out. Hopefully we can resolve this quickly and painlessly so we can all get back to what we were doing. :) Thanks!