Theme Review Team Update: 07/22/13



  • 5 new tickets are waiting for review.
    • 0 tickets are older than 2 weeks
    • 0 tickets are older than 1 week
    • 0 tickets are older than 3 days
  • 65 tickets are assigned.

In the past 7 days

  • 127 tickets were opened
  • 218  tickets were closed:
    • 131 tickets were made live.
      • 38 new Themes were made live.
      • 93 Theme updates were made live.
      • 2 more were approved but are waiting to be made live.
    • 82 tickets were not-approved.
    • 5 tickets were closed-newer-version-uploaded.

Theme Review Incentive Program

This week, the Theme Review Team unveiled a trial incentive program for Theme reviewers:

With the recent changes to the Theme Review infrastructure, we are now much more easily able to track Theme review and approval of new Themes, previously reviewed Themes, and updates to currently approved Themes.  While we have a much higher volume of tickets being closed than we have had in the past, we end up mired in a cycle of Themes being not-approved, re-submitted, and then not-approved again. If we are ever to gain long-term control of the review queue, we need to break that cycle – and the only real way to do that is to ensure that, as often as possible, all Themes submitted eventually get through the review process, and approved.

In an effort not only to bring the Theme Review queue under control but also to further our ultimate objective: to ensure Themes are approved and made available in the Theme directory, we are going to try an incentive program of sorts. As a side benefit, this incentive program will partially address another area of frequent discussion: the Featured Themes list.

This Trac Report tracks new and previously reviewed Themes that have been approved and made live in the past 30 days. It will be the basis for our experimental incentive program.

Every month, we will determine the three reviewers with the most tickets in this report. Each of those reviewers will get to select a Theme to be included in the Featured Themes list for the next month.

The program appears to be initially successful. By taking advantage of infrastructure changes (thanks @nacin and @Otto42) in order to change the focus from closing tickets to approving Themes, we appear to have made a dent in breaking the cycle of repeat “not-approved” tickets for a given Theme. In less than a week, we’ve eliminated the backlog of open tickets waiting for review activity.  The one number that has increased is the number of currently assigned tickets – but this number represents reviewers working in-ticket with developers to address outstanding issues, instead of just closing as “not-approved” as had been the norm previously. This improvement bears watching, to ensure that it can be maintained long term, but this is a great first step.

I’ve changed the displayed stats somewhat, to highlight the number of new Themes being made live, as separate from the number of Theme updates made live.

#themes, #trt