Theme Review Team Update March 4th

Stats at a glance

Currently

  • 52 new tickets are waiting for review.
    • 0 tickets are older than 2 weeks
    • 15 tickets are older than 1 week
  • 11 tickets are assigned.

In the past 7 days

  • 154 tickets were opened
  • 258 tickets were closed:
    • 85 tickets were approved.
    • 114 tickets were not-approved.
    • 59 tickets were closed-newer-version-uploaded.

What we’ve been up to

Concerns

One of our biggest concerns currently is keeping the queue under control, especially with the recent increase in ticket submission rate. We currently have almost 100 people with ticket assign/close privileges, which means that we would only need, on average, every person to close 3 tickets every 2 weeks, to keep the queue clear. Even accounting for half of those people not being active, that still means that we only need half of those people to close 3 tickets every week. Theme Review is an intensive, yet all-volunteer activity that by its very nature will see waxing and waning of participation; so it is unrealistic to expect that even half of all reviewers will be active in a given week. That we got participation from almost 1/3 in a concerted effort over the past week probably indicates the realistic upper limit of week-to-week participation.

If average participation is even half of that upper limit, that now means that active reviewers need to average one ticket closed per day, in order to stay on top of the review queue.

On a related note: originally, we attempted to put a more formalized “trainee” program in place; but we found that the overhead for such a system had diminishing returns, at best. So, we scrapped that system, and simply started granting ticket assign/close privileges to anyone who showed enough initiative to request at least a few Themes to review. This has worked to increase participation, but what we gain in participation we potentially lose in review consistency and quality. One of my goals is to get enough consistent participation that the admins can focus on ensuring review quality/consistency, instead of spending most of our effort on clearing the queue. So, solving the former issue of keeping the queue under control will help us address this issue as well.

We are open to ideas.

#themes, #trt, #weekly-updates