Agenda for June 27

This weeks meeting is estimated to take longer than our usual 30 minutes since we have several topics.

Meeting times and agendas

  • Since we don’t have agendas each week and because people are not able to participate every week, the suggestion is to have two meetings a month: one with a set agenda, and one with an open floor.
  • The meetings are estimated to be one hour long, compared to the 30 minutes we have now.
  • We will try to document topics that come up often, maybe on the FAQ page or a dedicated meeting topics page.
  • If you want to lead a meeting please DM one of the Team Leads which dates you are available.
  • The two weeks per month when we don’t have meetings should be dedicated to working on our ongoing projects.
  • We would like to remind everyone to please stick to the topics so that we can have productive meetings.

Shindigs

Our next review shindig is on saturday july 8.

Two themes rule

We suggest allowing authors to have two themes in the queue, up from one. This should then be evaluated after one month.

WordCamp Europe, community summit and contributor day recap

  • New reviewer tool -Trac will be replaced
  • Readme file will be parsed
  • No more queue
  • Reviewers will not be assigned to tickets
  • Tickets will not be closed just because there is no reply within 7 days
  • Tickets will not be closed because of code errors
  • Updates via SVN

Discussion: What can we do now/short term?

X-posting Proposal: WordPress Community Conduct Project

Please read + comment on the original post.

Proposal: WordPress Community Conduct Project

The goal going forward

The goals for the near future are:

  • Comparing the theme and plugin review processes to see how we can work towards a goal of making both processes very similar so that any improvements made to one will also profit the other.
  • Put a team together to tackle the theme previews as this is a major issue. Ask Matt to put resources towards this.
  • Automation is still a priority to catch common issues and giving us more confidence that themes cannot be shipped that give sites a white screen of death.
  • Define the minimum requirements that need to be manually reviewed for the themes to be hosted on w.org
    • Licensing
    • Security Issues
    • PHP Errors
    • Illegal
    • Dishonest
    • Morally offensive
    • Egregious issues that we haven’t listed
    • Accessibility Reviews will be done after the themes go live.
  • Agreeing to try out a way to get more feedback from users on different aspects of themes and using the extra data on the theme to rank them appropriately in the repository.

Comparison between the Theme and Plugin review process

  • As we are discussing how we can improve the Theme Review process it would be good to see how our process compares to the Plugin Review process. It would help both teams to align so that we could both profit from improvements made to either system. Both processes are always changing. Parts of the process today may change tomorrow. The plugin review team is working on making the plugin review process public so that others can contribute.
Review Process Themes Plugins
Submission zip file  zip file
Review management in trac (Public) in the WordPress admin area (Private)
Review updates  via trac via email
Reviewers Anyone with a w.org account select group of experienced reviewers (will change in the future but only experienced reviewers will communicate with the developers)
Number of reviewers varied between 10 – 30 active reviewers 5-6 reviewers
Experience level of reviewers variable highly experienced
Reviewer training regular  not needed
Order of reviews order of submission  order of complexity of the plugin
Volunteer time minimal sponsored time sufficient time sponsored
Approval A final review is done by an experienced reviewer before going live The review approves the theme and then the developer must commit the code to SVN
Theme Updates Via zip files via SVN
Theme update reviews only automated reviews no additional reviews
Feedback on the review process Anyone can give feedback and work to improve the review process The small group of reviewers make the decisions
Automated Testing checking for common issues simple testing for PHP errors
Requirements A single detailed document of all of the requirements. Multiple pages on the guidelines: The general guidelines, a reviewer’s checklist and a page explaining how to check for these items.

My learnings from this have been:

  • A small highly experienced team with the resources can sometimes been more effective than a larger team.
  • Having separate information for developers and reviewers may change the public perception of the number of requirements.
  • As the plugin and theme reviewers are looking for similar issues it may be better to work together to unify some of the documentation.

Agenda for 2017 April 18

The Theme Review Team holds a meeting weekly and we encourage all members to attend.

Channel: #themereview | Time: Tuesday at 17:00 UTC 17:00 UTC

Recap:

Topics:

This week we want to start defining specific items that we want to prioritise so that we can get back to Matt with an update since out last chat.

  • Prepare a post comparing the theme and plugin review process to see how we can work towards a goal of making both processes very similar so that any improvements made to one will also profit the other. Responsible: @grapplerulrich
  • Put a team together to tackle the theme previews as this is a major issue. Get Matt to put resources towards this.
  • Automation is still a priority to catch common issues and giving us more confidence that themes cannot be shipped that give sites a white screen of death.
  • Define the minimum requirements that need to be manually reviewed for the themes to be hosted on w.org. The basic items I could think of are checking for licencing, anything illegal, dishonest, or morally offensive.
  • Agreeing to try out a way to get more feedback from users on different aspects of themes and using the extra data on the theme to rank them appropriately in the repository.

Once we are agree on these items we can get back to Matt and discuss the next steps.

 

If you have any topics, then please reply to this post and include the topic, along with a brief description of what you would like to discuss.