Lets be clear about what is required or not.

I thought I’d do a little Monday morning public service blog post. I like all admins see a lot of reviews. As a result some trends occur and I want to start mentioning those and trying to combat things outside of tickets. This post doesn’t come from one single post, it comes from watching things and quite a few.

So, lets start with the big things I see lately that if I was a new reviewer or theme creator I’d be confused over. The big main point, is that we have to be clear about what is in the guidelines, what is opinion. This also goes for the tools we use. To that point there are a few things we need to stick to and will stop the confusion.

Lets get it right out there:
https://make.wordpress.org/themes/handbook/about/join-the-team/
https://make.wordpress.org/themes/handbook/guidelines/

These are the only tools currently recommended and what you can hold a theme review to along with the guidelines. We all have our own methods and that rocks. Some of us, use extra tools. Some of us, use spreadsheets.. some of us have our own amazing unique way of doing things. But, lets get a few things straight in how we present things on the theme ticket.

If something is in the guidelines it’s been decided and passed as part of the theme review. This is what you approve or don’t approve based on.
If something comes up in the mailing list, no matter who says it, this isn’t a rule until it’s in the guidelines. Therefore, you can not mark a theme as not approved as a result of not meeting that. You can however add a note to encourage the theme creator.
If you use a tool that’s not in the official list, you can’t pass or fail a theme review based on it.
If you use a tool that’s not in the official list, please don’t link to it in your review. External links from a reviewer that don’t relate to a bug or code fix, should be avoided in any section on tracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/.. People can find out about in other ways.
If you have a tool that’s not in the official list but you want it to be, start communicating with the admins. Work with us to make this process better for everyone.
Lets be clear about the language we use. Call all the official tools by their right name. Lets not call any official tools by the name an official tool has – it gets really confusing fast otherwise.

It’s worth noting nobody is saying you can’t have a notes section where you pass on advise to the themer. I love it when people do that and would love people to do that more. Use that section to educate, pass no the amazing knowledge you have.

If we can all keep to these points, we can limit confusion and make sure we’re all on the same page as reviewers. At all times it’s good to think what it’s like for first time reviewers and theme uploaders. If we have them in our mind, we can ensure we make this a good process for anyone.

Mentors, can you also please set a good example by following these and also making sure we keep everyone from the start on the path of less confusion.

Now go do some awesome reviews, I’ve taken up enough of your time!

Greg Priday 1:26 pm on October 13, 2014

All points noted. I’m really glad to hear this part:
If something comes up in the mailing list, no matter who says it, this isn’t a rule until it’s in the guidelines.

Although, does that mean we can’t trust what you’re saying now until it’s part of the guidelines? 😉 kidding.

In the past, we have had admins stating things as if they were part of the guidelines (saying “as per the Guidelines”) even though I could never find anything in the guidelines to back up what was said.

We could avoid a lot of confusion if we have a policy of admins answering reviewer’s questions by sending an actual guideline URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org. This is really important for when we’re doing reviews or training new reviewers.
- Tammie 1:32 pm on October 13, 2014
  
  If something is decided in the mailing-list then becomes a guideline, it would be updated in the guidelines. This has happened time and time again and is the process that works.
  
  I think a lot of us as admins do give guideline links when appropriate.
  
  It’s worth noting that admins are humans. As such, we try and do our best but we are not keyed into the guidelines matrix style, nobody is. We should all be aware we’re all human and do our best to support and ensure that the theme review family sustains itself and is a healthy, nourishing environment for all involved.
  - Greg Priday 1:52 pm on October 13, 2014
    
    I know. Sorry, I wasn’t blaming the admins. Rereading what I wrote, I think unintentionally came across like that.
    
    For reviewers, understanding the guidelines is the steepest part of the learning curve. Especially understanding how specific points in the guidelines relate to real world examples. But if anything comes up in the mailing list that I can’t relate back to the guidelines, I’ll make a point of asking for guideline-clarification. This is the kind of information I’d like to pass on to reviewers I’m mentoring 🙂
    - Tammie 2:03 pm on October 13, 2014
      
      No issue at all. I just think it’s really important we all remember we’re a family in this together. I see it as a team effort to get reviews done and progress the team.
      
      Mentors as you point out are a really useful way of supporting admins. As a mentor you have a one-to-one chance of getting the right things into the hands of a reviewer before their journey has even started. I think that’s so cool.
      
      It’s worth saying a huge thanks to you as a mentor and anyone else that does it. You really are working hard to foster the next generation of theme reviewers. It’s amazing to see.
Courtney Ivey 3:17 pm on October 13, 2014

Well said, Tammie… Kudos!
Justin Tadlock 2:51 pm on October 17, 2014

If something is in the guidelines it’s been decided and passed as part of the theme review. This is what you approve or don’t approve based on.

Most of the time, this is correct. However, it’d be impossible to list everything in the guidelines. If code is incorrect, it needs to be fixed.

I’ve seen many instances of reviewers requiring several things be changed that are neither in our guidelines nor incorrect though. Then, a few lines down in the same file, there’ll be something that clearly violate the guidelines that goes unmentioned. The best thing is to make sure it’s clear what is required vs. what is recommended.

Maybe we should go back to the one-page guidelines we used to have. It was almost like a checklist you could go down and check. Right now, you need to either memorize the guidelines or spend precious time going from page to page. I think it’s been mentioned before, but the front page should be a checklist with the other pages having a deeper explanation if need be.

If you use a tool that’s not in the official list, you can’t pass or fail a theme review based on it.

I don’t see why not. If you manage to find a required issue, what does it matter what tool you used to find the issue?

If you use a tool that’s not in the official list, please don’t link to it in your review. External links from a reviewer that don’t relate to a bug or code fix, should be avoided in any section on tracTrac Trac is the place where contributors create issues for bugs or feature requests much like GitHub.https://core.trac.wordpress.org/.. People can find out about in other ways.

Part of our job as reviewers is to educate and provide theme authors with tools that will make them better theme authors. Linking to additional tools that they can use is always beneficial, regardless of whether they’re in an “official” list. If I’ve got a tutorial that explains step-by-step how to do something the right way (for example, I have one I wrote about sidebars), would it not be beneficial to share that with the theme author?

If you have a tool that’s not in the official list but you want it to be, start communicating with the admins. Work with us to make this process better for everyone.

What are the official tools? Assuming they’re the tools listed on the “Join the Team” page, a theme author may never see those. We should probably have a page in the guidelines specifically for that. I use Theme Check, the Theme Unit Tests, and Notepad++ to perform reviews. Maybe we can make this list easier to access for theme authors.
- Tammie 2:59 pm on October 17, 2014
  
  I think if it’s not in the guidelines and it’s being advised. It should become part. After all that way lies confusion 🙂
  
  Part of our job as reviewers is to educate and provide theme authors with tools that will make them better theme authors.
  
  Absolutely agree. That’s why we should be careful about confusing with tools and other things not everyone is using. If you note, my example is someone failing something due to a tool. If it’s useful then lets make it a standard and all use it.
  
  What we need to be extremely careful about is setting an uneven path for people. This is true for both those starting to review and the themers submitting themes. The issues come when we are not.
  
  I totally agree we need to have better documentation and be clearer. This is a long road to tread but an important one. We should also not see the guidelines as unable to be ammended – more they should be seen as a living document in that sense. Same goes for the tools – if something rocks lets recommend it.

Welcome to the Themes team!

Handbooks and review requirements

Coding standards and plugins

Code Packages

Communication