Theme Review Team

Recent Updates Page 2 Toggle Comment Threads | Keyboard Shortcuts

  • Ulrich 7:30 pm on July 12, 2016 Permalink |  

    Meeting notes for 2016 July 12 

    We had a great dicusssion where we discussed the action plan and the timeline for it and looked at a few items that needed a descision for the theme check sniffs. Read the meeting transcript in the Slack archives. (A Slack account is required)


    Lead @karmatosed

    • update the handbook with make.blog posting guidelines.
    • Give the key reviewers the access
    • Update the about page accordingly
    • Get key reviewers access to edit the theme trac tickets

    Future Project

    Lead : @grapplerulrich

    • Create a new page for current, future and possible projects
    • List current and future projects
    • Enable comments to collect ideas

    Changing the review flow

    Lead : @karmatosed

    • Create a flowchart of the review process – Deadline 2 weeks
    • Remove reviewer if there is no response within 24 hours. Intially this will be manuall will plan to automate this in the future.
    • @emiluzelac updated the handbook to represent this

    Review baseline

    • Themes that do not fullfill the review baseline will be closed. The ticket will be closed if the theme has more then 3 prefixing or security issues. This also applies to the admin review queue.
    • It is not the end when a ticket is closed. The discussion can still continue and an update can always be updated.

    A sample text that you can use

    Reviewers are now allowed to close the ticket if 3 or more security or prefixing issues are found.
    This theme is being closed because the following security/prefixing issues have been found :

    Please do not get discouraged by the ticket being closed. If you have questions you can ask them here or in the #themereview channel on Slack. Fix these issues and ensure you have checked your theme for additional security or prefixing problems.

    You can read up more on security and prefixing on our resources page: https://make.wordpress.org/themes/handbook/review/resources/

    Also take this opportunity to carefully review your theme against theme requirements listed in the reference below.

    Once the issues are fixed upload the new version and we will review it again. https://wordpress.org/themes/getting-started/

    Theme Check Sniff

    The following items were voted on to include in the theme check

  • Ulrich 11:36 am on July 12, 2016 Permalink |  

    Agenda for 2016 July 12 

    The Theme Review Team holds a meeting weekly and we encourage all members to attend.

    Channel: #themereview | Time: Tuesday at 18:00 UTC 18:00 UTC

    Topic : Theme Review Team action plan

    We recently published the Theme Review Team action plan which we discussed partially in the last meeting.

    This time we will be discussing the timeline for the

    • Whole action plan, including line by line the action plan. Lets get a ticket, timeline or action for each bit.
    • Theme Check update

    I have an initial update on the Theme Check update. We have made a fork of the WPCS to the WPTRT GitHub repo. This will allow us to manage all of the checks as separate issues. You can help in a few ways:

    • Open a issue for all of the checks that we have mentioned in the WPCS issue 578.Please use the standard format.
    • Once an issue is opened we could use pull requests for unit tests. These allow us to be sure that our sniffs are working correctly. The test is normally a `inc` file that contains PHP code with a comment at the end to mention if it is bad or good. The test PHP file then list the lines where the should be an error.
    • Once a test is created a sniff can be built upon that.

    We already have a few issues open and a few of them need input from us. The first four are quite easy and I personally would give the go ahead.

    The following issues needs feedback. Please give feedback in the issue.

    The meetings typically last 30 minutes. If time permits, additional topics may be discussed.

    If you have any topics, then please reply to this post and include the topic, along with a brief description of what you would like to discuss.

  • Tammie 3:23 pm on July 1, 2016 Permalink |  

    Theme Review Team action plan 

    Sometimes, things need to change; that’s true for everything. The Theme Review Team is aware that we currently have problems. This post proposes some suggestions for how we should change. The objective of these changes are to reduce queues and make reviewing easier, both for those being reviewed and those doing the review.

    A big thanks goes out to everyone that has helped with writing this post. Specific props to @emiluzelac, @grappleulrich, @greenshady, @samuelsidler, @cais and @jcastaneda. The admin team have signed off on this in agreement.

    Our role as a team should be to check that the theme has no licensing, security, or “breaking” issues. Any issues beyond those three categories should be dealt with after the fact, not during review. We all want to do more, but without ensuring we provide the minimum review to themes in a timely manner, we aren’t succeeding.

    Let’s have a look at the following sections and see how we can improve.

    In order for us to function, we should change the structure of our team.

    • Reduce down to 2 tiers for reviewers: key reviewers and reviewers. No more admins. If you have not been actively contributing to theme review (in either reviews or projects; more on this later) for 6 months, then your key reviewer status will be removed.
    • Key reviewers can reassign tickets and have make.blog access.
    • Key reviewers can delete tickets and edit comments on tickets.

    Communication is key. As we move to two tiers, we will add in a co-review process on the make blog to ensure we are a united voice.

    • No posts will be published unless two key reviewers sign off the post.
    • We will update the handbook with this and posting guidelines.

    Release focus
    To make changes, we need to focus on specific projects.

    • Projects should be linked to WordPress releases.
    • Before the start of each WordPress release, the team will have a meeting to decide focus this cycle, just as the core team does.
    • There will be a post at end of the cycle, listing those that contributed to a project, not only to reviews.
    • Tying projects to a release cycle helps us focus on digestible pieces that can be completed quickly.
    • A page will be created to list current projects and collect ideas for future projects.

    Review flow
    How themes are reviewed is important. We have a lot of blockages in our process to resolve.

    • We should get the process as close to the plugin review one as possible.
    • A ticket is not assigned to one reviewer.
    • The reviewer going through the queue replies to author.
    • Tickets aren’t just allocated to one reviewer. Anyone can review and then pick up a ticket if reviewer doesn’t respond.
    • Keep response time to 7 days for both reviewers and those being reviewed, however as with above, anyone can review.
    • Tickets with no response from a reviewer after 24 hours, will be move to the “new” queue; this change makes it easier to dive in and help.

    Review quality
    If our theme quality isn’t looked at, the queues will get back up.

    • We need to provide education into what is a good review.
    • We should show example reviews for people to learn from.
    • We need to promote anyone that is a good reviewer to key reviewer and shout about it.
    • We all should be encouraging each other to help make reviews better, not by commenting on tickets, but by directly encouraging the reviewer.
    • Our culture should be supportive, to help each other become amazing reviewers.
    • By the end of year, we need to bring back mentoring. Mentoring should be something every reviewer sees as a way to help the team. All key reviewers should participate in mentoring, where possible.

    Review baseline
    What we review needs to change to help the queues and those reviewing.

    • Requirements should be reduced to those that cause security issues or break sites: for example not using WordPress functions.
    • If you find the following in a review, the review is automatically closed:
      • Multiple prefixing issues.
      • Multiple security issues.
    • Other requirements will exist but be automated.

    To go along with all of this above we need an initial roadmap for the next release. Here is our suggestion:

    Theme check plugin rewrite

    • Led by @grappleulrich.
    • Move development to GitHub, giving access to more contributors.
    • Develop the plugin using an easier base, for easier (and more!) contributions.
    • Initial action plan here.

    Reviewer flow

    • Led by @karmatosed.
    • Rework current flow to match the above proposals.
    • Improve uploading flow.
    • Improve reviewer flow.

    Everything about is doable and doesn’t require a lot of changes, just several small important ones. That’s the important thing: the small changes combine to create a large impact.

    There’s a lot to discuss, so please leave your comments below. We will discuss further during the next team meeting on Tuesday Tuesday at 18:00 UTC

    • Omaar Osmaan 3:32 pm on July 1, 2016 Permalink | Log in to Reply

      Happy to see the action plans! Let’s keep moving it forward- 🙂

    • Sakin Shrestha 4:02 pm on July 1, 2016 Permalink | Log in to Reply

      Looks great. Thanks @karmatosed for the post and all other admins for great initiative 🙂

    • janhenckens 4:15 pm on July 1, 2016 Permalink | Log in to Reply

      Excited for this as i get back into reviewing 🙂 Let’s to this! 👍🏻

    • Joy 6:04 pm on July 1, 2016 Permalink | Log in to Reply

      Sounds like a good time to get involved. Where’s the best place to start?

    • Frankie Jarrett 6:38 pm on July 1, 2016 Permalink | Log in to Reply

      Love this! And +1 for GitHub 🙂

    • ThinkUpThemes 7:10 pm on July 1, 2016 Permalink | Log in to Reply

      Awesome ideas here! Thanks so much @karmatosed and the rest of their great ideas 🙂

    • kevinhaig 8:15 pm on July 1, 2016 Permalink | Log in to Reply

      I really like what’s being proposed, reducing the review down to a few key elements. It will definitely go a long ways to speeding up the process.

      I’m a little unsure of the multi reviewer concept, because ownership and responsibility are usually important to getting things done. But given reduced reviewing requirements, it could work.

      Looking at the Review baseline and the Theme check plugin rewrite plan, it appears that many requirements in the current process are not being addressed; content creation, demo content, static page staging, pseudo custom posts, excessive up-selling to name a few. These are all things that will be very difficult to automate. Will these requirements be waved or will additional processes be put in place to address them?

      • Tammie 8:18 pm on July 1, 2016 Permalink | Log in to Reply

        @kevinhaig I think we as a team have to discuss things in this week’s chat regarding requirements that don’t seem included. Some may still be, some we may agree just get in the way at this point.

        • kevinhaig 8:23 pm on July 1, 2016 Permalink | Log in to Reply

          Yea, just thought I would suggest a few things to think about for Tuesdays session.

          • Tammie 8:33 pm on July 1, 2016 Permalink | Log in to Reply

            Thanks, please do.

          • Abby 1:51 am on July 9, 2016 Permalink | Log in to Reply

            Great initiative, I am so glad we’re trying to work this out. I’ve been reviewing themes non stop for 5 months now and it is a great learning experience. Regarding the points @kevinhaig brought up – we didn’t discuss them in last week’s meeting and I really think this is important. The time spent on reviewing one theme is mostly spent on pointing out, testing, and then redoing the very things that he has pointed out. Knowing what the next steps for the reviewers are would be very helpful! All the best!!

    • Emil Uzelac 8:32 pm on July 1, 2016 Permalink | Log in to Reply

      Boom! 🙂

    • Thiago Senna 8:46 pm on July 1, 2016 Permalink | Log in to Reply


    • PlutonWP 9:31 pm on July 1, 2016 Permalink | Log in to Reply

      This is a very great news, thank you very much 🙂

    • Turker YILDIRIM 10:09 pm on July 1, 2016 Permalink | Log in to Reply

      It’d be good not to expect or require full review from reviewers.

      Sometimes i dont have much time to do full review but instead i can quickly check screenshot or licence issues for copule of themes.

      I’m sure some people dont do reviews because they dont feel “confident” to do full review.

      My suggestion is if an reviewer just want to check screenshots, let him/her do. Other parts could be handled by different reviewer.

      • Justin Tadlock 12:17 am on July 2, 2016 Permalink | Log in to Reply

        You can even do this now. If you have time to just pop in the list and point out issues like screenshots and licensing, do it.

        I think that’s one thing we’ve never put enough emphasis on. Just drop in and do what you can if you don’t have time to do a full-on review.

    • Ahmad Awais 8:00 am on July 2, 2016 Permalink | Log in to Reply

      Awesome stuff. I am all for making the review process easy and quick. I think shifting it to GitHub can greatly improve the workflow. CI can help with automated testing and many reviewers can contribute. It’s also intimidating to think about submitting a theme that has to wait for 6 months to get accepted (which I completely understand why).

      I am considerate of the fact that TRT is trying very hard to get better on so many levels. I can only expect good things to happen from here on out 🙂

    • Patryk Kachel 9:02 am on July 5, 2016 Permalink | Log in to Reply

      I have some proposition about encouraging people to “learn by review”.
      We should improve email message send to authors when they submit new/updated theme. Something like:
      “We need your help, please help us reviewing themes.
      Some links to get started, maybe some info about our fancy badge, link to slack channel, etc”
      I think it should be easy task 🙂
      Many authors even do not know about such possibility…

    • Stanko Metodiev 5:14 pm on July 6, 2016 Permalink | Log in to Reply


  • Ulrich 5:41 pm on June 28, 2016 Permalink |  

    Agenda for 2016 June 28 

    The Theme Review Team holds a meeting weekly and we encourage all members to attend.

    Channel: #themereview | Time: Tuesday at 18:00 UTC 18:00 UTC

    Topic : Improving the theme review process

    In the last meeting we discussed the main pain points that @poena found in her analysis. We will continue the discussion today.

    Related meta tickets:

    She has also create a proposal for a FAQ for theme authors.

    The meetings typically last 30 minutes. If time permits, additional topics may be discussed.

    If you have any topics, then please reply to this post and include the topic, along with a brief description of what you would like to discuss.

    • Carolina Nymark 1:36 pm on June 29, 2016 Permalink | Log in to Reply

      We don’t need to call it a FAQ if that is a problem.
      But the questions are frequently asked for a reason. They are either not covered or the answer is too difficult to find. As an author you are less inclined to read all the information that is written for the reviewers.

  • Jose Castaneda 3:18 am on June 28, 2016 Permalink |  

    Thank you reviewers 

    Believe in yourself! Have faith in your abilities! Without a humble but reasonable confidence in your own powers you cannot be successful or happy. Norman Vincent Peale

    Reviewing themes is not an easy feat for many. It means taking a plunge and making mistakes along the way. I want to thank the following people for not only believing in themselves and doing at least one review in the last week but for also sharing that knowledge in Slack and Trac, and not being afraid of making those mistakes and learning from them as well.

    @AbdulWahab610, @DenUsenko, @Fortisthemes, @Heiko_Mamerow, @Jake.WPsupport, @JohnnyPea, @Kenshino, @LittleBigThing, @Nemanja, @ThemeZee, @WEN-Solutions, @acalfieri, @afrothemes, @amannawhal, @anand_kumar, @andrescifuentesr, @bedefy, @benlumia007, @bogdand, @bubblegumlabs, @catchthemes, @chrsi, @clarionwpdeveloper, @correctdev, @cristian-ungureanu, @dariodev, @edmund.turbin, @electricwp, @fashblogger, @fellyph, @grapplerulrich, @holger1411, @imon-Hasan, @infotheme.com, @ionutn, @janhenckens, @juhise, @jurajk, @kafleg, @kapilsharma12, @kevinhaig, @kharisblank, @kolenad, @kristarella, @lyrathemes, @ma_wordpressorg, @mehedi7, @meissudeep, @monikarao, @mynamevenu24, @octavianus, @ombresetlumieres, @patilswapnilv, @pixelflower, @poena, @ptasker, @rabmalin, @renamer, @rmdiaconu, @robertacausin, @rodicaelena, @rohitink, @saumya010, @scui2, @siobhyb, @sketchbookkeeper, @smartcat, @sonjajaakkola, @stevekamn, @sushil-adhikari, @swetashrestha, @tadaimushi, @taniatasmin, @tejaswini, @thelukemcdonald, @themexpand, @thinkupthemes, @trkr, @uxl, @yehudah, @zhangrongfeng

    Many thanks to you all!

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc
Skip to toolbar