Theme Review Team

Recent Updates Page 2 Toggle Comment Threads | Keyboard Shortcuts

  • Tammie 11:18 am on July 20, 2015 Permalink |  

    Update queue process 

    In the attempt to work out what steps we need implemented and what we need to build. Here are the steps we are looking at for the updated queue. This is the process a theme will go through when updated. Note: this is a work in progress and some things need to be talked about.

    1. A theme is updated in the normal way.
    2. The theme check plugin runs over the theme.
    3. If theme check passes, the theme goes live.
    4. If theme check fails, the theme doesn’t progress until it’s passed.
    5. If the theme goes live, no trac ticket is made (or is.. see questions).
    6. We need a queue of recently updated tickets – note this may need tickets or something to indicate this. This queue is only for people if want to check updates.
    7. A button is on all themes in directory. This button is ‘report this theme’.

    Report theme process
    1. The report button is clicked.
    2. If logged in the reporter is logged and also a ticket made and put in reported queue.
    3. The theme review team monitor this queue as a priority. Once a ticket appears there someone takes it and does a full review.


    • Should we make tickets but not count them as open?
    • Do we need tickets to have a trac of what has happened to the theme?
    • Do we have to log the reporter of the report theme button? It could act as a good citizen effect and make people think before clicking.
    • Should we add a step to the button to get the reason reporting?
    • noplanman 11:32 am on July 20, 2015 Permalink | Log in to Reply

      Since each theme review has a ticket which shouldn’t get closed anyway, wouldn’t it make sense to use the existing ticket and proceed with it? Maybe add a label “reported” to also let the author know about it?

      Personally, I think people might be scared off by the fact that they would be “logged” as the reporter of an issue. I can see the advantages of logging the reporter, but if potential reporters don’t want to be logged, there might be a checkbox or something to opt out of being logged.

      As to giving the reason of reporting, I think that’s very important for everyone involved. It helps the author to address the problem and/or explain his view and it helps the reviewer to set a focus on what the reported issue is.

      Note that I haven’t done any theme reviews yet and so have no experience of the current workflow, but I feel it might even be advantageous that I’m not bound to the way it is now.

    • Towfiq I. 12:13 pm on July 20, 2015 Permalink | Log in to Reply

      Great! I have few concerns about the process.

      1.What should be the criteria of reporting a theme? I guess you guys already assumed that a lots of users will click the button if the theme doesn’t function as they want.

      Scenario A: Upon installing the theme, user’s site doesn’t look like the theme’s screenshot, user will assume that the theme is broken, what will he/she do? “Report the theme” as broken.
      Scenario B: Suppose there is a bug in the theme that prevents user to upload a logo. User will act as a good citizen and report the theme. And this may happen every time a some users find a bug.

      How do we prevent unwanted reporting like this?

      2. Auto Update + Theme options Panel:

      The themes that has theme options panel instead of customizer, they will be able to get past the “Customizer Only” rule if this “auto update process” is live.

      To prevent this, the Theme Check should show errors when a theme contains “Settings API”. Right?

      • Tammie 10:02 am on July 21, 2015 Permalink | Log in to Reply

        I think it’s pretty much been decided in comment you’d have to select reason to report. So that solves it. Yes, we probably will get some false positives but that’s not a bad thing, we adapt.

        Your second point isn’t valid yet. We’re just dealing with one issue now.

    • Andre 12:30 pm on July 20, 2015 Permalink | Log in to Reply

      First, one has to appreciate the effort and hard work that the TRT puts in to review themes that get submitted and I know it takes a lot of time and many volunteers to make that work. I also know that the queue of submitted themes can build up into long wait times, as long as 2 months from the time a theme is submitted to the point of going live.

      As a theme developer, I have to say I have concerns with the new direction of this queue process where themes, if passing the initial upload and check, go live automatically….unless I’m missing something here? Based on the number of themes that get submitted, there’s the potential of seeing a fast rotation of of themes being pushed out of the “Latest” themes list within as little as 2 days. For the theme author, this will become discouraging because with the amount of work someone puts into a theme (up to a month) means their theme will be out of sight within a short period of time. For the end-user who comes to wordpress.org or views the latest releases from the dashboard, they might end up missing a theme that could fit their needs because of the number of themes going live will increase the list daily.

      On average, when a theme goes live, the theme author will have their theme near the top of the Latest Themes list for at least a week. Based on the number of themes that get submitted, I’m seeing anywhere from 3 to as many as 12 themes per day being added.

      Something else to consider….for the end-user, there’s always the potential that something might not be done correctly or not secure with the theme, or simply just bad coding style and layout. Even if the automated review process passes and puts the theme live, isn’t this a rish? One of the reasons why the human side of reviewing is necessary so things that can be found that otherwise might not be from automation. Remember that every theme author codes differently….can this automated process differentiate and acknowledge how a theme is built?

      The whole concept of having a theme reviewed by an actual person, not to mention followed up by a second one (an admin or lead reviewer), gives assurance and confidence to end-users when they know a theme went through the extensive review process.

      Regarding the Report button…are we really sure this will be used responsibly and legitimately…not to mention that an end-user really knows and understands the purpose of it? Towfiq I. offers a couple scenarios of many that could happen.

      As a side note….I’d really like to know what brought this new theme submission process to fruition in the way that it’s being proposed?

      • ThemeZee 12:37 pm on July 20, 2015 Permalink | Log in to Reply

        @ Andre: The proposal is about the update queue (theme updates) and not about the new themes queue. Therefore any themes submitted for the first time would still have to pass through the normal theme review process with a human reviewer.

        • Justin Tadlock 9:11 pm on July 20, 2015 Permalink | Log in to Reply

          While this specific post is about auto-approving updates, we hope to eventually auto-approve new themes as well. This is just the first step.

      • Andre 12:44 pm on July 20, 2015 Permalink | Log in to Reply

        Little update ….as ThemeZee pointed out, this process is for the Theme Update Queue. My apologies for ranting off a big long comment that turns out most of it is not valid, lol. Oops! Couldn’t sleep so I got up at 430am. Still, I would guess my Report button part could still be valid with some concern.

        ***Incidentally, why does the comments here at make.wordpress.org not have an “Edit” button? Rather annoying.

        • Tammie 10:03 am on July 21, 2015 Permalink | Log in to Reply

          The report button is enabling the community to self regulate, so it’s not an issue. It will be something we adapt and learn with.

    • ThemeZee 12:35 pm on July 20, 2015 Permalink | Log in to Reply

      1) In my opinion tickets should be made with every update. It is always useful to see the update history and be able to track all changes in the diff if you need it for a review. Tickets could be closed and made live automatically.

      2) I am also not sure if the report button will work. Most users simply do not know the review guidelines and what the button is meant for. And in case the button is displayed on a prominent spot in the directory the reported queue could get out of control pretty fast.

      However, I guess it is worth testing it. If the report button does not work maybe an automatic reporting is an alternative. After 10 or 15 updates the theme is added automatically to a report queue. Updates would be live immediately without any waiting time, but themes would still be reviewed from time to time if they match the guidelines.

      • Andre 12:39 pm on July 20, 2015 Permalink | Log in to Reply

        Is that what this is about, updates only? lol…ah, hence me saying “Unless I missed something here”. This is what happens when one does not get enough sleep.

      • Tammie 10:03 am on July 21, 2015 Permalink | Log in to Reply

        This is a flow just for updates.

    • Chip Bennett 3:02 pm on July 20, 2015 Permalink | Log in to Reply

      My thoughts, somewhat random:

      1. Generate a Trac ticket for all actions, even if it is auto-closed, and auto-made-live. That way, the history of the activity of the Theme is preserved.

      2. Reporting MUST require two things: a) a w.org username, and b) a reason for reporting the Theme – for two reasons: a) so that the system won’t be ripe for abuse, and b) so that the reviewer will know why the Theme needs further attention (there’s no need for a full review if the reason for reporting is a specific issue).

      3. If a Trac ticket continues to be generated for every instance of a Theme uploaded, I don’t think there’s any particular need for a new/additional queue.

      • Tom 3:48 pm on July 20, 2015 Permalink | Log in to Reply


        • cristiano.zanca 9:25 pm on July 20, 2015 Permalink | Log in to Reply

          An automated way like plugin check in the first phase is good to avoid copy and paste with check results using many tickets with the theme submitter.
          The human check in the code and in the UX and in many other aspects I think would be done anyway before go live.

          • Tammie 10:05 am on July 21, 2015 Permalink | Log in to Reply

            We’re not saying a human check is done for updates. That’s the point of this work flow.

      • Justin Tadlock 9:15 pm on July 20, 2015 Permalink | Log in to Reply

        +1 to the reporting ideas here. I was just about to write the same things.

      • @mercime 10:39 pm on July 20, 2015 Permalink | Log in to Reply

        +1 to all of Chip’s points.

      • wpHoot 10:58 pm on July 20, 2015 Permalink | Log in to Reply

        +1 for all, especially point 2.

      • Sakin Shrestha 3:55 am on July 21, 2015 Permalink | Log in to Reply

        1. For update auto live, keeping track ticket will be beneficial to track changes. If there is any alternative to that then it’s fine.
        2. +1 for Reporting as we don’t have unnecessary reporting.

      • Stanko Metodiev 6:34 am on July 21, 2015 Permalink | Log in to Reply


      • Tammie 10:04 am on July 21, 2015 Permalink | Log in to Reply

        Ok, so I’ve been considering this and agree:

        • Tickets should have a ticket.
        • The report button has the username and a page showing reasons. I think this at least helps us monitor and reduce spam.
        • There is a need for a new queue in sense all tickets even if closed would go there. We just have an ‘all tickets’ queue.
        • Chip Bennett 1:55 pm on July 21, 2015 Permalink | Log in to Reply

          It might just be a terminology difference. To me, “queue” means “list of things that need acting on”. If you’re merely meaning “report”, then I certainly agree.

        • Ulrich 9:04 pm on July 21, 2015 Permalink | Log in to Reply

          We should also not forget that we have tags in trac that we can use.

      • Emil Uzelac 11:39 pm on July 23, 2015 Permalink | Log in to Reply


    • Carolina Nymark 3:17 pm on July 20, 2015 Permalink | Log in to Reply

      +1 for keeping the tickets. -Live themes that has or adds an accessibility tag should not go live without being checked -but at the same time it is not right to “punish” accessible themes…

    • progmastery 3:59 pm on July 20, 2015 Permalink | Log in to Reply

      Is not there the list of reasons for reporting theme?
      When clicking the ‘report’ button, it can be offered to choose the reason for reporting from this list. Additional comments can provide more details.
      To avoid abuse of updates TRT can also include in the “latest ” only the new themes, and not newly updated as it is now.

      • Tammie 10:06 am on July 21, 2015 Permalink | Log in to Reply

        Lets not assume abuse of something that’s not built yet. It’s a far better frame of mind to not skip right to the negative. Our goal is to encourage the community to self regulate with this button also. We’re along with this looking at the directory – that’s another consideration outside of this post.

        • Chip Bennett 1:57 pm on July 21, 2015 Permalink | Log in to Reply

          I think it is both safe and wise to assume that a “report” button will get abused, whether inadvertently or maliciously. I also think that tying reports to username and requiring a reason for reporting will mitigate the vast majority of risk of abuse. Anything else, we can deal with in a one-off fashion, unless/until we see any trends.

    • Nilambar Sharma 9:58 am on July 21, 2015 Permalink | Log in to Reply

      In Report process we should also consider behavior of clicking button just to check what it does. We have all seen the `Request Theme` button and its side effects (increase in number of tickets in Report 30). One click and `boom`. So, may be we should pull ticket to `reported queue` only after at least 2 or 3 user clicks.

  • Tammie 6:49 pm on July 14, 2015 Permalink |  

    Theme review team weekly meeting notes

    The logs are here:

    We voted on the roadmap:

    We agreed to follow it.

    The following people have stepped up to be point people in groups:
    Directory: @greenshady
    Tools: @jcasteneda
    Reviews and queues: @emiluzelac
    UX and research: @karmatosed

    Documentation doesn’t have a point person yet. If you are interested, add a comment. Ideally we’d have someone that has been in team for at least a little while and reviews.


    • I’ll post a flow for the updates auto approval. This will be the first thing we do.
    • I’ll post a second flow for new tickets ready for next week’s matting.
    • Next week we’ll discuss the next phase of the roadmap: auto approval of uploads and the directory changes. In preparation for this all the point people will consider the points in roadmap for their groups.
    • Chip Bennett 7:17 pm on July 14, 2015 Permalink | Log in to Reply

      I can be the point person for documentation, but will need someone who can help with the multi-lingual aspects, as that is not my forte.

      • Tammie 7:33 pm on July 14, 2015 Permalink | Log in to Reply

        Awesome. The idea isn’t so much the point person does things as they report back so that’s great and you don’t need to do multi-lingual.

  • Samuel Sidler 5:50 pm on July 14, 2015 Permalink |
    Tags: translations   

    Theme Translations on WordPress.org 

    tl;dr: Theme translations and language packs are coming to WordPress.org and they’re awesome.

    Howdy all you wonderful themers and theme reviewers,

    The meta team has been working hard to enable theme and plugin translations on translate.wordpress.org. For themes, we plan on importing all active themes into WordPress.org – that is, any theme updated within the last two years. We expect to import them in the next few days or weeks, at most. This will involve importing ~1500 themes, which, combined, have about 315,000 total strings. After duplicates, the number drops to only 80,000 unique strings.

    Below are some things you might want to know.

    Why do I want WordPress.org managing translations for my theme?

    WordPress.org provides translations in dozens of languages and is ever expanding as new contributors join. (There are currently 140 locales on translate.wordpress.org, but not all are active.) While you may have translated your theme into a few languages (or none!), there are likely more translators on WordPress.org in more languages.

    But that’s not all! Themes in the WordPress.org directory will be able to take advantage of language packs! That means smaller download sizes for users, because themes will no longer need to ship translations. Eventually, we also plan to give priority to localized themes in localized directories; e.g., someone searching the Romanian theme directory will see Romanian themes prioritized over English-only themes.

    What if my theme already ships translations?

    Translations that are already shipped in themes will be initially imported into translate.wordpress.org. Again: we’ll import the strings and the translations on the initial import. We won’t continue to do that because the end goal would be for theme authors to remove the translations from their download, allowing language packs to fill the void.

    What if I don’t want to use WordPress.org to manage translations for my theme?

    Then you don’t have to! Translations shipped in a theme take precedence to language packs. If you continue to ship translations with your theme, WordPress will ignore the language packs. However, if a translation is available in a language your theme doesn’t support, WordPress will use the language pack for that language.

    How do I add support for translations and language packs?

    @Otto42 wrote up a great post on the topic back in 2013. (Wow, it’s been a long time!) There’s also a great page in the theme developer handbook which walks through how to internationalize your theme.

    To fully support language packs, you’ll want to remove translations from your theme in your next update.

    What if I want my translators to approve translations on WordPress.org?

    We’ve written up a plan for working with the polyglots team to enable this. There will be some initial pain in adding new, project-specific (aka theme-specific) translation editors, but afterwards your translators will join a growing group of WordPress translators and help make the entire ecosystem better.

    Other questions?

    Just ask! We’ll watch this thread and answer any questions you might have.

    • Jose Castaneda 7:12 pm on July 14, 2015 Permalink | Log in to Reply

      Sweet! Looking forward to this!

    • Milan Ivanovic 1:21 am on July 15, 2015 Permalink | Log in to Reply

      This is huge. Nice!

    • Marcoevich 6:50 pm on July 15, 2015 Permalink | Log in to Reply

      Wow this is good stuff! However it would be even better if you’d allow non-wordpress.org themes to also store their translation files on wordpress.org. Is there a possibility that you add this in the future?

      If you’d allow that, you can make wordpress.org the Nr 1. place to be for theme translations :)

      • Samuel Sidler 7:12 pm on July 15, 2015 Permalink | Log in to Reply

        Currently, we have no intention of opening the door to non-WordPress.org themes. I don’t see that changes in the future either, but anything is possible.

        Essentially, translations (and language packs) will now be one of the benefits to being part of the WordPress.org ecosystem. If, as a theme author, you do not wish to release your theme for free, under the GPL, on WordPress.org, and within the bounds of the theme review team, you don’t get benefits of the ecosystem.

    • Emil Uzelac 9:11 pm on July 16, 2015 Permalink | Log in to Reply

      Good job!

    • Justin Tadlock 1:54 pm on July 18, 2015 Permalink | Log in to Reply

      I just wanted to step in and say that I’m excited! I’ve actually been excited about this proposal for a very long time.

      Keep up the awesome work!

    • Stanko Metodiev 8:37 pm on July 18, 2015 Permalink | Log in to Reply

      That’s nice!

  • Tammie 9:33 pm on July 13, 2015 Permalink |  

    This week’s meeting agenda

    We have a meeting tomorrow at 18:00 UTC in #themereview on Slack. Lets get as many people as we can and vote on the roadmap: https://make.wordpress.org/themes/2015/07/07/roadmap-for-phase-one/

  • Tammie 9:48 pm on July 9, 2015 Permalink |  

    Theme review team weekly meeting notes

    The logs are here:

    We had a short meeting. There are no plans for a meeting on Thursday. The roadmap is up:

    Lets try and get as many people as we can next Tuesday and vote on the roadmap. We also need to get point people for the various sections.

compose new post
next post/next comment
previous post/previous comment
show/hide comments
go to top
go to login
show/hide help
shift + esc