WordPress.org

Ready to get started?Download WordPress

Review WordPress Themes

Recent Updates Toggle Comment Threads | Keyboard Shortcuts

  • Jen Mylo 9:33 pm on April 18, 2014 Permalink | Log in to leave a Comment
    Tags: incentive program   

    Theme Review Incentive Program 

    Hi theme reviewers! It seems like there’s been some disagreement about the implementation of the theme review incentive program lately. Disagreement draws attention, and once it was looked at more closely, turns out the theme review incentive program (as it exists right now) is actually kind of a problem. In this post, I’ll lay out the issues that jump out, and ask for your feedback; we’ll gather up the team opinion and take it to a conversation with Matt about what should be done moving forward.

    When this team suggested the program, I think there may have been a miscommunication about the intent/outcome, or perhaps it just sounded like such a cool idea that some of the more questionable aspects didn’t get the scrutiny they could have, but when Matt approved trying the idea as an experiment it sounded more like it would keep themes rotating through the Featured section for the benefit of users, not that people would be featuring their own themes in exchange for reviewing themes for the directory.

    Issues:

    • People choosing their own themes to feature. Basically, what this amounts to is a form of pay-for-play, where someone gives something of value (in this case reviewing time and expertise) in exchange for something of value (advertising in the form of a featured theme listing). That’s pretty much the opposite of how contributing to the WordPress project is supposed to work. No other team has that kind of tit for tat, nor should they. Recognizing contributors needs to be absolutely separate from formalized promotion of their products, businesses, etc.
    • Themes being chosen repeatedly. The featuring was intended to be a way to move more themes through that lineup (and not by putting more themes on that page), and though that page wasn’t getting the attention it deserved before the TRT program, we should still be working toward that goal. Any theme that is always on the Featured page might as well just be called a default theme; if someone thinks their theme is so awesome that it should basically be a default theme, then they should consider donating it to core as a potential base for the next year’s default theme.

    The goal of Featured Themes is to present people with a variety of quality themes they might not see otherwise. So featuring the most popular themes (which are already listed under Popular Themes) or the same themes month after month doesn’t really serve that goal. And telling people that if they volunteer their time they’ll be rewarded with advertising is pretty unfair to people volunteering in other areas of the project. Recognition is something we need to get better with, but we shouldn’t be using promotion of products — even if those products are free — as a reward. So! What does that mean for this program?

    Basically, it has to change. In talking to Matt, we came up with a few possibilities for the featured themes page:

    1. Abandon the program altogether, and use an algorithm to select featured themes to auto-rotate so there’s always something different there rather than a static selection for any period of time. Note: this is Matt’s preference.
    2. Switch to an algorithm to choose featured themes, but build in a feature for theme reviewers to suggest themes for inclusion that they reviewed and think are really good, so that it pulls from a pool recommended by TRT members (incentive or just in general, whatever).
    3. Keep the incentive program but put two rules in place: no one can feature their own theme, and no theme can be selected to be featured more than once in a six month period. That addresses both the project pay-for-play issue and the stagnation issue, but still retains the possibility of gaming the system (you choose my theme, I’ll choose yours, etc).
    4. Something we haven’t thought of but that addresses these issues.

    So. In the comments, I ask that you weigh in, with these caveats:

    1. Be constructive. No accusations, no name-calling, no anger. Just working toward a common goal. Your comment should move this conversation forward, not rehash old ones.
    2. Focus on the goals of the project and how this impacts users, the team, and the overall project, not your own personal goals. If you care about your own goals and not the goals of the WordPress project, that’s fine, but that’s not what project-wide decisions are based on, so it shouldn’t be the subject of a novel-length comment here.
    3. Think about other ways we can recognize (rather than reward) contributors on this team. Putting recognition on the pages of themes they’ve reviewed? Doing something like the about page they do for core but for the theme review team? Any and all suggestions encouraged, as long as they don’t veer into pay-for-play territory.

    Sorry if this bums you out. Hopefully we can resolve this quickly and painlessly so we can all get back to what we were doing. :) Thanks!

     
    • Rohit Tripathi 9:41 pm on April 18, 2014 Permalink | Log in to Reply

      The first 2 suggestions are actually great. Non Static featured themes is a good idea, and must be implemented. At least to experiment, to see how it goes.

    • Ulrich 9:57 pm on April 18, 2014 Permalink | Log in to Reply

      I like the first suggestion. The question that arrises, is how will the algorithm work?

      It would be nice to have a page with the top reviewers with a time filer to see for all time and for the last month. Something like the about page for core sounds good.

    • acosmin 10:01 pm on April 18, 2014 Permalink | Log in to Reply

      First two suggestions are quite awesome. It was starting to get frustrating to put a lot of time in a theme and knowing it will not get featured even a single day.

    • tskk 10:02 pm on April 18, 2014 Permalink | Log in to Reply

      #2, human input will enhance the algo results.

    • Matt Beall 10:14 pm on April 18, 2014 Permalink | Log in to Reply

      My vote is for #1 or #2, but two questions come to mind.

      First, in general, how would the algorithm work? We shouldn’t publicize the details of the algorithm, otherwise it may be able to be manipulated, but I’m curious how it would work in general.

      Secondly, how do we encourage involvement in reviewing themes? The incentive was a great way to get people involved, but it also only rewarded the top contributors, and didn’t recognize all the others. So how do we incentivize getting involved for everyone?

      One last thought: we don’t want reviewers to review their own theme, but reviewing a theme is the best way for a theme developer to understand the review process. Maybe instead of, or in addition to, an incentive program, we require or at least strongly recommend that theme developers review x theme(s) before submitting their own for review.

    • CyberChimps 10:15 pm on April 18, 2014 Permalink | Log in to Reply

      I’m glad something is being done on this front, at CyberChimps we felt the review contest was extremely unfair and biased. Our contributions these past few months felt more like extortion then they did volunteering.

      On the other hand, some consideration needs to be put in place as to which themes are featured.

      While the review process is designed to make sure all the themes on the repo are quality themes, the reality is a lot of them are not maintained, and the theme authors can’t support themes with as much traffic as a featured theme receives. Even at CyberChimps we’re struggling to maintain Responsive which is by far the most popular theme on the repo, and has been for over a year.

      We’re also not opposed to version 2.0 being considered as a default theme, but at very least we’d like to keep our work under our name if that is at all possible. @Jen please reach out to trent at cyberchimps dot com if this is something worth discussing with Matt.

      With all of this said, I’m fine with an algorithm, but the themes it should pool should be curated by a human.

      In other words, instead of the algorithm grabbing from all themes on the repo, it should only pool from a list of maybe say the top 100 themes on WordPress.org to ensure the themes are regularly updated and maintained.

      There needs to be a reliable theme author behind the featured theme to make sure it is being maintained and supported.

      • CyberChimps 10:51 pm on April 18, 2014 Permalink | Log in to Reply

        I’m fine with an algorithm, but the theme pool should be curated by a human.*

      • Chip Bennett 10:56 pm on April 18, 2014 Permalink | Log in to Reply

        Extortion?

        I would think that the 48 New Themes that are approved and Live in the directory as a result of @cyberchimpscode reviews in the past 90 days would be a point of pride – a reward in and of itself.

        That, after all, is the point of a community contributor group, isn’t it?

        As for the fairness to reviewers of the review incentive program: in those 90 days, 288 New Themes were made live, and 10 incentive winner “slots” were named over three months. 2 of those (20%) were @cyberchimps code, who completed 17% of the reviews of New Themes made live during that period. I’d say that the program – which was intended from the beginning to be a fun way to encourage more participation, rather than a serious contest with rigidly defined rules – accomplished its objective of fairness.

        Personally, I’m glad to see the program changing. I’ve wasted more time managing a program that ended up getting taken way too seriously, which detracted from my time available for actual TRT admin duties.

        • Jen Mylo 11:04 pm on April 18, 2014 Permalink | Log in to Reply

          That, after all, is the point of a community contributor group, isn’t it?

          Yes, it is, flat out. Anyone expecting any form of reward or compensation other than feeling good, knowing you’re helping the project and that your volunteer efforts are helping millions of people, and recognition by your peers in the project, has expectations that are not in line with project values. For wp-based businesses, it’s considered an investment in the platform upon which the business is based (instead of, say, license fees ike proprietary platforms would charge).

          Ask not what WordPress can do for you, but what you can do for WordPress. :)

        • CyberChimps 11:07 pm on April 18, 2014 Permalink | Log in to Reply

          Numbers do not justify the cost and harm to the community.

          I’ve never seen pay-to-play rules in the WordPress community before, and I had never seen contributions being punished before either.

          We use to contribute reviews to contribute to the community, not for some reward of being featured.

          Due to the review contest we were forced to pull our developers away from doing what they do best, which is developing themes, and instead were forced to do theme reviews to keep our themes relevant. That is why it felt like extortion.

          To make matters worse our reviews were being disqualified from the contest if we made even a minor mistake for rules that are mostly subjective and mostly a matter of opinion.

          I don’t really care how many reviews were conducted or what the numbers are, contributions shouldn’t be forced.

          • Jen Mylo 11:11 pm on April 18, 2014 Permalink | Log in to Reply

            You’ve never seen it because it gets cut off before it gets that way, as other areas have much tighter oversight by project leaders/Matt. I have told MANY potential contributors thanks but no thanks when they offered to give us something awesome (code, video content, docs, etc) but wanted proprietary credit. We tend not to dwell on the bad things, and promote the good.

          • tskk 11:13 pm on April 18, 2014 Permalink | Log in to Reply

            Nobody forced anyone to compete.

          • Jen Mylo 11:13 pm on April 18, 2014 Permalink | Log in to Reply

            No contribution is forced. Period. If you were doing it just for the possibility of getting to promote your own theme on the Featured Themes page, then that’s pretty much the definition of pay for play. Anyone who isn’t here for love of the project/the work is absolutely encouraged to spend their volunteer hours with a cause that will give them those good feelings. No one should be building a business model around the site pages of wordpress.org.

            Further, I specifically asked for comments to focus on the discussion of what to do with the program moving forward, not to rehash old complaints, so please stick to that. Thanks.

          • Chip Bennett 11:31 pm on April 18, 2014 Permalink | Log in to Reply

            I would like to address this point, because it is important for moving forward:

            Numbers do not justify the cost and harm to the community.

            In the eight months since the inception of the program, 691 New Themes have been approved and made live in the Theme Directory. That equates to 1,036 Themes in a 12-month period. If you recall from Matt’s last State of the Word address, he mentioned that 300-some New Themes had been added to the directory.

            The review incentive program contributed to four times as many New Themes being added to the Directory. And that doesn’t even get into the considerable reduction in turn-around time for Themes to be reviewed (ah, yes: the glory days of 60-day waits in the review queue). Nor does it address the almost three thousand Updates to approved Themes that were reviewed and approved during that same, 8-month time period.

            That also doesn’t address the many developer comments I saw in the Trac tickets, thanking the reviewers for helping them through the review process, expressing appreciation for how helpful their reviewer was, and saying that they learned a lot through the process. (That’s a remarkable change from what used to be the typical response: frustration and resignation.)

            Those, far from being harm to the community, are measurable, demonstrable benefits to the community.

            It is that benefit that I want to ensure we maintain, in whatever changes we make to the review incentive program. To be honest, I’m not especially confident, given the amount of energy that was expended into the rules of the incentive program, rather than in working to improve the quality of reviews, and finding ways to work together as a team.

            So, I don’t plan to have much input on the discussion here. I’d like to see what the whole team has to say, and what we can come up with, as a consensus, for how to move forward.

      • Jen Mylo 11:06 pm on April 18, 2014 Permalink | Log in to Reply

        Any interest in contributing to a default theme should be directed to Matt. That said, no, we would not keep anything in a company name. That’s not really how contributing code works here. When something is contributed, it gets hammered on and changed by a bunch of people, and the default themes all carry worrdpressdotorg as the author, even if only one person wrote and designed most of it.

        • CyberChimps 11:17 pm on April 18, 2014 Permalink | Log in to Reply

          Understood.

          We’d just hate to give up the name Responsive, but perhaps a better alternative would be to contribute our 1.x code base, and then still release 2.0 under CyberChimps. Best of both worlds that way.

          Will think on this.

          • Jen Mylo 11:19 pm on April 18, 2014 Permalink | Log in to Reply

            That’s always an option. The themes that went into some previous default themes live on as separate entities.

    • arpowers 10:17 pm on April 18, 2014 Permalink | Log in to Reply

      Great, to hear you guys have figured this out!

      It seems to me that ‘featuring’ has a specific role in a marketplace (and economics).

      Specifically ‘featuring’ helps prevent good products from getting buried from products already on the leaderboard. For example, this is how Apple views featured Apps on the App Store for example. It’s all about visualizing things and giving good products a chance.

      TL;DR – If you guys want to do it correctly, then featuring is somewhat of a soft art, but it has a clear function. Admins should select featured themes monthly, objectively, based on an application and peer review. You can’t fight economics.

    • eminozlem 10:27 pm on April 18, 2014 Permalink | Log in to Reply

      Couldn’t agree more. It seems for reviewers, promoting their own theme is a huge motive to make more reviews. I too don’t like the section to be occupied by biased themes and same themes.
      So, the best would be a hybrid solution ie; If there are 10 featured themes; make 6 of them a random selection, 4 of them reviewer’s choice, like it’s right now. That could make everyone happy.

      As for the algorithm, I think everyone should be taken into account.
      1- Download count. (Should be calculated together with time. Otherwise older themes would gain advantage)
      2- Rating. ( Should be calculated together with number of reviews, Otherwise 5* theme with only 3 reviews would win over one with 4.8 rating with 155 reviews)

    • tskk 10:55 pm on April 18, 2014 Permalink | Log in to Reply

      This is off topic, since the decision is already made to scrap the program, but did you guys have any plan to deal with the issue of 2 months waiting time or will that be something theme authors have to live with?

    • Bryan Hadaway 12:06 am on April 19, 2014 Permalink | Log in to Reply

      Let’s look at the program in a purely objective way.

      What’s the point of the program? To get more reviews done, faster and ultimately get more themes approved. Why is that important? Because, the more themes and plugins there are available to the end-user, the more popular WordPress becomes. Why is that important? Because WordPress is a product of Automattic, a business, like any other. The more popular the brand/trademark is, the more people use it, the more people use it, the more money it makes, through affiliated hosting referrals, through .com services etc.

      WordPress.org is an upsell to WordPress.com (regardless of any idealism held by WordPress being open source software under the WordPress Foundation, which is great, or even regardless of intention, that’s still the reality of it), just like a lot of our free themes are upsells to premium versions.

      That’s the objective bottom line of any business, understandably and there’s nothing wrong with that.

      The program works, period. All of these things are being accomplished.

      However, subjectively, people simply don’t like the program, regardless of the fact that it works. That to me, seems like the problem that needs to be addressed.

      It’s an Incentive Program, again, that’s INCENTIVE Program.

      Basically, I see only 2 options being presented to us:

      1. Retire the incentive program.
      2. Remove all the incentives, but keep the program. That doesn’t make a whole lot of sense, how is that different than removing the incentive program? If there’s no additional incentive, then the program simply dissolves.

      There are a lot of unpaid volunteers, theme developers, plugin developers, reviewers, admins etc that keep the project alive and popular. Yes, it is our choice to volunteer our time to help out, but let’s not demonize receiving incentives in return for our effort like it’s some dirty, unethical thing.

      There’s no such thing as a purely selfless act, so please, let’s stop pretending and stop spinning the desire for gain with a negative connotation.

      We all gain something from the project (ALL of us), for one or all of these reasons:

      • It’s fun to volunteer
      • It makes us feel good to help out
      • It’s fun to build something that other people actually use
      • It’s a good way to stay sharp and keep informed about web design trends and best practices
      • We use WordPress for client projects (and our own projects)
      • We make products for WordPress for monetary gain

      Matt is a millionaire because of WordPress after all, we’re not really going to continue to spin incentive as a negative are we?

      I personally use/have used WordPress for all these things, I love WordPress and I receive incentives daily from contributing to the project. These are all good things.

      One hand washes the other. That’s the true spirit of WordPress and really, any community in general. We wouldn’t do it if it served no benefit to us. It’s called being human.

      I vote that we keep the program, which must include logical incentives or there’s no program at all. If Matt thinks volunteers receiving incentive is wrong, of course the program will and should be removed, but I think if we could address what about its current implementation people find unfair, that would be the most constructive and productive approach.

      Here are my thoughts for an actual solution for the main Theme page. Keep the program exactly the way it is now (with maybe a few minor tweaks), only change the design of wordpress.org/themes to have 5 even columns/blocks of listed themes, in this order:

      • Featured
      • Reviewer Recommended
      • Popular
      • New
      • Updated

      Featured will be admin picked and “Reviewer Recommended” or just “Recommended” will be what the incentive program is for. Yes, reviewers should still be able to pick their own theme, that’s what drives competition (another word that’s not a bad word) and makes the program work in the first place.

      Fairness. I don’t see why there’s even a fairness problem to begin with. Anyone that can submit their own theme and get it approved can also review other themes. It’s entirely someone’s choice how much they want to contribute. Those who choose to contribute the most should naturally receive the most incentive. That’s the way the world works.

      For the record: This comment is not made in a rude or accusatory or off-topic or angry manner. It’s just an honest and calm construction of my thoughts on the issue.

      Thank you.

      • tskk 12:23 am on April 19, 2014 Permalink | Log in to Reply

        If there is a vote, then i would vote to keep the program at least the earlier version.

        Removing the incentive of not selecting our own theme will bring back the problem of 2 month wait time. Volunteer reviewers can only do a little, 2 month wait time is after the volunteer effort.

        Only other viable alternative is to hire full time reviewers.

        • Chip Bennett 1:24 am on April 19, 2014 Permalink | Log in to Reply

          If people stop contributing because the “incentive” goes away, then the entire Review Incentive program was a failure from the start. Regardless of how many more Themes were approved, and how much the review queue wait-time was solved, if that all came about only because of the pay-to-play aspect of the program, then we failed.

          • Bryan Hadaway 1:27 am on April 19, 2014 Permalink | Log in to Reply

            I don’t think so.

            The problem isn’t really an objective one, but a subjective one.

            2 + 2 always equals 4 even if not everyone agrees on the method 4 was reached. That’s really what we’re talking about.

            The program works.

            • Chip Bennett 1:33 am on April 19, 2014 Permalink

              I’m not a believer that the end justifies the means. The Theme Review Team is a community contributor group. It exists to give people an opportunity to contribute to the WordPress project. There are many benefits inherent in participation in the Theme Review Team: become a better Theme developer by learning more best practices, help other developers get their Themes approved and listed in the Theme Directory, and help ensure end users have a wider variety of the highest quality Themes they can find anywhere.

              Those are the reasons the Theme Review Team exists. If the Review Incentive program caused the team to lose sight of its raison d’etre, then yes: the program failed, and needs to be replaced with something that similarly advances those objectives without turning the Theme Review Team into a pay-for-play operation.

            • Bryan Hadaway 1:40 am on April 19, 2014 Permalink

              @Chip – Agreed, I was here before the incentive program and I’ll be here after, but I did review more with an incentive because I could justify the time spent more. I think that’s the ideal.

              I think a simple pros and cons list would still prove the program useful.

          • tskk 1:30 am on April 19, 2014 Permalink | Log in to Reply

            I think it all came from only that aspect, but i will be happy if proven wrong.

            • tskk 1:34 am on April 19, 2014 Permalink

              correction : not all, but most of it.
              A few reviewers were there before the program and continue to do so and will continue to do so.

            • Bryan Hadaway 1:37 am on April 19, 2014 Permalink

              @tssk – I was one of those reviewers that was there before the incentive and will be there when it’s gone. I’m not going to lie though, I reviewed way more with an incentive than without because it allowed me to justify allocating my time away from other things I was working on.

      • Jen Mylo 12:29 am on April 19, 2014 Permalink | Log in to Reply

        WordPress.com is a product of Automattic, but WordPress is not. Have you checked out .com lately? They have a vastly different user experience, and lots of developers that do not have anything to do with .org.

        I also think you have it backwards… if anything, we generally pitch .com as the entry-level freebie, with self-hosted/.org being the kind of site you ‘graduate’ to when you need more complex stuff.

        • Bryan Hadaway 1:23 am on April 19, 2014 Permalink | Log in to Reply

          “WordPress.com is a product of Automattic, but WordPress is not.”

          Yes, I know that. But, that’s only a legal technicality.

          “if anything, we generally pitch .com as the entry-level freebie, with self-hosted/.org being the kind of site you ‘graduate’ to when you need more complex stuff”

          I also understand this.

          Both of your points I already expanded on and rebutted in my original comment, because I had already predicted someone would say .com is a “product” of Automattic and .org is a “project” of the WordPress Foundation. We’re talking about words, not function.

          I’m referring to the reality of brand building as seen from the consumer’s eyes and the bottom line dollar reality, not what’s written on paper. The point I’m making is that every little thing volunteers do to contribute to WordPress.org, builds up the name of the “WordPress” trademark overall. Without the popularity of .org, .com wouldn’t make as much money. Keeping in mind that .org also directly makes money via wordpress.org/hosting. But that’s of course chump change in comparison.

          Again, it doesn’t matter what the legal technicalities or the intended ideals are, the reality remains unchanged. Matt and other paid employees gain direct (not indirect like “feeling good”) incentive from our efforts, why is it a problem for us to receive direct incentive in return? Again, one hand washes the other. Community is about give and take, not hierarchy. In that case, we really are talking about volunteers being employees, admins being managers and Matt being our boss.

          Of course, I don’t expect Chip or Emil to all of sudden get paid for their hard work or any volunteer for that matter. We are here by choice. But, to be able to feature a theme (that in and of itself is a contribution) for our contribution efforts of reviewing themes for others seems like a very mildly generous incentive, not something to feel bad about or frown upon. That’s pretty disheartening to say the least.

          Again, that’s great that Matt and company is so successful, I’m casting no negative shadow on that whatsoever. What I’m talking about is the double standard that’s been presented here.

          But, we can get deeper and deeper into the philosophical state of the issue, and we’ll probably just rest that portion on disagreeing and move onto the next…

          Do you have any thoughts on my proposed solution? I don’t think it would be too difficult to implement and maintain and I feel like it kills two birds with one stone.

      • Sami Keijonen 7:36 am on April 19, 2014 Permalink | Log in to Reply

        Very good points Bryan, I pretty much agree all of them. And your our solution seems best so far. I’d like to add some kind of random theme section. Also default Twenty themes should not be featured all the time. They should be treated like any other theme.

        Community is a two way street. It’s not just about giving, it’s also about getting.

      • Zulfikar Nore 11:17 am on April 19, 2014 Permalink | Log in to Reply

        +1 to “Let’s look at the program in a purely objective way.” and everything else Bryan said!

    • Carolina Nymark 12:45 am on April 19, 2014 Permalink | Log in to Reply

      I dont mind nr1 or 2, but I wish there was a way to make sure theme authors are actually ready to submit a theme. I mean the que will go back to two months because there are so many themes that goes into the “not approved” bin and you put time and effort into a review and the authors don’t bother to try and fix it. The incentive should be the authors not the reviewers.

    • Josh Pollock 1:34 am on April 19, 2014 Permalink | Log in to Reply

      This isn’t a vote, or an opinion about what to do, it’s just a comment:

      I actually stopped reviewing themes soon after the incentive program started.

      I started reviewing themes, like a lot of people, when I looked up how to submit my own theme, and saw that it involved a 6-8 week wait and thought “that sucks, but this looks like a good way to contribute to WordPress.” So I started reviewing themes and learned a lot from doing it. This was the first thing I did to contribute to the WordPress community.

      Then the incentive program came along and at if there were ever tickets available, they were always “problem” tickets. So it became way less fun and I stopped. I didn’t feel bad as the queue wasn’t 6 weeks long anymore, which is great.

      Since then I’ve started contributing to the community in many other ways. I’m not upset, I like the fact that themes get reviewed quickly and I’m just saying that before theme reviews was a competition, it was a gateway to becoming an active contributor to and participant in the community.

      • Chip Bennett 1:40 am on April 19, 2014 Permalink | Log in to Reply

        Your experience is a big part of why I knew the program was in trouble when I had to post about not hogging or selectively assigning tickets. Despite all of the good that resulted, it turned Theme Review into a competition. I would rather see 100 people review and approve one new Theme a month, than have a dozen people fight over a competition.

        • Jen Mylo 1:59 am on April 19, 2014 Permalink | Log in to Reply

          One of the other infrastructure improvements I’d like to propose if we have bandwidth is a ticket assignment system that works like a customer service call center, where it just assigns the next theme in the queue to whoever’s next (or whoever is next with the right qualifications) so that cherry picking isn’t just discouraged it becomes pretty much impossible.

          • Zulfikar Nore 11:20 am on April 19, 2014 Permalink | Log in to Reply

            “or whoever is next with the right qualifications”

            What would be the “right” qualification and how would that be determined?

            • Chip Bennett 11:40 am on April 19, 2014 Permalink

              I think Jen is referring to the “reviewers” group in Theme-Trac – the group that has ticket-assigning and ticket-closing privileges.

    • Drew Strojny 2:12 am on April 19, 2014 Permalink | Log in to Reply

      Observing this from a distance it certainly looks like paid compensation for reviews. Almost all the winners are featuring for profit themes. If you’re selling freemium themes, being featured in the theme directory is like rocket fuel for that business model. The “cost” of reviewing themes is incredibly cheap for the real dollar return you’ll get on that work. If you were selling those featured spots they’d probably go for a few thousand dollars a month or more.

      When you receive something of significant dollar value for work performed, it’s called compensation, even if it isn’t real dollar bills in your pocket. I think we’re just seeing basic economic realities play out here.

      My vote is for a completely randomized featured theme list. It’s simple and it takes the drama out of the whole thing. This will eventually turn the “popular” list into a meritocracy, which I’ve always felt it should be.

      There is no point in subjectively choosing featured themes unless the person or persons choosing the themes has absolutely zero incentive bias. They shouldn’t be selling themes themselves, affiliated with anyone selling themes, or doing freelance work for someone selling themes. This is hard to accomplish in such a small and tight knit community. Even just being friends with someone might make you more likely to feature their theme.

      • Jen Mylo 2:17 am on April 19, 2014 Permalink | Log in to Reply

        If comment likes were in Jetpack already, I would totally “like” this comment. :)

      • Bryan Hadaway 2:24 am on April 19, 2014 Permalink | Log in to Reply

        You have some good points here, but you’re also not offering a solution.

        The incentive program was put in place because theme reviews were absolutely stagnant. It was taking months for theme developers to get their themes approved, that is, if they didn’t give up altogether first.

        The trac (https://themes.trac.wordpress.org/) is an absolute graveyard of abandoned themes.

        I think those of us who actually review themes, as well as submitting our own themes have a much better insight into this issue. A lot of you are only looking at one side of the issue because you really don’t understand the whole picture, first-hand.

        • Drew Strojny 2:46 am on April 19, 2014 Permalink | Log in to Reply

          In my opinion the solution is to scale back the theme review standards so it isn’t such a colossal project to get a theme approved. This makes the reviewers job easier and the whole process less intimidating.

          It’s the same balance government has to strike when they pass laws. If you add too much regulation, it ends up stifling innovation. You need to keep tweaking until you find the right balance. At the moment, WordPress.org themes feel overregulated to me. I think the recent changes that Chip enacted are a step in the right direction.

          • Bryan Hadaway 3:12 am on April 19, 2014 Permalink | Log in to Reply

            You’re right. I think that’s a very good solution, probably gets more to the root issue than anything else we’ve discussed so far.

            Honestly, the only mandatory checks should be for security, spam and bug issues. Any subjective design and function reviews we could probably do without.

            But then again, we’re probably just opening a whole new can of worms because naturally, there’s likely some who think we should only be more strict in our filtering of themes and what they’re allowed to do, not less.

            With that, I guess we’ll see what those in charge decide.

            • Chip Bennett 11:34 am on April 19, 2014 Permalink

              There are no “subjective design and function” Guidelines. We even down-rated the entire Theme Unit Tests to recommended.

              Security issues are where most (especially, new) reviewers struggle the most. Security issues generally derive from Theme Settings, which are the one aspect of Themes that have the highest learning curve. So, eliminating everything but security issues really won’t help the review process at all.

              And I disagree that the other aspects of the Theme Review Guidelines aren’t important. Ensuring that the Theme uses proper hooks, enqueues scripts and stylesheets properly, separates presentation from functionality, uses proper core functions, and maintains a consistent UX are all important. Directory-hosted Themes should play well with core and with Plugins, and should provide as consistent user experience as possible from Theme to Theme. These may seem like unimportant issues to some Theme developers, but they are certainly important to end users, to core, and to WPORG.

              But wholesale changes to the Guidelines really are a separate discussion, that would only sidetrack this one.

          • Ulrich 6:45 am on April 19, 2014 Permalink | Log in to Reply

            Only have security checks part of the review process would make the reviewers job easier but I don’t think this the best solution.

            The problem that I find is that a number of themes that are submitted are half baked or are just submitted so that they have a theme on the repository and somehow is free marketing for their company. These are the themes that you spend the most time and normally the theme author has put no effort to read the guidelines.

            I would love to see lower number of themes but more quality themes.

            Perhaps even require theme developers to review themes before submitting them.

          • Frank Klein 8:50 am on April 19, 2014 Permalink | Log in to Reply

            In my opinion the solution is to scale back the theme review standards so it isn’t such a colossal project to get a theme approved.

            I disagree with this, because it values quantity over quality. In my opinion, the problem with WordPress themes isn’t that there aren’t enough of them. There are just not enough good ones.

            We can’t review for design, but we can review for code quality and observance of best practices.

            I think that the biggest problems right now for theme reviews is that the process is entirely manual. You have to open the theme files, verify a whole laundry list of things and type up a report with all the issues you find.

            It also requires the reviewer to be an experienced developer and often to dig through code that may not be easy to read because it lacks things such as basic formatting.

            It’s very time consuming and repetitive, which is why it isn’t fun. It also wastes human brain power on things that machines could detect easily.

            So instead of lowering standards and thus reducing the quality of themes in the repository, we should look at our review process, note the things that could be automated and write tests for it in the Theme Check Plugin.

            Because if we invest the time to write a check that saves a minute for every review, we end up saving hours of reviewers’ time due to the volume of submissions.

            AFAIK, the Theme Check Plugin is not on Github, which would be a first step towards accepting community contributions.

      • Philip Arthur Moore 3:24 am on April 19, 2014 Permalink | Log in to Reply

        Spot on.

      • Frank Klein 9:05 am on April 19, 2014 Permalink | Log in to Reply

        My vote is for a completely randomized featured theme list. It’s simple and it takes the drama out of the whole thing. This will eventually turn the “popular” list into a meritocracy, which I’ve always felt it should be.

        Absolutely. I think that the proposed solution #1 is the best way to approach this. The algorithm doesn’t have to be complicated.

        The only two things that I would check for in this code is to make sure that themes that weren’t updated in a while don’t make the featured section and keep a list of the already featured themes during a certain time period (6 months? A year?) so that a theme isn’t featured too often.

    • Konstantin Kovshenin 7:31 am on April 19, 2014 Permalink | Log in to Reply

      I think #1 is the best approach.

    • Stanko Metodiev 9:18 am on April 19, 2014 Permalink | Log in to Reply

      I like the general idea of this initiative, but different people understands contribution differently. I like #1, but it would be great to have place where top reviewers could list their themes

    • nikeo 9:30 am on April 19, 2014 Permalink | Log in to Reply

      Let’s have a quick look at the different actors of the game and their “interests” here :

      • wp.org => featuring the best of what you can do with WordPress. This contributes to keep its position of first open source CMS in the world. This of course benefits to the WP.com business=> brings a huge popularity, talented developers, ideas, etc…
      • users => finding awesome free WordPress themes and beeing sure that they work fine and contain no malicious code
      • theme developers => beeing able to propose their work and ideas whithout waiting for weeks, having the possibility to support their theme’s users, build a freemium business
      • theme reviewers => contributing to a great project. But since this review activity is the key of success for everyone, it seems that some kind of reward/incentive is normal.

      Like @chipbennett highlighted it, looking at the theme review stats, the incentive program has brought really great results. Both from a user perspective => more great themes beeing available in the rep and from a developers perspective => faster reviews of course.
      To me, abandon this program and all the efforts that have been put into would not be optimal then.

      That says and following @jenmylo and @matt ideas, I think that an adapted solution #2 would be the best :
      Use an algorithm to select featured themes and keep the incentive program.
      How? The featured themes list would be a monthly merged list curated both by the TRT admins and the other half by the top reviewers of the month.
      Let’s say we keep the idea of 10 featured themes. The theme review admins would choose 10 themes and the reviewers would choose another 10. The algorithm would then randomly grab 10 themes among this pool of 20.
      With this solution, reviewers should still be able to choose their own themes and would have a statistically significant chance to see their theme featured. All other actors : wp.org, users and developers, could also benefit from it.

      Thanks for opening this discussion!

    • Greg Priday 9:44 am on April 19, 2014 Permalink | Log in to Reply

      I’d like to see #1 for the featured themes list. A weighted randomized list of themes that takes into account downloads, rating and newness. We could also allow admins or other trusted community members to boost themes they like so they generally land up higher on the featured themes list.

      This way we could keep the incentive program if we wanted to (it did solve the 2 month queue problem). We’d just allow the winners to give their theme a small, month-long boost in the algorithm. Maybe something that would translate to ~50 extra downloads per day.

      At the moment, incentive program winners are getting 1000+ additional downloads/day on their themes. I wouldn’t be surprised if this worked out to thousands or tens of thousands of dollars in monthly sales. That’s far too much of a bonus for what should be a fun little program.

    • Zulfikar Nore 11:33 am on April 19, 2014 Permalink | Log in to Reply

      If a decision to abandon the incentive program has been made then #1 is the viable alternative.

      The only reason the program should be abandoned thought is because it is no longer what it was meant to be and we should not loose sight at what the program actually achieved, cut down the waiting Que :) – to me that is in no way a “failure”.

      Just my 2 cents :)

      • Schwarttzy 11:47 am on April 19, 2014 Permalink | Log in to Reply

        It has serious problems, when themes are being check out the second they get uploaded. Means the reward is way too high.

    • Schwarttzy 11:42 am on April 19, 2014 Permalink | Log in to Reply

      I’m so up for changes! I’m sick of these people blasting their theme month after month.

    • tskk 11:50 am on April 19, 2014 Permalink | Log in to Reply

      Out of the 10,
      2 for twenty somethings
      4 for best themes feature wise, decided by TRT admins/reviewers, these can be unseated only by themes with even better features.
      4 themes from last month’s new themes voted by TRT admins and reviewers who reviewed at least 5 tickets last month.

    • ThinkUpThemes 12:26 pm on April 19, 2014 Permalink | Log in to Reply

      Removing 100% a program that has contributed so significantly to improving the theme review process is highly illogical and is a major step back. I struggle to see any sense in entirely removing a program which will result in the theme review process becoming a lengthy and cumbersome process. Surely an objective of any operation, whether it be for profit or not for profit, is to deliver a service in a streamlined and efficient manner? The review incentive program helped achieve this.

      Also considering @Chip comment “a remarkable change from what used to be the typical response: frustration and resignation” I would hate to see WordPress be viewed in this light again, particularly by potential .org theme authors, who are essential in ensuring the continued growth and future success of WordPress. Theme developers will simply go elsewhere, to other platforms or even marketplaces. We might brush this off and say “let them go”, however this would be a terrible attitude as for me it will indicate the start of the fall for WordPress.

      I personally think that competition is great, it drives developers and others to push their themes to be even better. As surely if they have an opportunity to become featured, they are more likely to bring to market a product that looks great, works fantastic and ultimately represents WordPress in a positive light. Removing the incentive program 100% will remove the friendly competition that’s currently in place and will inevitably result in reviewers going elsewhere. However if change is required, then lets approach the issue and seek a resolution in a manner that aims to satisfy all parties.

      I should stress that I don’t believe reviewers will actively seek to abandon their involvement with the review team, however this will happen due to the realities of life. If a reviewer is able to identify an “opportunity” elsewhere for them to gain exposure for their theme then it’s likely they will pursue the opportunity.

      Any incentive program that does not allow a reviewer an opportunity to benefit, I’m sad to sad, is not going to work. No matter how much we all believe it will. In this case I agree with many above that #1 is the most viable option.

      • ThinkUpThemes 12:27 pm on April 19, 2014 Permalink | Log in to Reply

        Just a note, change is always a good thing. So I’m excited to see what’s next in store for the review process. :-)

    • Justin Tadlock 3:38 pm on April 19, 2014 Permalink | Log in to Reply

      I originally thought the incentive program might be a good thing because the winners would choose themes that they actually reviewed during the month to be featured. Their reward would be to choose a theme that they found interesting, thought was good, etc. That’s the only way I would really like to see incentive program stick around.

      I’ve seen a number of good themes come through since the program started that have no chance of being featured though, and that’s something I’d like to see changed.

      But, we have to look at why the incentive program was introduced in the first place to get at the root of the problem. That problem is that we have way too many themes coming in and not enough reviewers/time to do full-scale reviews, which creates about a 2-month long wait time before a submitted theme is ever looked at. Then, it could be another month or so before the theme is ever approved (we’ve taken major steps in improving this second wait time with how we now do reviews).

      Therefore, we need to look more closely at why it takes so long to get these reviews done. I don’t believe this is a secondary topic or unrelated. All of these things are intrinsically tied together. What we’ve done over the past few years is create a review system that introduces a high barrier to entry for theme authors and makes for long and arduous reviews for reviewers. There are a lot of good reasons for our guidelines, many of which I’ve taken part in creating. However, this is probably an area we need to take a long, hard look at because it is what has led to the situation we currently have.

      Something has got to give somewhere because I don’t think any of us want to go back to the days where we had those 2-month long wait times. Like it or not, the review incentive program fixed that. While changing how themes are featured is good, we need a way to fix the problem if we drop the incentive program. We can’t fix one problem without addressing the other.

      On the specific subject of featuring themes, here’s what I’d like to see:

      1) All reviewers should get an opportunity to propose a theme (not their own) to be featured. At the end of the month, the admins choose from any themes proposed by the reviewers.

      2) The algorithm approach. Of themes not specifically chosen by the theme review team, they should be auto-rotated.

      3) Themes that are chosen as featured should not be featured for more than a month nor should they be chosen again within a specific time-frame (3 months, 6 months, year, etc.).

    • ThinkUpThemes 3:58 pm on April 19, 2014 Permalink | Log in to Reply

      I like points 2 & 3, however there are a number of issues introduced with point 1. A number of theme reviewers, especially those heavily involved during the incentive program run their own theme shops, many of which are small businesses. Do you truly believe that someone who runs a small business will (or can afford to) spend the amount of time they’re currently doing so to review themes without even the potential of benefiting their own business?

      It sounds great, however lets all be honest. If we all truly took actions solely out of the kindness of our own hearts to help others without the need for recognition or benefit, then we’d all be volunteering at our local hospices. This is simply the reality of it. So I really do hope that when arriving at a solution we are realistic in our expectations on how it will impact the review process.

      I’m fairly new to WordPress and honestly am truly grateful for this whole project. The reality is that reviewers can still build successful theme shops without ever being featured (e.g. Elegant Themes, StudioPress, and the many MANY theme authors on ThemeForest).

      Please lets not alienate our theme reviewers and punish them for wanting to benefit their own situations whilst working to developing WordPress. If we do this then the experienced reviewers (who I would assume it’s safe to say are also experienced developers) will simply go elsewhere. Then we’ll be left with a situation where the majority of the review team consists of those that are new to WordPress reviews, and therefore less experienced, and possibly stick around for less time as they do not even have a prospect of benefiting. Inevitably those that will suffer are the admin, prospective .org theme authors and eventually the WordPress users that are left with low quality themes which are rarely updated and developed by inexperienced developers.

      I say we return to the old program where only 3 reviewers can get featured. There’s no need to completely scrap something that is working and greatly benefiting the community.

      • Chip Bennett 4:09 pm on April 19, 2014 Permalink | Log in to Reply

        I just did a quick count; we have about 150 people with “reviewer” privileges. We’re currently approving just under 100 New Themes per month. If every person with “reviewer” privileges reviewed one New Theme every two months, we would be able to keep up with the review queue. (That’s not counting Theme Updates, of course; but those are much simpler, diff-reviews.)

        We say that we want quality-over-quantity for approved Themes; I think that we have reached a critical mass of reviewers that we can set a goal of quality-over-quantity for Theme reviews now, as well.

      • Jen Mylo 4:09 pm on April 19, 2014 Permalink | Log in to Reply

        Do you truly believe that someone who runs a small business will (or can afford to) spend the amount of time they’re currently doing so to review themes without even the potential of benefiting their own business?

        That’s how contributing to WordPress works. Some people won’t do it without getting paid or rewarded, and that’s okay.

        • ThinkUpThemes 4:15 pm on April 19, 2014 Permalink | Log in to Reply

          Hi Jen,

          I should stress that I’m a part of the group that will still continue at my current rate of contributing. I don’t do many reviews, but do as much as I can afford to. Lets just hope that every reviewer has this view. However, I do believe that we’ll have a drop off from reviewers that are running small businesses. Hopefully though this wont impact the review queue too much. Like I said I’m new, and grateful for everything everyone here is doing. :-)

          @chip. I didn’t know we had so many reviewers! in that case, maybe my predicted impact wont be as significant as I’d thought.

          • Jen Mylo 4:38 pm on April 19, 2014 Permalink | Log in to Reply

            Don’t second guess yourself. Removing the possibility of featuring your own theme will definitely mean some people don’t participate anymore, or at least at the same level. What I’m getting at is that that’s how the WordPress project works, and the loss is something we’ll deal with. Just like if someone has a job that pays for them to contribute to core loses that job and doesn’t want to put in as much time anymore because they ant to spend more time on paying work or other hobbies — it’s a built-in expectation in open source projects.

            • ThinkUpThemes 4:42 pm on April 19, 2014 Permalink

              “Don’t second guess yourself.”

              Not sure what you mean? :-)

          • Chip Bennett 12:09 am on April 20, 2014 Permalink | Log in to Reply

            I’m not sure I understand why those with commercial Theme shops are really any different from those without commercial Theme shops. We all have families, and day jobs, and other commitments and priorities. We are (or should be) all here because we want to contribute to the WordPress project and community. We do so because we prioritize that contribution for its own sake.

            Idealistic? Of course. But then, WordPress itself is idealistic.

      • Justin Tadlock 4:41 pm on April 19, 2014 Permalink | Log in to Reply

        No, I don’t believe all will continue reviewing as they do now and said as much above. I don’t think any of us believe that. The whole point of my post is that if we scrap the incentive program, we’re probably going to have to rethink other parts of the theme review process to be able to keep up.

    • Emil Uzelac 6:47 pm on April 19, 2014 Permalink | Log in to Reply

      Quick Background.

      At the beginning I was not for the program, but it kind of grew on me.

      First it was great seeing this entire group of people at one place, reducing the ticket number and approving them quickly.

      What a fantastic feeling, @chipbennett and I were simply relieved.

      Soon after that we started getting poor reviews, tickets re-opened and some complaints from Theme authors. We figured that this was only a temporarily but we were wrong.

      Reviewer approves the Theme and admin evaluates each review before the Theme is “marked” live. After the evaluation admin decides the scoop and “holds the right” to re-open the ticket if the bare minimum is not set.

      So if you have 100 tickets, admin goes through all and tries our best not to re-open them because that can create more work for author, reviewer and delay the overall approval process.

      The contribution turned into competition and competition became a “battle”, not only among the reviewers, but also toward the TRT (not a healthy “relationship” if you ask me).

      Please note that program was just a trial and because of that we don’t have to go over a lengthy discussion to decide weather we need to continue or discontinue.

    • Jose Castaneda 9:52 pm on April 19, 2014 Permalink | Log in to Reply

      This doesn’t bum me out. When it was first introduced I liked the idea of empowering the reviewers and letting us share some cool ways of creating themes. As Justin mentioned previously it took a side step and wound up in this path. I really like the first implementation and think that the second could be very feasible.

      When I first chose to start reviewing themes it was to give back to the community. If it benefits me that’s fine. If it can benefit everybody: even better.

  • Chip Bennett 5:14 pm on April 8, 2014 Permalink | Log in to leave a Comment
    Tags: , ,   

    Selective Ticket Assignment and Ticket Hogging 

    Based on recent comments, I would like to clarify a couple points about ticket assignment.

    Selective Ticket Assignment

    First, when you assign a ticket to yourself, assign the first ticket in the queue. That means assign from tickets in Priority #1 queue, before assigning tickets from Priority queues 2, 3, or 4. Also, and more importantly: assign the first-available ticket in the queue. Do not cherry-pick tickets based on ease of review, reporter (i.e. Developer), etc.

    Selectively assigning tickets is unfair to developers who are required to wait their turn in the review queue, and is a means of gaming the incentive program. Several reviewers have alleged that this practice takes place. We have not monitored it, but we will do so – something that will take even more time, and result in further delays in getting Themes approved and live. Anyone found to be selectively assigning tickets will risk being disqualified for the incentive program.

    Ticket Hogging

    Until now, we have not set limits on the number of open/assigned tickets a reviewer can have at any one time. Several reviewers have complained that often there are no tickets available, while some reviewers have several open/assigned tickets. So, going forward: reviewers may not have more than 5 open/assigned tickets at any one time.

    In order to ensure that reviewers don’t get stuck with developer-abandoned tickets, admins will close tickets older than one week with no activity or developer response. For this reason, it is even more important not to “bump” tickets by posting comments requesting status updates.

    Also, while this will be much more difficult to monitor: reviewers should only assign themselves one ticket at a time. Assign a ticket, conduct a full review, post comments, and only then assign another ticket.

    Also, this clarification isn’t intended to introduce delays in developers getting their Themes reviewed. Any ticket that appears in the Priority #3 queue (tickets older than 2 weeks) are fair game to anyone. (But if the priority queue system is followed properly – see above – there should never be any tickets in the Priority #3 queue.)

    Please discuss in the comments below.

     
    • Rohit Tripathi 5:30 pm on April 8, 2014 Permalink | Log in to Reply

      Although, I don’t disagree with any of the above, there is one thing which I would like to say in regard to Selective Assignment.

      When, I review a ticket from a Developer say XYZ. It took me an hour, to perform a complete review and point out the mistakes. Now, another theme from the same developer is towards the bottom of the queue. I am obviously inclined to pick it up, as I have already reviewed the code. And chances are that 90% of the code is same in 2 different themes from the same developer. This will save my time, and others’ too as they won’t have to put in the same amount of effort which I had to put in on the first theme of XYZ.

      Is this still an unhealthy practice?

      • Chip Bennett 5:34 pm on April 8, 2014 Permalink | Log in to Reply

        Yes, because all those tickets in between represent developers who have also patiently waited their turn to have their Theme reviewed.

        Also, it is healthy for more reviewers to be exposed to more Theme code from more Theme developers, because it helps make us all better at reviewing.

    • tskk 5:41 pm on April 8, 2014 Permalink | Log in to Reply

      If a ticket at top is way over my head, i have to wait till its picked up, tickets below have to wait unnecessarily.

      Tickets are flying off the list, there won’t be much unjust to if we skip the one’s over my head.

      • Chip Bennett 6:12 pm on April 8, 2014 Permalink | Log in to Reply

        If you always skip tickets for more difficult Themes, one, you force other developers to review the Themes that take more effort; and two, you never benefit from learning from those more difficult Themes. If you need help with a ticket, ask. Everyone on the mail-list should be willing to jump in and help another reviewer who needs help with a complex Theme (especially when tickets are getting assigned so quickly that we have none in the queue). We are a team, and the expectation is that we will act like a team, and help one another like a team.

      • Frank Klein 6:17 pm on April 8, 2014 Permalink | Log in to Reply

        I agree with Chip.

        If you go into the queue and look for small updates, you can do a huge number of tickets in a short amount of time. It’s simply not cool for the theme developers and for the other reviewers if you go in and only take the low hanging fruit, and leave the large and difficult to dissect reviews for others.

        At the same time, we should encurage developers to do smaller commits, instead of mashing together different features and bug fixes in one gigantic kitchen sink commit.

    • Emil Uzelac 5:43 pm on April 8, 2014 Permalink | Log in to Reply

      Additionally: (very much related)

      Current program is to help community and in return TRT features your Theme. This is not a race or competition.

      At the same time our program was not designed as a promotional tool from i.e. Theme Name to Theme Name Pro, but it has been used as such and we have no problems there.

    • robin90 5:48 pm on April 8, 2014 Permalink | Log in to Reply

      Thanks Chip for outlining all those. All the points that you have outlined above were already there and has been discussed many times except this one I think 5 open/assigned tickets. But reviewers seemed to neglect that and grab more tickets simultaneously. It’s sad that we have to outline each and every point. If reviewers would have been little more sensible than we wouldn’t have needed all this.

    • ThinkUpThemes 7:29 pm on April 8, 2014 Permalink | Log in to Reply

      Thanks Chip for this! I totally agree that limiting tickets to 5 per reviewer really helps to give everyone a better chance at getting hold of a ticket and getting that valuable learning experience gained from reviewing. Quick question, if a theme is set to “approved”. Does it still count towards the 5 limit until it’s either made live / reopened / etc… by an admin?

    • CyberChimps 8:23 pm on April 8, 2014 Permalink | Log in to Reply

      We are glad we’re addressing this now, but as long as the reopen ticket rule exists, the system can still easily be gamed.

      A blog post on this topic doesn’t negate the fact that this practice lead to several of this months winners winning the contest, when completely legitimate reviews were discredited for erroneous reasons.

      You guys rewarded those who cheated this month and punished legitimate reviews which disqualified two winners.

      A blog post pointing out that people cheated while still honoring those who cheated and punishing legit reviews doesn’t undue the damage caused. This isn’t a fair contest.

      • Chip Bennett 11:06 pm on April 8, 2014 Permalink | Log in to Reply

        We are glad we’re addressing this now, but as long as the reopen ticket rule exists, the system can still easily be gamed.

        How so? Unless you’re suggesting that the Admins are attempting to game the system?

        A blog post on this topic doesn’t negate the fact that this practice lead to several of this months winners winning the contest, when completely legitimate reviews were discredited for erroneous reasons.

        Untrue. And for the third time: if you have issues with tickets that you believe were unopened unfairly, the place to raise that issue is in the specific ticket.

        You guys rewarded those who cheated this month and punished legitimate reviews which disqualified two winners.

        Again untrue. There is no evidence of “cheating”, and no reviews were “punished”.

        A blog post pointing out that people cheated while still honoring those who cheated and punishing legit reviews doesn’t undue the damage caused.

        Still untrue. There is no evidence of “cheating”, and no reviews were “punished”.

        This isn’t a fair contest.

        If you think the contest isn’t fair, you’re welcome not to participate.

        • CyberChimps 12:47 am on April 9, 2014 Permalink | Log in to Reply

          If reviewers are selectively reviewing themes, and more complicated tickets are being left to the remaining reviewers the odds of the tickets being reopen is higher. Therefore the reviewers not selectively reviewing are at a greater risk of taking on tickets that may get reopened negating all of their review work. Which is exactly what just happened, and unfair.

          The tickets we had weren’t reopened unfairly, the rule of disqualifying those reviews is unfair.

          We’ve sent a list of the theme reviewers who were selectively reviewing to you personally to review as we would prefer not to present this data publicly, although it is clear upon reviewing Theme Trac who the offenders are.

          We consider disqualifying legitimate reviews as being punished because it lost us the contest this monthly unfairly, where as reviewers who selectively reviewed were rewarded and won the contest. This what we consider to be unfair.

          CyberChimps has been providing Theme Reviews for 3 years now long before the contest and will continue to contribute no matter how unfair the circumstances. We’ve sponsored many WordCamps, the Community Summit, and continue to do what we can to support the community. Our reputation in this community alone, as well as having the most popular theme on WordPress.org and in the world should be more then enough to back up our credibility on such matters.

          • tskk 6:58 am on April 9, 2014 Permalink | Log in to Reply

            So you are saying people should be punished for not following rules of future while you yourself are breaking the rules of the present? you have 13 open tickets right now and your employee’s are still assigning tickets to them self, while the current limit is 5. You should be disqualified from this month’s competition as you have more tickets than the limit.

            • CyberChimps 1:07 am on April 10, 2014 Permalink

              We took on those tickets before the new rule.

              We’re going to review all of those tickets, we could easily be doing 15-20 reviews a month. We have several new people being trained to do reviews.

              Like CyberChimps, you have been one of the few reviewers not selectively reviewing. Thank you for that.

          • tskk 9:20 am on April 9, 2014 Permalink | Log in to Reply

            I personally don’t think all these rules are unnecessary, program was working for months and certain company will complain no matter what, as long as they don’t get their theme featured.

            Not assigning next ticket without posting the full review is enough.

            All this is too much work for admins and will only result in program getting cancelled.

    • ThinkUpThemes 8:54 am on April 9, 2014 Permalink | Log in to Reply

      Hi All,

      I wanted to share some thought on the cap on open/allocated tickets and hear the thoughts of others. Although the need of a cap, of sorts, is a great idea. I worry that this new cap although will reduce the risk of ticket hogging (which affected reviewers), it will however introduce a new and greater risk of ticket churning (which will affect developers).

      Consider a scenario where a reviewer has 5 allocated tickets, all of which have been reviewed, and on which a developer response is required. Reviewers might:

      1. Rush a review to the approval stage simply to free up a slot to allocate a new review.
      2. Rush to disapprove a theme purely because it may take longer to review or require more support to be provided to the developer.

      In addition, inexperienced developers with a passion to learn may not get the support they need as reviewers simply may not be inclined to keep tickets open.

      Also the risk of selective reviews will increase, however the point at which the risk arises will simply shift to a later stage of the review cycle. Rather than being at the ticket allocation stage (which personally I don’t think happens) the selective review risk will shift to when a ticket is now allocated. Essentially reviewers will ‘cherry pick’ by choosing to decline complicated themes that may take a while to review in the hope that the next theme they choose will be more simple.

      Another worry is that the move of closing tickets after 1 week is a move to keep reviewers happy so that 1 of the valuable 5 allocations isn’t unnecessarily filled by an unresponsive developer. I feel a little uneasy about changes which shift the focus of the review process from benefiting developers to benefiting reviewers. The previous rule of admins following up on a ticket after 1 week, and them closing after 2 (if no response is received) is great for developers as it gives them a chance to take action. Disapproving tickets for lack of response after 1 week, especially if the developer hasn’t been warned, might lead to them bring disheartened with our process and discouraged from resubmitting in future.

      Just an initial thought, could there be a way to impose a cap for example, reviewers cannot allocate more than 5 tickets unless all allocated tickets have been fully reviewed. I know this makes it harder to monitor and more time for admins. So to discourage poor reviewer behaviour, I was thinking that a penalty can be imposed whereby any tickets in addition to this 5 will not count towards the incentive program if an admin feels a “good” attempt has not been made in reviewing their existing tickets. Obviously the definition of “good” here is subjective and would be purely be based on the admins view of what is considered “good”.

      Anyway, thanks so much to anyone who’s taken the time to read this. I’d love to hear your thoughts.

      Afzaal

      • Chip Bennett 1:02 pm on April 9, 2014 Permalink | Log in to Reply

        Note that both of these are counter-productive (and the latter is contrary to our current workflow):

        1. Rush a review to the approval stage simply to free up a slot to allocate a new review.
        2. Rush to disapprove a theme purely because it may take longer to review or require more support to be provided to the developer.

        Neither of these would result in a ticket being made approved and live, and so wouldn’t count toward the incentive. And premature ticket closure is contrary to our workflow.

        Another worry is that the move of closing tickets after 1 week is a move to keep reviewers happy so that 1 of the valuable 5 allocations isn’t unnecessarily filled by an unresponsive developer. I feel a little uneasy about changes which shift the focus of the review process from benefiting developers to benefiting reviewers. The previous rule of admins following up on a ticket after 1 week, and them closing after 2 (if no response is received) is great for developers as it gives them a chance to take action. Disapproving tickets for lack of response after 1 week, especially if the developer hasn’t been warned, might lead to them bring disheartened with our process and discouraged from resubmitting in future.

        This is a valid concern. I would be fine with saying “no more than 5 open/assigned tickets less than a week old“, and then Admin-closing after two weeks of no Developer response. But, I think that a week is more than enough time for a developer to provide some kind of response in-ticket. Let’s see how it goes as-is, and we’ll adjust as necessary.

        Just an initial thought, could there be a way to impose a cap for example, reviewers cannot allocate more than 5 tickets unless all allocated tickets have been fully reviewed. I know this makes it harder to monitor and more time for admins. So to discourage poor reviewer behaviour, I was thinking that a penalty can be imposed whereby any tickets in addition to this 5 will not count towards the incentive program if an admin feels a “good” attempt has not been made in reviewing their existing tickets. Obviously the definition of “good” here is subjective and would be purely be based on the admins view of what is considered “good”.

        My bigger concern – and the concern that could be the downfall of the entire incentive program – is that reviewers are performing every action and basing every decision on how those actions/decisions impact their ability to win the incentive program. If we have reached that point, then the program has, in fact, failed. If reviewers are expending more effort on gaming the incentive program than they are expending effort on contributing to WordPress via Theme reviews, then we’ve failed. If Admins are expending more effort running and managing the review incentive program than we are expending effort on final-approving Themes/pushing Themes live, finding ways to make the review process easier for developers and reviewers, and helping to educate about Theme development best practices, then we’ve failed.

        That we’re getting this much into the weeds with the incentive program is a pretty major indication that it’s becoming more of a detriment than a benefit – which is a shame, because it’s been instrumental in more than doubling the rate of new Themes being made live in the Theme Directory.

    • PageLines 9:40 am on April 10, 2014 Permalink | Log in to Reply

      Hey Chip! Thank you for the comments and additional guidelines, we’ll do our best to stick with them.

      We do have one question, though, related to the cap:

      What if we have 5 reviews, all completed, but waiting for dev answer? This means that we cannot assign any more reviews until those have been finalised, but devs often take a few days to come back and upload the new version.

      We should be able to assign further tickets as long as the review is started and posted immediately after the ticket assignment, and *before* any further tickets are assigned, right?

      Thank you for your time.

      • Chip Bennett 12:14 pm on April 10, 2014 Permalink | Log in to Reply

        What if we have 5 reviews, all completed, but waiting for dev answer? This means that we cannot assign any more reviews until those have been finalised, but devs often take a few days to come back and upload the new version.

        I’ve been informally monitoring the number of assigned tickets per reviewer for a while. It is very rare for more than a handful of reviewers to have more than 5 open/assigned tickets at any one time. It is also very rare for any reviewers to have more than 20 closed/live tickets in any one month. So, any real impact here is going to be on the extreme edges, based on current review activity.

        If the 5-ticket cap starts causing issues with the queue, we’ll revisit it.

        We should be able to assign further tickets as long as the review is started and posted immediately after the ticket assignment, and *before* any further tickets are assigned, right?

        Of course. Just make sure the first review is thorough and complete, leave review comments, and then feel free to move on to the next ticket.

        • tskk 12:25 pm on April 10, 2014 Permalink | Log in to Reply

          “Of course. Just make sure the first review is thorough and complete, leave review comments, and then feel free to move on to the next ticket.”

          He is talking about further tickets beyond the cap of 5.
          Can you consider increasing the limit to 10?

          • Chip Bennett 1:56 pm on April 10, 2014 Permalink | Log in to Reply

            Can you consider increasing the limit to 10?

            Not at this time. We’ll reevaluate as we move forward, but as I said: based on informal monitoring of assigned tickets, it is extremely rare for any one reviewer to have more than 5 tickets open/assigned at any one time.

            Also: the more tickets open/assigned, the more work it is for a given reviewer to perform follow-up actions when developers respond: actions such as responding to questions, providing guidance for resolving required issues, and reviewing submitted updates.

            I’d rather see a reviewer put more time into a single review, and do one review a day, than to race through multiple reviews a day. And the current throughput and reviewer activity level doesn’t really indicate a need for any single reviewer to have more than 5 open/assigned tickets at one time.

    • Catch Themes 10:49 am on April 10, 2014 Permalink | Log in to Reply

      Thanks Chip and the Admin Team for highlighting the points so that we have clear winner :)

    • ThinkUpThemes 1:09 pm on April 11, 2014 Permalink | Log in to Reply

      Hi Chip,

      In relation to the cap I just wanted to seek clarification on one of your points:

      I would be fine with saying “no more than 5 open/assigned tickets less than a week old“

      So am I correct in thinking that if a ticket has been open for more than 5 days then a reviewer can assign a new ticket? So essentially the 5 cap applies only to themes opened less than 7 days ago, thereby eliminating the issue of hogging?

      Thanks as always!

      • Chip Bennett 1:36 pm on April 11, 2014 Permalink | Log in to Reply

        For now, please stick to the 5-ticket cap. Admins will close tickets that have no response for longer than a week.

    • tskk 1:26 pm on April 15, 2014 Permalink | Log in to Reply

      @chipbennett how do we notify selective assignment if spotted?
      Or is that something you will take care of yourself?

    • Ulrich 3:55 pm on April 18, 2014 Permalink | Log in to Reply

      This is my suggestion to the admins.

      Reward the top 10 reviewers by letting them choose a theme that will be featured. This should help reduce the amount of competition. We have ten spaces then why not use them all.

      Every theme should be reviewed by second theme reviewer. We can use the workflow keyword “2nd opinion” to mark the themes are ready to be approved but need to be reviewed by another reviewer.

      The report can be seen here.
      https://themes.trac.wordpress.org/query?keywords=~2nd-opinion&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=time&col=changetime&report=5&order=priority

      This should reduce the work for the admins and cultivate team work. This will reduce the risk of a theme not being reopened due to human error or not knowing something. This should also help standardize the theme reviews.

    • Emil Uzelac 9:53 pm on April 18, 2014 Permalink | Log in to Reply

  • Chip Bennett 8:05 pm on April 7, 2014 Permalink | Log in to leave a Comment
    Tags: , ,   

    How To Ensure Your Ticket Passes Final Approval Audit 

    Over the past couple of months, the number of approved tickets that have been reopened due to issues found during final-approval audit has declined, but many still get reopened. As a team, we want to ensure that tickets get approved (so that new Themes get added to the directory, and into the hands of end users), and we want reviewers to be able to take advantage of the incentive program.

    So I want to step through the things I check when performing a final review audit. We’re looking for some high-level and/or high-impact things that would cause problems for end users:

    Overall File Structure

    • Does the Theme look like it is derived from a common Theme (Underscores, Twenty Ten-Fourteen, etc.)? Are there included functional files that I’ll need to check. Are there asset folders (fonts, scripts, etc.) that I’ll need to check?

    style.css

    • Check ThemeURI and AuthorURI. Are they appropriate?
    • If ThemeURI or AuthorURI reference commercial Themes, are those Themes sold as GPL-compatible?
    • If the Theme appears to be derived, does it include a proper derivative-work copyright/license attribution?

    readme.txt

    • If the Theme has bundled resources/assets, are they listed in the readme with copyright/license attribution, or will I need to check file headers?
    • Is license for all bundled resources GPL or GPL-compatible?

    header.php

    • Are Theme options properly escaped on output?
    • Is favicon, if used, disabled by default?
    • Does the TITLE tag include anything other than the call to wp_title()?
    • Does wp_nav_menu() reference theme_location, and not menu?
    • Are any stylesheet or script links output instead of being properly enqueued

    footer.php

    • Does the Theme only use one credit link? Is that credit link exactly ThemeURI or AuthorURI, with no SEO-seeding of link text, title attribute, etc.?
    • Are any footer scripts output instead of being enqueued properly?

    index.php

    • Does template markup look generally appropriate?

    Front/Home Page Templates

    • Does the Theme have front-page.php? If so, is it used properly? Does it account for both a static page and the blog posts index as site front page?
    • Does the Theme have home.php? If so, is it used properly as the default blog posts index template?
    • Does the Theme have any custom page templates intended to be used as either front-page.php or home.php?

    comments.php

    • Does the Theme properly output wp_list_comments() for the comments list?
    • Does the Theme properly output comment_form() for the comment reply form, rather than hard-coding the form?

    page.php

    • Does the page template properly call comments_template()?

    functions.php

    • Are all functions and other things in the public namespace properly prefixed?
    • Is all functional output properly wrapped in callbacks and hooked into appropriate actions?
    • Is any of the functionality Plugin territory?
    • Does any of the functionality replicate core functionality?
    • Does the Theme use Theme options? Are they handled properly (single DB entry, proper settings page, sanitized on input, etc.)?
    • Do any of the Theme options replicate core options?

    This list comprises 99% of what I look for in an audit, and accounts for the vast majority of issues encountered that require reopening tickets. So, if you verify these things before resolving the ticket as “approved”, the chances that your ticket will get reopened will go down considerably.

    (Note: @emiluzelac may have other things to add to the list, for things he checks during an audit.)

     
    • ruphel 9:16 pm on April 7, 2014 Permalink | Log in to Reply

      Thanks for the checklist Chip

    • Emil Uzelac 11:47 pm on April 7, 2014 Permalink | Log in to Reply

      My process is very similar to @chipbennett and there is nothing more to add.

      I would like to say that rushing through reviews will most definitely resolve getting your ticket reopened. Take your time and you will be golden.

      Your time and efforts are greatly appreciated!

      For some weird reason, my post got published 4 times, sorry about that.

    • ThemeZee 7:28 am on April 8, 2014 Permalink | Log in to Reply

      “Does the Theme have any custom page templates intended to be used as either front-page.php or home.php?”

      Does this mean we are not allowed to use for example template-homepage.php for a custom front page template, but have to use front-page.php instead ?

      • Justin Tadlock 3:10 pm on April 9, 2014 Permalink | Log in to Reply

        home.php and front-page.php have very specific meanings in WP. They should only be used when they’re meant to be used. template-example.php, even if intended to be used on the front page, can also be used if the template can be used with any page.

        Personally, I prefer that devs make a custom page template that’s usable on any page. It offers more flexibility for users.

        • Chip Bennett 3:22 pm on April 9, 2014 Permalink | Log in to Reply

          Intent is important here. If it’s obviously intended as the site front page template, then it should be `front-page.php`. If the developer doesn’t necessarily intend for it to be the site front page, then call it a “featured content” template (or something similar), and name it `template-featured.php` (or whatever).

    • Jose Castaneda 11:25 pm on April 8, 2014 Permalink | Log in to Reply

      One of the things I often look for are hooks and filters. I’ve seen a few themes that register meta boxes and some that add post types. It just depends on how they are being used ( though post types should really be avoided in themes ).

      One thing I’ve learned is using the command line to my advantage and using grep to find all the add_action/add_filter calls a theme uses.

    • PageLines 12:30 pm on April 9, 2014 Permalink | Log in to Reply

      Thanks for the checklist guys!

  • Chip Bennett 2:53 pm on April 3, 2014 Permalink | Log in to leave a Comment
    Tags:   

    Discussion: Theme Activation Standards 

    Currently, we are proposing a change to the Guidelines that would blanket-prohibit Themes from adding admin notices, redirects, or other similar functionality on Theme activation. The intent of the Guideline is to curtail the proliferation of nuisance/obtrusive Theme marketing in the user’s Admin.

    In the vast majority of cases, such activation functionality should not be needed. Themes are required to use add_theme_page() to add a Settings page, which ensures that end users can always find the Theme’s settings page under the Appearance admin menu. The new requirement for sane defaults will ensure that Themes will always work “out of the box”, at least at a baseline (i.e. no-broken-sites) level. The recommendation for Themes to hook settings into the Customizer will help ensure that end users can visually preview Theme settings changes via the core Customizer (Appearance -> Customize). And the Contextual Help API ensures that Theme developers can add as much rich-text setup/configuration detail as necessary.

    That said, there is still room for appropriate, standardized admin interventions on Theme activation. If we can agree on a consensus for standard admin interventions, then the Guidelines could reflect that consensus, and reviewers would have an objective standard to use when reviewing Themes.

    So, let’s discuss. What’s appropriate? What isn’t? How can we define appropriateness objectively?

     
    • Han 3:28 pm on April 3, 2014 Permalink | Log in to Reply

      How about adding admin notice about important changes of the new version? For example when we move the header and footer script, or custom css feature from theme to a plugin.

    • mudthemes 7:04 am on April 4, 2014 Permalink | Log in to Reply

      Even if the theme redirects the user after activation, the redirect must be to a page where only the documentation regarding the theme configuration is written.

      • eminozlem 5:51 pm on April 4, 2014 Permalink | Log in to Reply

        I think the extent may be defined with two rules a.) The redirected page should be theme’s page (most likely options page or documentation help page) b.) The page should be a related page and not advertising purpose page. Just like the same in many rules such as author & theme uri etc.

    • eminozlem 9:16 am on April 4, 2014 Permalink | Log in to Reply

      I don’t understand what’s the big deal with redirection on activation. It’s essential for themes with Option Framework that the options are initially saved for the theme to properly function.
      If you put your mind to it you can use anything out of its intended purposes so I think it’s not fair to punish everyone else because of the aggresive advertisers by prohibiting all kinds of redirecting / notice boxes that could otherwise prove useful.

      Semi OT: You mentioned sth. about the screenshot. I totally agree with that, screenshot guidelines shouldnt be that restrictive. After all, no theme ever really does look like the demo out-of-the-box. Screenshots should be allowed to display what theme might look like with proper modifications and / or content.
      Also I dont see any harm including other material in screenshot (like theme options). I mean some themes provide 4 theme options, others a hundred. It’d be eye-catching and easier for the user to spot the one for them if other aspects are allowed in the screenshot.

      • Chip Bennett 2:23 pm on April 4, 2014 Permalink | Log in to Reply

        Its essential for themes with Option Framework that the options are initially saved for the theme to properly function.

        This statement is untrue. Refer back to the Sane Defaults requirement. Properly implemented Theme options work out of the box, using developer-determined sane defaults, without any user configuration required. Themes that “break” if the user doesn’t configure Theme options have not properly implemented those Theme options.

        If you put your mind to it you can use anything out of its intended purposes so I think its not fair to punish everyone else because of the aggresive advertisers by prohibiting all kinds of redirecting / notice boxes that could otherwise prove useful.

        I do agree, which is why I’ve linked to a separate discussion, so that we can work out some meaningful standards/guidelines.

      • Chip Bennett 3:02 pm on April 4, 2014 Permalink | Log in to Reply

        It’s essential for themes with Option Framework that the options are initially saved for the theme to properly function.

        This statement is untrue. Refer back to the Sane Defaults requirement. Properly implemented Theme options work out of the box, using developer-determined sane defaults, without any user configuration required. Themes that “break” if the user doesn’t configure Theme options have not properly implemented those Theme options.

        If you put your mind to it you can use anything out of its intended purposes so I think it’s not fair to punish everyone else because of the aggresive advertisers by prohibiting all kinds of redirecting / notice boxes that could otherwise prove useful.

        I do agree, which is why I’ve linked to a separate discussion, so that we can work out some meaningful standards/guidelines. Please provide feedback there.

    • tskk 2:35 pm on April 4, 2014 Permalink | Log in to Reply

      I have opposed banning redirection in earlier discussion too, taking the user back to themes page after activation is meaningless, doesn’t serve any purpose, they have to click on theme options link and go select options anyway to get the best out of the theme. Wasting their time and effort is no good.

      Redirecting to theme options should be allowed.

      • Chip Bennett 3:07 pm on April 4, 2014 Permalink | Log in to Reply

        I disagree that user configuration of Theme options should be necessary. That’s partly why we’re adding a requirement for Sane Defaults. Users should know where to find the Theme Settings page should they choose to modify any of the defaults; users should not be forced to deal with settings configuration out-of-the-box.

        Theme-activation redirection UX is something that should be defined and handled by core, IMHO.

      • Frank Klein 3:21 pm on April 4, 2014 Permalink | Log in to Reply

        I disagree with this.

        When a user installs a theme, the experience should be the same, no matter what theme he just installed. Forcing the user to visit another screen without any action from his part makes the experience very confusing, especially if it doesn’t happen that way consistently.

        Imagine that you are a user. You found this great theme, installed it and activated it. The last thing you want to see is an options panel. You want to see how your site looks with this theme applied!

        Is the current theme activation experience as good as it can be? Probably not, but the Core team has been working on this, and they will continue to iterate on it. We should let Core handle this, because it is part of the core experience of WordPress.

        Themes should look good without having to configure options. Nobody likes setting up something, it is boring and often complicated.

        Also if a user can’t figure out quickly how to get a free theme to look like he wants, he’ll either post a support forum message or just download another theme. Either way that is not the outcome you want to achieve.

    • Devin Price 4:02 pm on April 4, 2014 Permalink | Log in to Reply

      I don’t think we should do a blanket-prohibition of admin notices. I am using two in the latest version of Portfolio Press if users are upgrading (so, actually, re-activation). One is a notice about important updates in the version, along with a link to a post about them. A second notifies users that they should update their Reading Settings. I think room should be left in the theme review guidelines for situations like this.

      I agree that upsell and marketing links should not be allowed on activation.

      I agree that notices about an option panel do not need to be displayed on activation. Users are becoming more aware about the “Customize” link. Good theme documentation can also help educate users.

      @eminozlem, Chip is absolutely right about defaults in Options Framework. You should pass a default when you call an option, e.g. of_get_option( ‘my-option’, ‘default’ ), if the theme requires something other than “false” as the reply if the option is not set. Does that make sense? Happy to explain more if needed: http://wptheming.com/contact

      • eminozlem 9:13 pm on April 4, 2014 Permalink | Log in to Reply

        Sure, my theme with OF does work defaults. But there are other specific reasons my implementation of the theme is better with an initial redirect (refresh) on activation.
        I agree with notices.The point is, no matter what the case is, be that notice, redirection, prompt or whatever, I think all should be allowed as long as it’s used in good faith and not in an abusive manner for marketing purposes.

    • @mercime 2:50 am on April 5, 2014 Permalink | Log in to Reply

      +1 blanket-prohibit Themes from adding admin notices, redirects, or other similar functionality on Theme activation.

    • CyberChimps 6:13 am on April 8, 2014 Permalink | Log in to Reply

      We’re strongly opposed to banning all theme activation notices.

      Some kind of default activation notice is required to alert users of a successful activation and what to do next.

      A better solution would be to have a default activation notice similar to what we’ve built into Responsive that directs users to theme options and documentation.

      In a perfect world, we’d all love if a theme just worked out of the box, but the truth is users need a help in knowing what the next step is to configure their website, and some customization will always be required, and the Customizer does not include all use cases yet.

      When .org forced us to removed redirects in our themes we saw significant increases in theme abandonment, and increased support from people who could not locate the theme options. Adding a customize button to the wp-admin bar is an illogical UI progression that most users don’t follow.

      Also, what about notices for companion plugins? Does this proposal intend to ban those as well?

      • Frank Klein 6:55 pm on April 8, 2014 Permalink | Log in to Reply

        TGM Plugin Activation notices or similar are not prohibited by this guideline.

        The Customizer has been introduced a while ago, and users definitely are getting used to it. Having to go to a separate options page to configure the front end of your website is more illogical than a contextually relevant and prominent link to the Customizer.

        I agree that there will always be users that struggle with setting up a theme, and we should help those. But I think that overly complicated themes are more to blame than the banning of admin notices or redirects.

        Instead of every theme shop rolling their own, we should use the feedback and experiences made to develop a Core proposal that will alleviate some of these pain points.

        • CyberChimps 12:21 am on April 17, 2014 Permalink | Log in to Reply

          The Customizer is fine for front end settings, but what about template configurations? Layouts?

          The Customizer is a great tool, but not an end to theme options.

          We would love standardized ways of addressing these pain points, which is why we would prefer if admin notices aren’t banned until a solution is actually in place.

      • Emil Uzelac 4:13 am on April 14, 2014 Permalink | Log in to Reply

        Companion plugins are excluded from the get-go and proposal does not affect them.

        Please note that @chipbennett was asking and not making solo decisions here, otherwise this post would not even exist.

        What is the best way to handle the activation and can we standardize?

        Emil

    • sanerdesign 10:22 am on April 8, 2014 Permalink | Log in to Reply

      I’m all for standardizing the theme activation notices rather than applying a blanket ban. I really feel that the admin notices can benefit the user if we can agree on the information that is displayed.
      This is just a suggestion, from my experience, but I think the following should be allowed in an agreed layout:

      Thank you message
      Link to developers site
      Link to the options if the developer isn’t just using customizer (optional)
      Link to that theme’s support pages

      It’s important to link to the developer in recognition of their hard work.
      A link to the theme options, I feel, is still important whilst they still exist. Theme options allow the developer to not just add options but to add a lot more information than they could on the customizer. Yes I am talking upselling (dirty word I know). Whilst we are probably all against over-the-top upselling and promotion of pro options etc we also have to accept that businesses are making money from this format and it is that money that pays for ongoing development and support of the theme. If we make it too difficult for developers to upsell we could end up with a lot of poorly supported themes that don’t get developed. I do think that this would be bad for the end user.
      The link to support pages is self explanatory but it is really important to let the user know immediately that there is somewhere to get help if they struggle with their chosen theme.

      Whilst I believe that it is the user experience that is always the most important for every action that we take we should consider the developer and their business for it will always be them that keep the WordPress themes fresh, supported and developed.

      Hugo

    • Schwarttzy 4:04 am on April 9, 2014 Permalink | Log in to Reply

      I don’t understand why after a theme gets activated that they get redirected to the page “Appearance.” Are they trying see how many themes they can activate in a minute or what? What’s so wrong with a page explaining the theme? Or possibly things they may want to do now that they activated this theme?

      Why is it so crucial that they get taken back to all the other possible themes they could change too?

      • CyberChimps 1:09 am on April 10, 2014 Permalink | Log in to Reply

        +1.

        Valid points.

        The explanation I was told was because they want to improve the on boarding process for WordPress, and there is some concern that a lack of selection for themes may be a pain point. However, they are simply creating a user experience nightmare with all of these new rules.

  • Chip Bennett 2:35 pm on April 3, 2014 Permalink | Log in to leave a Comment
    Tags:   

    WordPress 3.9 Guidelines Revision Proposal: Round 2 

    Based on the previous discussion, the Guidelines proposal has been revised as follows. The proposed guideline change for Credit Links has been removed, and a proposed guideline change for Custom Logos has been added. Other proposals have been clarified based on previous discussion.

    The two most significant clarifications regard arbitrary header/footer scripts, and Theme activation.

    The proposed Recommended guidelines have not changed. While we understand the argument that there may not be consensus around certain implementations of features, we maintain that it is within the overall scope and objective of the Theme Review Team to encourage, facilitate, and drive best-practice consensus, and that it is in the best interest of end users to establish best-practice consensus. Thus, arguing that the Theme Review Team should not be establishing/promoting such best-practice standards in order to encourage consensus is moot – though we absolutely encourage discussion regarding what implementations should be considered/promoted as best-practice standards.

    Other Discussion Topics

    Screenshot

    I would be interested in discussing our screenshot requirements. It seems that “reasonable facsimile”, as a standard, isn’t really sufficiently meeting anyone’s needs. While the intent was to avoid blatant marketing via screenshot, the requirement tends to be taken so literally that developers cannot even display user-configurable features in their screenshot. This is especially limiting for Themes with featured-content front-page templates. As currently interpreted, the “reasonable facsimile” standard prevents the screenshot from displaying the featured-content front-page, because the featured content first has to be configured. So, let’s discuss a more sane/useful guideline for screenshots.

    Translation-Ready

    I would also be interested in discussing the establishment of at least some “best practice” guidelines for translation-readiness – primarily, proper i18n of strings. I think it is all but a given that, eventually, we should be requiring Themes to be translation-ready; but currently we don’t have objective standards to follow. The first step, then, is creating those standards, and then educating developers on them.

    Required

    Sane Defaults: Themes are required to use sane defaults.

    Themes must not save default settings to the database without explicit user action, and Themes must function properly out-of-the-box without user configuration (that is: if the user does not save settings, the Theme will still function properly)

    Bundled Plugins: Themes must not bundle Plugins.

    Themes may incorporate support for Plugins, and may integrate Plugin code directly into the Theme (if that Plugin code meets the presentation-vs-functionality requirement), but Themes must not bundle Plugins as a whole.

    Arbitrary Header/Footer Scripts: Themes must not provide Theme options for arbitrary header/footer scripts.

    Arbitrary scripts are non-Theme-specific scripts added to the document head or footer to provide non-Theme-specific functionality. Such scripts are Plugin territory, and pose a significant security risk if not handled properly.

    Custom CSS is acceptable, if properly sanitized.

    Note: this guideline does not apply to content “scripts” added to the template outside of the document head/footer, such as script widgets (Google Maps, Twitter Feed, etc.). Essentially, if a script can be hooked into the template via wp_head() or wp_footer(), then it falls under this guideline; otherwise, it does not.

    Theme Activation: Themes must not implement activation redirects, admin notices, or similar functionality on activation.

    Note: this guideline does not apply to admin notices such as TGM recommended Plugins (if implemented properly). I will open a separate discussion regarding consensus/standard “unboxing” admin notices. The purpose of this Guideline is to curtail the proliferation of unnecessary/obtrusive marketing in the user admin area, not to prevent legitimate, useful activation information for the end user.

    Theme Documentation: Themes must place any required documentation in a clearly marked readme file.

    Note: this guideline applies only to required documentation. Required documentation includes copyright/license attributions, Theme design constraints/limitations, description of non-standard Theme functionality, etc. Plugin-standard readme.txt is preferred, but not required.

    License: Themes are required to document in the Theme readme or license file the copyright/license attribution for all bundled resources.

    Themes are required to provide this documentation in the Theme readme file, regardless of where or how those bundled resources provide their own attribution. The purpose is to ensure that end users (and also, reviewers) don’t have to search for this important information, and to ensure that the developer has done due diligence to ensure that licenses for all bundled resources are GPL-compatible.

    Note: “blanket” statements for developer-owned resources (images, etc.) bundled with the Theme are perfectly acceptable. It is only third-party bundled resources that need to be listed explicitly.

    Custom Logos: If implemented, custom logos must be user-configurable, and disabled by default.

    This guideline will bring the handling of Theme options for custom logos in-line with the handling of Favicons. Currently, we require that “default” logos be generic/unbranded, but generic/unbranded logos are of no benefit to the end user. Likewise, Theme-branded logos are of no benefit to the end user. The most logical approach, then, is to require that custom logos not be displayed at all, unless the end user has configured one.

    Recommended

    Theme Documentation

    Themes are recommended to include a Plugin-standard readme.txt file.

    Themes are recommended to meet the WP core standard for inline documentation.

    Translation

    Themes are recommended to be translation-ready.

    Social Profile Links

    Themes are recommended to use a custom navigation menu when incorporating social network profile links

    Theme Options

    Themes are recommended to integrate Theme options into the core Theme Customizer

    Discuss

    Please discuss in the comments below.  If you have any additions to propose, please post them in the comments below, as well.

     
    • Rohit Tripathi 2:41 pm on April 3, 2014 Permalink | Log in to Reply

      +1 For Everything. Especially, the guideline change related to Screenshots. I hope it goes through.

      • Edward Caissie 4:19 pm on April 3, 2014 Permalink | Log in to Reply

        Aside from maintaining the premise of not allowing any sort of spammy / marketing type images I would be fine with opening the screenshots to something relatively easy to configure at installation provided how to create the screenshot’s “look” is documented in at least the `readme.txt` file … or similar documentation available to the end-user within the theme’s package.

    • tskk 2:57 pm on April 3, 2014 Permalink | Log in to Reply

      How much time do we have before recommendation of using theme customizer turns into requirement?

      • Chip Bennett 2:59 pm on April 3, 2014 Permalink | Log in to Reply

        Who knows? It’s something that may never happen. Right now, we just want to promote it as a best practice.

        Is there any particular reason not to want to integrate Theme options into the Customizer? I’m genuinely curious what those reasons might be.

        EDITED TO ADD: We made the Settings API *recommended* years ago, and have never made any move toward making it *required*.

        • tskk 3:03 pm on April 3, 2014 Permalink | Log in to Reply

          the left pane is very cramped,
          no framework like OF which has all the controls we need,
          no drag and drop setting/option

          • Chip Bennett 3:06 pm on April 3, 2014 Permalink | Log in to Reply

            True, the left pane can be a bit cramped – but I think they payoff (real-time preview of the setting change) more than compensates. Also, if you segregate your Theme options into logical sections, it makes the UI much more manageable.

            Almost every control imaginable is built-in to the API. Which one(s) are you missing, specifically?

            Can you explain the drag-and-drop? I’m not sure what you mean.

            • tskk 3:21 pm on April 3, 2014 Permalink

              The ability to use images in place of radio buttons like in optionsframework, ability to have a vertical accordion like i have in optionsframework. Is it possible to add stand alone text and maybe apply css to that text?

              If you see one of the cyberchimp themes, they have a drag and drop option where user can drag page elements around like reordering the site elements.

          • Zulfikar Nore 4:03 pm on April 3, 2014 Permalink | Log in to Reply

            I think with a little thought the drag and drop for page builders can also be integrated in the customizer.

            My thoughts here are based on the new incoming Widget control section which has some drag and drop options for the widgets.

        • tskk 2:08 am on April 4, 2014 Permalink | Log in to Reply

          Also customizer doesn’t have a reset button, import/export option.

    • daveshine (David Decker) 3:39 pm on April 3, 2014 Permalink | Log in to Reply

      *Tranlation-ready* should totally be REQUIRED!

      Why?
      Translations are essential, and I never understand why an international used platform like WordPress.org/themes should promote untranslateable themes in the year 2014+ ?!?

      *Translation-ready* also means a theme can be customized with another tone/ type/ of the same language, for example, still in English (but use British English, formal tone for business etc.).

      Also, to have it as a requirement will improve overall theme code and quality – and especially make them more user-friendly! And not to forget, it will all help prepare for upcoming launch of language packs (that are prepared in core, but “not yet launched” here on .org for themes, plugins).

      Standards:
      There are already standards all there, for years already:

      • load_theme_textdomain()
      • load_child_theme_textdomain()
      • all the translation functions for strings like __(), _e() plus all the related stuff there…
      • using sprintf() and printf() to keep as much markup as possible out of translation strings
      • using _x() context translation functions to add context to strings and/ or be better help for translators

      I don’t know any reason why *Translation-ready* should not be required for any theme here (and plugins also!)? So, if someone has such a reason for me, I am eager to know what speaks against all that.

      As a international developer and user I am fighting the good fight for years already, to get more and more themes, child themes and plugins translateable at all, and moreover to have them in the “standards” (see points above). WordPress.org theme repository should be the leading example in this field!

      PLEASE make it a requirement! Thanks so much!

      Dave from Germany :)

      • Chip Bennett 3:55 pm on April 3, 2014 Permalink | Log in to Reply

        I agree that making it *required* is our end goal. But we have to start somewhere. We have to have an established, objective string-translation standard. Then we have to educate developers how to implement it – and we have to educate reviewers how to evaluate Themes against it. We’re simply not ready for that.

        I want to get there. We need to get there. I agree that WPORG-hosted Themes should all be translatable. But we’re simply not at the point that we can make it required, and be able to enforce it fairly and consistently.

      • Ulrich 4:55 pm on April 3, 2014 Permalink | Log in to Reply

        I think it would be great if every theme and plugin were internationalized. From my experience reviewing themes a number of theme developers do not have a deep understanding of PHP so they find it difficult to internationalize the theme. By placing it as a requirement we are increasing the level of entry. Do we really want to do that?

        Somehow I think it is a theme developers decision to choose to support i18n or not. Users can show the demand by asking for i18n support.

        We did once discuss what is needed for a theme to be allowed to use the translation-ready tag. At the time we decided that all strings had to be internationalized and the header should contain the text domain but does not need to include a POT. How far do theme reviewers enforce best practices for creating strings?

      • Justin Tadlock 5:01 pm on April 3, 2014 Permalink | Log in to Reply

        I’m one of the biggest supporters of theme authors internationalizing their themes that you can find. I’ve done this myself for many years. However, I cannot get behind this being a requirement, at least at this point in time. The reason for this is that it is a huge barrier to entry. I probably would’ve never made my first theme if I had to do this.

        We need to start with education. I’d also like to see all articles in the Codex and Handbooks that provide code examples with text strings to actually show those strings with the proper functions.

      • Sami Keijonen 5:30 pm on April 3, 2014 Permalink | Log in to Reply

        I’m with David here. Make it required. This has been in theme/plugin development best practice for so long that it should be required. I don’t mind if it makes a little bit harder to get your theme on WP.org. It should be about quality, not quantity.

        Do you have data how many new themes are not Tranlation-ready?

      • wpweaver 8:02 pm on April 8, 2014 Permalink | Log in to Reply

        I am 100% for making theme translatable on the front side (visitor view).
        But, as usual, my theme (Weaver II) is on the extreme edge of things.

        It has a very complex admin interface, full of long, complex, and technical descriptions. It uses complex HTML and scripts to make the admin user interface work with its hundred of options.

        Making this in a translatable form (it has LOTS of direct HTML output) is next to impossible. And this doesn’t even mention how difficult it would be to get technically correct translations.

        But for the visitor side, my theme has already been translated to something like 25 languages, and that number is growing.

        So – visitor side translation capable – REQUIRE asap.
        admin side translation capable – leave recommended.

    • Frumph 3:45 pm on April 3, 2014 Permalink | Log in to Reply

      … no, no and no.

      If you want to write up guides and point to them to the designers, go for it.

      • daveshine (David Decker) 3:54 pm on April 3, 2014 Permalink | Log in to Reply

        Yes, sometimes to much guides/ guidelines make things more complicated and could take away some passion…!

        However, it’s not only about design here: it’s also about usability, security of code, making stuff translateable, avoiding aggressive marketing in themes etc. Over the years, the experience was some guidelines should be in place.

    • Morgan Feeney 3:47 pm on April 3, 2014 Permalink | Log in to Reply

      Theme options in the customizer! The purpose of which is for live preview of your website, including styles, now widgets too, so why exclude any other content which can be edited on the back end from this?

      It makes total sense to me, I am a designer and front-end guy and I think it would be of benefit.

      if WordPress is to carry on competing on a global scale with new and existing CMS, anything which enables a real-time preview into how the site looks with on-the-fly changes being made only makes it a stronger contender in the long-run.

    • Zulfikar Nore 4:13 pm on April 3, 2014 Permalink | Log in to Reply

      +1 on all proposed changes!

      The Screenshot issue has been too stringent – allowing for configurable options to be included in the screenshot is a sane proposal.

    • Justin Tadlock 5:07 pm on April 3, 2014 Permalink | Log in to Reply

      I still disagree with the requirement for license info to be placed into readme.txt. WordPress itself uses a license.txt file for this. I’d be more supportive of this if we allowed for license info to be placed in either license.txt or readme.txt.

    • Ron 5:23 pm on April 3, 2014 Permalink | Log in to Reply

      Most of the guidelines seems to be in order but I have to disagree with the Custom Logo being disabled by default. Unlike the favicon, a logo affects the overall design of a theme.

      Currently, we require that “default” logos be generic/unbranded, but generic/unbranded logos are of no benefit to the end user. Likewise, Theme-branded logos are of no benefit to the end user.

      True, but what about one that is not visible? There might not be any benefit to the end user whether a custom logo is enabled or disabled by default but then why make it a requirement?

      If the generic logo requirement isn’t good enough then maybe we could even have a standard logo image for every theme with a logo option to use. Maybe a WordPress logo?

      • Chip Bennett 5:55 pm on April 3, 2014 Permalink | Log in to Reply

        You might be misunderstanding what “disabled by default” means. It just means that, if the user configures a logo, then that logo is output; otherwise, if the user does not configure a logo, then no “default” logo is output.

        I’d be curious to see an actual case where a “disabled by default” custom logo would adversely impact a Theme’s design.

        • Ron 6:10 pm on April 3, 2014 Permalink | Log in to Reply

          Yup, that’s exactly how I understood the guideline. It might not be a very big difference but a visible logo helps a user envision the overall look of the theme after he/she either removes or changes it.

          My point is, are there actual benefits at all from a custom logo being either enabled or disabled by default that would command it to be a requirement? If there aren’t any then this would just unnecessarily add to the growing set of guidelines.

          • Chip Bennett 6:20 pm on April 3, 2014 Permalink | Log in to Reply

            Yes, there are actual advantages. A company is very likely to have a pre-defined logo that they would then configure for use with the Theme. But an individual (or even a non-commercial entity or group) likely won’t have a logo. For them, it is detrimental to have an arbitrary image (or worse: an image that says “Logo”) prominently displayed on their site. (Note: it would be equally bad to have an image displaying the Theme name prominently displayed on their site.)

            This is not an onerous guideline. It simply means wrapping the logo output in if ( '' != $themename_options['logo'] ) {}.

            • Ron 6:50 pm on April 3, 2014 Permalink

              A company is very likely to have a pre-defined logo.
              An individual (or even a non-commercial entity or group) likely won’t have a logo.

              Is this really the case? I think a user, whether intending a theme for company or individual use would decide on what theme to use basing on it’s looks (screenshot & preview). The visibility of a logo and seeing its positioning helps make that decision. I think it’s pretty reasonable to expect a user who chooses a theme that has a logo option, to use one. I think the fact that logos are required to be user-configurable is good enough.

              I’m not really that much against it but I just can’t see the guideline as an actual benefit to warrant being a requirement.

            • Chip Bennett 9:26 pm on April 3, 2014 Permalink

              I doubt that users primarily base their decision on whether or not a Theme supports a custom logo, nor should individuals be barred from using a Theme simply because it outputs a logo, and they don’t have or need one.

              The decisions to add this guideline is born out of the numerous instances of logos being problematic during the review/approval process. This guideline actually simplifies things for Theme developers, who no longer have to come up with a “generic/unbranded” logo to output by default (most of the time, Themes have had a Theme-branded default logo, and had to go to the extra effort to create an unbranded logo).

        • Catch Themes 3:26 pm on April 5, 2014 Permalink | Log in to Reply

          I think the only problem with this will be in Theme Preview. If we have option to make that logo available in preview then there will be no effect on design. For example in Catch Kathmandu theme, I added in generic logo just to show in Theme Review at http://wp-themes.com/catch-kathmandu/

    • Emil Uzelac 6:20 pm on April 3, 2014 Permalink | Log in to Reply

      +1 for all @chipbennett!

      • alex27 10:34 am on April 4, 2014 Permalink | Log in to Reply

        I totally agree with Chip on the logo. What good does the fancy custom logo does to the user anyway, when theme authors do not provide source files to customize it (not that I think we should require it).

    • Codice e Bella 2:47 am on April 4, 2014 Permalink | Log in to Reply

      I only developed my first theme for the repository about a month ago. While I created a very basic and simple theme to learn the process so that I could start developing themes that weren’t so “basic”, I didn’t think learning to develop it as translation-ready was that difficult. Someone else said it should be about quality, not quantity. Even as a new developer to the theme repository, I would agree with that, and say making it a required practice shouldn’t stop someone from developing new themes for the repository.

    • mudthemes 6:57 am on April 4, 2014 Permalink | Log in to Reply

      Screenshot Proposal is brilliant.

      To make it more transparent, why not ask theme developers to include the exact method used to achieve the screenshot in readme.txt or link the plugins/widgets used to create it.

    • Frank Klein 11:47 am on April 4, 2014 Permalink | Log in to Reply

      Screenshot

      I think that the screenshot should reflect the intended use case of the theme, so a blogging theme (like TwentyThirteen) should show the blog index. A theme with a front page template (like TwentyTwelve) could show this specific template.

      I’d argue though that themes should always show the front page (whether a blog index or static page). It is okay to have a screenshot that shows user-configurable options already set up. However the screenshot should not contain elements that are dependent on the installation of plugins.

      Translation-Ready

      I would be in favor of making this a requirement. I agree with Justin Tadlock that this represents a further barrier to entry for theme developers. But I would say that we need to think about the users first and only then about the developers.

      With an increasing number of WordPress users that don’t have English as their native language, themes that are not translated are effectively useless. So in this case, we should side with the users and make this a requirement.

      I would also add that Internationalization is no longer a mystery or novelty. With the amounts of tutorials and WordCamp talks on this subject, learning this is not an issue.

      Custom Logos

      I agree fully with this. Custom logos are an option. As such they should optional.

      Licensing information

      Having a file (whatever name it has) that contains all the licensing information for the theme is a big help for reviewers. It also makes the life for users easier that want to use themes for their own projects, as they don’t have to worry about finding out what licenses apply to the different elements of a theme.

    • Devin Price 4:09 pm on April 4, 2014 Permalink | Log in to Reply

      +1 for all these changes.

      I think we should put a note about how to properly enqueue Google fonts as this has come up a lot in my recent reviews. It could also perhaps be somewhat automated: https://github.com/Pross/theme-check/issues/6

      I would also love to get more theme support for translations. Perhaps there is a better way to match theme developers who are knowledgeable in this area with developers who wish to learn it?

    • Catch Themes 3:32 pm on April 5, 2014 Permalink | Log in to Reply

      +1 for the changes. Ready to work on Theme Review. Will need to adjust some on my themes as well. Thanks :)

    • Scott Smith 2:18 am on April 7, 2014 Permalink | Log in to Reply

      How should I be escaping Custom CSS? I crawled the internet for a while and wasn’t able to find a helpful bit of advice on this.

    • Catch Themes 12:49 pm on April 17, 2014 Permalink | Log in to Reply

      Has this guideline been finalize. Can we use this proposed guidelines while review new themes. WordPress 3.9 is out now. But I don’t see final guidelines.

  • Chip Bennett 12:23 pm on April 2, 2014 Permalink | Log in to leave a Comment
    Tags:   

    Theme Review Incentive: March 2014 Winners 

    We are pleased to announce the following winners for March 2014.

    New Themes

    Theme Updates

    Please post a comment below, indicating your Featured Theme selection.

    Monthly Stats

    • New Themes:
      • Approved/Live: 97
      • Reviewers: 15
      • Eligibility: 10
      • Reopened: 19
      • Missed Eligibility due to reopened tickets: 2
    • Theme Updates
      • Approved/Live: 468
      • Reviewers: 19
      • Eligibility: 117

     

    Featured Theme Rotation

    Per the program changes, all Themes but the following will be removed from the list:

    • Twenty Fourteen
    • Twenty Thirteen

    Note: review totals are taken as a snapshot at an arbitrary point in time, based on this Theme-Trac report, and varying the filters as appropriate.  Totals are as accurate as reasonably possible.

     
    • acosmin 12:26 pm on April 2, 2014 Permalink | Log in to Reply

      Congrats to the winners! :)

    • tskk 12:34 pm on April 2, 2014 Permalink | Log in to Reply

      Thanks Chip and congratulations to other winners.
      I will continue with Alexandria.

    • ThinkUpThemes 12:39 pm on April 2, 2014 Permalink | Log in to Reply

      Congratulations guys!

    • alex27 1:09 pm on April 2, 2014 Permalink | Log in to Reply

      I’ll go with Sugar & Spice for this month. Thank you!
      http://wordpress.org/themes/sugar-and-spice

    • Rohit Tripathi 1:31 pm on April 2, 2014 Permalink | Log in to Reply

      Hi Chip, I will go with Fifteen -> http://wordpress.org/themes/fifteen/

      Congratulations to All and Thank you!

    • PageLines 1:45 pm on April 2, 2014 Permalink | Log in to Reply

      Congrats to everyone! We’ll go with DMS -> http://wordpress.org/themes/dms/

    • Themonic 1:54 pm on April 2, 2014 Permalink | Log in to Reply

      Congrats guys!

      @Chip So even the reopened tickets like these won’t count?
      https://themes.trac.wordpress.org/ticket/17544
      https://themes.trac.wordpress.org/ticket/17281

      • Chip Bennett 4:30 pm on April 2, 2014 Permalink | Log in to Reply

        Correct. If Emil or I have to reopen the ticket due to required issues that are missed prior to approval, then the ticket won’t count toward the incentive.

        • Themonic 10:29 am on April 8, 2014 Permalink | Log in to Reply

          The thing I wanted to show you was, that those were super minor issues and could have easily approved while asking the theme author to submit a new version asap. Even after 12 reviews this month, I didn’t make it to the list :( . I am relatively a new reviewer ~100 reviews, I am kinda feeling like if you have to then you can re-open any ticket this way and then just due to couple of tickets whole contribution effort seems to be wasted. If I am feeling this than what about the would be reviewers, wanting to get into theme reviews? I hope you get my point.

          May I also propose a small change :

          Approved/Live: 97
          Reviewers: 15
          Eligibility: 10
          Reopened: 19
          Missed Eligibility due to reopened tickets: 2

          Eligibility now 97/10 = 9.7

          Eligibility should be 97 – 19 = 78

          New Eligibility = 78/10 = 7.8

          This is more fair right?

          The current program is kind of perfect. Kindly look into it based on the above, it would be great if something can be done for reopened tickets with minor issues.

          • Chip Bennett 12:00 pm on April 8, 2014 Permalink | Log in to Reply

            97-19 is not correct. The 97 is the number of Themes approved and made live. Of the 97, 19 had to be reopened but were subsequently re-approved, and then made live.

            I am kinda feeling like if you have to then you can re-open any ticket this way…

            We’re not trying to reopen tickets for minor issues. We have no desire to “punish” reviewers for minor mistakes, nor do we have any desire to create more work for everyone involved. That’s why I published this list: so you know exactly the things I’m looking for.

            …and then just due to couple of tickets whole contribution effort seems to be wasted. If I am feeling this than what about the would be reviewers, wanting to get into theme reviews?

            Honestly, if the only reason that people are reviewing Themes is to win the incentive, then the entire system is failing, and w’ll have to reconsider it. The Theme Review Team is a community contributor group, which exists in order for WordPress community members to have an opportunity to contribute back to the WordPress project. Anyone involved merely for the incentive is involved for the wrong reasons.

    • Catch Themes 2:47 pm on April 2, 2014 Permalink | Log in to Reply

      Thanks Chip and Congratulations to all the winners.
      I will go with Catch Kathmandu http://wordpress.org/themes/catch-kathmandu/

    • Zulfikar Nore 7:55 pm on April 2, 2014 Permalink | Log in to Reply

      Congratulations to all winners – especially Catch Themes, good to see you in the mix :)

      I’ll go with Itek for this month: http://wordpress.org/themes/itek – thank you

    • robin90 7:10 am on April 3, 2014 Permalink | Log in to Reply

      Thanks Chip and Emil for their support and Congratulations to all the winners. I will go with Spacious http://wordpress.org/themes/spacious

    • talentedaamer 3:01 pm on April 3, 2014 Permalink | Log in to Reply

      congratulations to all winners, specially new for this month (pagelines, catchthemes) keep it up.

    • CyberChimps 10:30 pm on April 7, 2014 Permalink | Log in to Reply

      I would like an explanation as for why we didn’t win when we had more reviews then some of the winners.

      My staff has informed me that two of our reviews were discredited?

      I seems odd that an entire review should be discredited if a ticket only has to be re-opened once. Even if a review has to be reopened before final approval, the reviewer still did the majority of the review did they not?

      • Emil Uzelac 11:54 pm on April 7, 2014 Permalink | Log in to Reply

        As previously noted: if the ticket gets reopened that will resolve getting your ticket discredited.

      • Chip Bennett 12:25 am on April 8, 2014 Permalink | Log in to Reply

        The rules are stated, quite clearly:

        So, starting this month, any ticket that has to be reopened after being closed as “approved” will not count toward the incentive program.

        So yes: if a ticket gets reopened, even once, after approval, it doesn’t count toward the incentive. The reasoning is explained in the linked post. We’re not looking for nit-picky things when we do a final approval audit (see my checklist above). We’re not looking for excuses to reopen tickets, to create more work for everyone involved. If you disagree with the assessment to reopen any particular tickets that your staff have completed, you’re welcome to comment, in-ticket, to discuss it.

        • CyberChimps 3:23 am on April 8, 2014 Permalink | Log in to Reply

          We were under the impression that this contest was designed so that the admins of WordPress.org weren’t the ones deciding which themes get featured?

          Yet now you’re telling us a new rule has been put in place that just so happen to discredited two of our reviews pushing us out of contention to win the contest? Hmm.

          A rule that basically allows any admin the ability to discredit any reviews they want by simply re-opening a ticket essentially granting the admins the power to determine who wins the contest. I thought this was the very thing we were trying to get away from?

          I can easily understand why such a rule might be necessary for tickets that require being open more then once, but a ticket that has only been reopened once should not be disqualified.

          We are volunteering to contribute reviews, and now you’re telling us we’re not even allowed a single mistake?

          Theoretically you can find a reason to reopen any ticket at any time.

          This rule is completely unfair.

    • CyberChimps 5:04 am on April 8, 2014 Permalink | Log in to Reply

      CyberChimps did 10 reviews, and legitimately won this months review contest for the following reasons:

      1) Selective reviewing.

      We keep getting stuck with the themes that the other theme reviewers know have a chance of being re-opened. Everyone has figured out that if they target themes from reputable theme shops including Automatic the odds of the ticket being re-opened are lower, and they have to do less work per review.

      Go take a look at your winners this month and decide for yourself.

      It is an unfair practice that you guys are rewarding.

      2) The reopen ticket rule is not fair.

      I understand why this rule was put in place, but almost any ticket could be reopened at any point for any reason. It isn’t a fair metric to judge disqualifying a review on. We’re putting in hard work that isn’t being credited because we’re getting stuck with the themes other reviewers don’t want to review, and then we’re being disqualified for reviewing them. How does that make sense?

      Authors should have at least 1 reopen per review, we are volunteering, an occasional mistake shouldn’t be punished, especially on difficult tickets that require more oversight.

      3) Not enough tickets. It’s nearly impossible to get more then 10-15 themes to review per month because everyone is still trying to assign themselves tickets and still selectively picking tickets. We easily could have done another 10-15 reviews this month but couldn’t even if we wanted to.

    • tskk 5:45 am on April 8, 2014 Permalink | Log in to Reply

      1) Selective reviewing, not everyone has the same technical know how, they pick tickets they can review.
      2) If the ticket is difficult, Chip has mentioned to ask him for help before approving the theme, he said he will be happy to help as much as required before approving the theme.

      • CyberChimps 6:27 am on April 8, 2014 Permalink | Log in to Reply

        1) We agree, which is why it is not fair to discredit reviews that require being re-open a single time. If a theme requires that much additional oversight there is typically a reason. Or at least a determination should be made, not just an automatic disqualification. If the reviewer put in considerable time to review a theme and then gets it disqualified that isn’t right.

        Either way, there is still a reviewers cherry picking tickets who are being rewarded for their practices, and those who take on the less desirable tickets are being punished.

        2) We received no help for the tickets that disqualified us out of the contest. The tickets were reopen and we lost. That’s what happened, and will continue to happen to legitimate reviews as long as this rule stands.

    • Vivacity InfoTech Pvt. Ltd. 7:39 am on April 8, 2014 Permalink | Log in to Reply

      Congratulations to all winners!

  • Chip Bennett 8:30 pm on March 3, 2014 Permalink | Log in to leave a Comment
    Tags:   

    Theme Review Incentive: February 2014 Winners 

    As you may recall, last month we announced changes to the Theme review incentive program, both opening up the incentive to more winners, and adding in a measure of accountability.  Per those changes, we are pleased to announce the following winners for February 2014.

    New Themes

    Theme Updates

    Please post a comment below, indicating your Featured Theme selection.

    For the month, there were a total of 88 new Themes added to the directory by 21 reviewers, and 369 Theme updates added by 16 reviewers. The eligibility requirements for this month were 8 New Themes approved, and 92 Theme updates approved.

     

    Featured Theme Rotation

    Per the program changes, all Themes but the following will be removed from the list:

    • Twenty Fourteen
    • Twenty Thirteen

    Note: review totals are taken as a snapshot at an arbitrary point in time, based on this Theme-Trac report, and varying the filters as appropriate.  Totals are as accurate as reasonably possible.

     
  • Jose Castaneda 9:41 pm on February 25, 2014 Permalink
    Tags:   

    Over on the Community section they are looking for input and feedback on user profiles. How do you use them and what changes would you like to see? Go and chime in! :)

     
  • Jose Castaneda 5:45 am on February 18, 2014 Permalink | Log in to leave a Comment
    Tags: , , ,   

    Improving our theme previews 

    As many of you may, or may not, have noticed there are currently over two thousand themes available in the repository. I think that is amazing. Seriously, huge thanks to all those that have contributed not only their time but their efforts as well.

    One thing I noticed some time ago was the mentioning of the theme previews. I can’t recall where it was brought up but I do recall it mentioned that it wasn’t the greatest preview of a theme, or themes really. I do hate to admit it but it is fairly true. The current preview is lacking on some things.

    One of the things being post formats. Currently the theme preview is just a few posts and a few pages. I think we can do a little better now. I’ve brought up a ticket a few times: #30 in the Meta trac.

    Here are some of the things I think we can not only improve upon but can contribute to.

    • Sample data
    • Images
    • Video
    • Audio

    Data

    Simple and to the point. We need posts and plenty of them. How about quick little tidbits like how to set up a front page, or changing the image header. Doesn’t have to be huge.

    Images

    Galleries. Sliders. Single images. We need more boats!! Okay not really but if you have them it would be awesome.

    Videos

    I think we can find a way to contribute a few videos here and there. I know there are some themes that have video format support and I would love to be able to accent that in some way.

    Audio

    I’m thinking podcasters and maybe musicians.

    So there you have it. Let’s discuss!

     
    • Rohit Tripathi 1:51 pm on February 18, 2014 Permalink | Log in to Reply

      +1 to this Idea. I feel Better theme previews are beneficial for the theme authors and users, both.

      • Jose 1:55 pm on February 18, 2014 Permalink | Log in to Reply

        I feel the same way. As theme developers and reviewers, how can we improve it? I’ve somewhat begun that content building: josemcastaneda.com/preview

        I would love to hear suggestions on what else can be used.

    • PeterRKnight 5:58 pm on February 18, 2014 Permalink | Log in to Reply

      This is such a great area to focus on as there is a lot of room of improvement. How often does it happen when a preview comes up really ugly when it is actually a nice theme. Thoughts:

      • I’d love for the preview to occupy the whole screen, maybe using a top bar like how theme shops and marketplaces tend to handle their theme previews.
      • It would be awesome if theme authors could submit their own demo url so that themes with more varied front page layouts can have a more representative way to showcase what the theme can do.
    • Ulrich 6:08 pm on February 20, 2014 Permalink | Log in to Reply

      I would keep it as simple as possible. So you want to just include samples of all of the features in WordPress.

      Style Guide
      Featured Image
      Custom Post Types
      Gallery
      A few images pages
      Three level drop down menu

      I think whatever we add will better then what we have now. The current content is from 2008, so about 6 years old.

      @jcastaneda I would recommend you extend your version abit and submit it. We can always improve on it afterwards.

    • Jan Dembowski 1:01 pm on February 24, 2014 Permalink | Log in to Reply

      Given the large amount of themes available this may not be doable: is there a way to get a theme-name-slug.preview.wordpress.org?

      Sometimes members don’t want or can’t to post the URL to their site. Having an easy to reference theme preview link that’s not in a frame might be handy.

      Apologies if this exists already. ;)

  • Jose Castaneda 10:33 pm on February 11, 2014 Permalink | Log in to leave a Comment
    Tags: , ,   

    What is your working environment? 

    I’ve actually wondered as to what most of us like to use when it comes to reviewing themes and possibly even creating themes.

    One thing I would like to see is the use of VVV. I know most of us have never tried it out or even heard of it but I feel we can all benefit from using it. Why? Because as the video states it does make it easier to be on the same page.

    I would love to see what you all use for your testing and development of themes. Or even plugins.

     
    • zamoose 10:35 pm on February 11, 2014 Permalink | Log in to Reply

      It’s not QUITE there, but I’m getting close.

      http://github.com/zamoose/vagrantrt

      I need to wrangle with rmmcue over how to best do a couple of things upstream in Puppet and then I’ll be ready for a beta release.

    • Zach Russell Philadelphia 11:30 pm on February 11, 2014 Permalink | Log in to Reply

      I have been working on my own vagrant setup as well. I’m sure it’s not as extravagant as the one Zamoose is making (I’m building it with my personal development needs in mind), but i’ll post a link here once it’s release publicly.

    • Rohit Tripathi 12:33 am on February 12, 2014 Permalink | Log in to Reply

      I prefer localhost, but I am definitely trying out VVV.

    • Frumph 2:43 am on February 12, 2014 Permalink | Log in to Reply

      I pay for hosting, so using whatever subdomain I create for whatever project I need – also; it helps keeping a methodology that allows clients/users to see what i’m seeing if I link them, can’ setup vvv on my machine to handle that.

      • rahul286 8:35 am on February 12, 2014 Permalink | Log in to Reply

        We develope on our own localhost and have a demo server with wildcard domains. We have https://github.com/rtCamp/easyengine on server to create demo sites at subdomain using 1-command.

        As of now we are using Grunt, git and some shell script to sync changes from local to remote server sites. Soon we will add this to easyenigne (disclosure: I am from easyengine team).

        This approach help us use a single $5 digitalocean VPS for giving demo of 100 of sites. As soon work is finished, we delete sites using easyengine delete commands.

    • Hari Maliya 8:32 am on February 12, 2014 Permalink | Log in to Reply

      I prefer localhost . but i try to use VVV.

    • tskk 10:49 am on February 12, 2014 Permalink | Log in to Reply

      I use web server, but local host seems to be preferred option.

    • Zulfikar Nore 8:24 am on February 13, 2014 Permalink | Log in to Reply

      For reviews I primarily use a live web server.

      For production its mainly localhost simply for the convenience of being able to readily access the files via explorer for edits.

      VVV looks interesting and some to start exploring soon.

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel