We would like to know: what is it that you as an author don’t like or really want to change around Theme Review and our entire process?
(no personal agendas please, we do see quite enough of that around here lately)
I’d just like to echo what christine said. If a theme hasn’t been updated in over 2 years, it would be great if there was a way to update or adopt it. Since it’s all GPL, as long as attribution remains with the new author and the old author that should be something that’s possible right?
Other then that, improving findability should perhaps be a bigger priority, as well as getting rid of very old themes.
Going over the support forums lately, we are noticing some changes about how the theme author’s prefer to support their themes and many would rather provide a support on their site -vs doing this from WPORG. In addition of having different members answering the questions whether or not they are the right one.
To take the load from WPORG and to help authors maintain a single support site, would it be possible to give them a choice of changing the green button link “View support forum” and also “About this Theme > Support Threads” to wherever author handles the questions?
Possible way to “handle” the requests would be something along the lines of:
Support URI: http://example.com/support/Question is for all, but without knowing what needs to be done in the “background” and if it would be possible at all.
What do you think?
P.S. Also see
Thanks!
My initial viewpoint: No.
In my opinion, if they want to provide support on their own site, then they should host their theme there as well.
Additionally, if they’re not supporting their code on wporg, then we have no metrics by which to gauge them, no data to present to users, and nothing by which users can judge the level of quality of the theme and the support by the theme author.
Having your theme listed on the directory is not a right.
Same goes for plugins.
Chip and I had a nice Skype talk earlier this evening and we were thinking that it would be nice to ask if this is doable. Not knowing what you just mentioned and I personally understand where you’re coming from.
With that being said, how about in addition to what’s currently in, basically WPORG support and an official support, would that not be good too?
Thanks!
When we brought the support views to the plugins last May, this same sort of thing was brought up there as well. I tend to agree with Nacin’s response to that notion.
I’m not against helping the authors do support, but I don’t think that making it easier for them to fragment their support structures is a great idea.
If more tools are needed, then we can discuss and implement them. If more control is needed, then that’s a doable thing too. But simply allowing them to shove it all off to elsewhere isn’t a particularly great idea for the users, IMO.
People ask questions about themes and plugins on the support forums, and were doing so before we had easier forms and locations to do it and to thus categorize them properly. Our adding of the support views to plugins/themes is more of a way to help authors by getting the questions being asked to be visible to them. Removing or redirecting those links isn’t going to cause the same questions to be asked elsewhere, it’s mainly going to cause those questions that will already be on our forums to not be as visible.
You know what, this makes perfect sense. It did not even cross my mind about the metrics and the overall engagement. Not such a great idea after all.
Live and learn I guess 
Thanks,
Emil
One concern with a blanket statement is that we then put upsell Themes in support limbo. Themes from developers who provide commercial support options aren’t supported in the WPORG forums, but upsell Themes are permitted in the directory. So, how do those developers direct users to their support offerings?
Another concern is developers who have a thriving contributor community of their own. (Off the top of my head, Responsive and Weaver come to mind; and I’m sure there are plenty of others.) What is the point or benefit of trying to force such communities to use one support medium over another? How is the end user better-served?
I totally get the WPORG-as-hosting-not-as-listing principle, but isn’t encouraging and facilitating developers to support their WPORG-hosted Themes the ultimate objective? Isn’t the fact that developers are providing that support far more important than the medium they use to provide it?
And the idea of a Support URI actually makes it easier for WPORG support mods to help direct users to the correct support medium, because the Theme developer has helpfully declared where exactly Theme support is provided.
In the end, Theme developers are going to provide support wherever they want, whether we like it or not – and unless we want to try to require developers to provide support (so that we can enforce providing that support in the WPORG forums), saying that WPORG is a hosting, not a listing, site and threatening not to host Themes for which developers provide support elsewhere other than the WPORG forums is futile. (And any attempt to require developers to provide support – in the WPORG forums or anywhere else – would have the single most-chilling effect on Theme submissions of all the Theme Review guidelines.)
So, since we cannot force Theme developers to support their Themes (in the WPORG forums or anywhere else) as a condition of hosting on WPORG, the only ones who ultimately get hurt by not facilitating the developer’s ability to indicate their chosen support medium are the Theme’s end users, and the WPORG support volunteers who have to try to deal with/redirect users to the appropriate support medium.
Thus, we end up with frustrated users and cluttered WPORG support forums with unanswered support questions – both of which reflect poorly on WordPress itself.
OK, I just wrote a very large comment, hit ‘Reply’ and it just disappeared, so I have to be quick this time.
I agree with Emil’s idea s it’s way more convenient for authors to control the support, however Otto’s points make sense for the overall community experience. Regardless, a number of features should be added or improved to these forums so that they could become even remotely useful:
-better notifications support (specifically email for both parties)
-add forums for a theme (news, basic info, advanced customizations, etc)
-better topic status – pins and promoted, closed topics, move to other forums
-images upload
-better profiles (with filtering and so)
-private messages of some point (bbPress had some private discussion plugin or so)
This looks more like a ‘bbPress multisite’ here, but otherwise the information should be hosted on the author’s site and the forums here won’t make any sense.
If you look at the metrics I wonder how much does that tell you about the theme? Here are the top four themes and TwentyTwelve
http://wordpress.org/extend/themes/twentyten
2 of 9 support threads in the last two months have been resolved.
http://wordpress.org/extend/themes/twentyeleven
13 of 43 support threads in the last two months have been resolved.
http://wordpress.org/extend/themes/responsive
83 of 113 support threads in the last two months have been resolved. – Responsive also has it own separate forum with over 50 replies/posts each day if not more.
http://wordpress.org/extend/themes/pinboard
24 of 100 support threads in the last two months have been resolved.
http://wordpress.org/extend/themes/twentytwelve
38 of 129 support threads in the last two months have been resolved.
I also have also seen by providing good documentation it is possible to reduce the amount of support tickets and also be able to reply quicker. The only way to provide documentation would be to host on your own site. The question also arises as to how can I search the previous topics of the theme to find answers.
I think it would be also nice to be able to give developers the option to provide premium support for their free themes.
I think a additional option to point users to the official forum would allow developer to give their users better support. The current forums would stay but with the possibility of pointing users to the official forums.
I was just wondering, does this compare with providing support on http://wordpress.stackexchange.com/. Does this not fragment the WordPress.org support structure?
Something else came to mind. How would the WordPress forums handle Support Volunteers. I have been helping Emil with the support of his theme. I have been given privileges to edit and delete posts on his forum. I wrote a post on the translation of Responsive. I have been continually updating it since then. Would this be possible on the WordPress.org forum?
http://themeid.com/forum/topic/915/translation-responsive-theme/
I am just trying to show some the limits of supporting users on the WordPress.org forum.
I wouldn’t be so quick to drop the idea until we gauge how more people feel about this. I’ve actually wanted the same thing for a long time.
The truth is that it’s actually the other way around. The theme/plugin forums that are on wordpress.org are the ones causing confusion and fragmentation.
It’s sort of a what came first, the chicken or the egg question. Well I’m willing to bet that most developers, designers, programmers, whatever, already have websites, blogs and forums for their many scripts, widgets, plugins, themes etc.
Contributing a theme/plugin to wordpress.org is fun, provides helpful stats and in some cases has monetary gains if you also sell premium themes.
But, I don’t think most authors are very concerned with losing traffic, that’s not the issue. The problem is that most of us already had forums for handling users/customers long before we ever submitting anything to .org. It’s actually better for everyone, authors and users, to redirect to the official forums instead of having them use a half realized unofficial pseudo-forum (in which authors can’t even get proper notifications about new topics, moderate, organize or really manage to any degree like making stickies for important notes [this may be available to plugin authors, but not for themes]).
I actually had a customer make a refund request for a pro theme on the free theme forum. I didn’t see that for 2 weeks. I’m sure they were upset, obviously confused because of the redundant forum. I’ve since started trying to manually redirect people to the official and correct forum as well as signing up for undependable 3rd party RSS feed services that actually deliver email alerts so I can try and keep track of the extraneous forums created on .org. But, it’s all very less than ideal.
There’s a case for both sides, that’s why it should be an option, not every case is the same. It should be either use the .org native support forum or redirect to your official forum. Deleting our own established forums and/or redirecting users/customers to the .org forums doesn’t make any sense and is a much worse, confusing and fragmented method.
Thanks, Bryan
Personally I like the idea of being able to provide a “Support URI” for themes for many of the reasons listed by several posters above (no need to repeat all of them here). I also can appreciate the ideals of having “full” support on the dot-org forums (again for the reasons noted above).
BUT (caps intended) to provide good support on the dot-org forums theme authors (and plugin authors) need more tools and “privileges” to best handle the support requests. Currently what is available is … well … basically not much. Authors for all intent and purpose on dot-org are simply just another poster whereas on their own site they pretty much have carte blanche to do as they see fit. Somewhere in the middle there must be a common ground that allows for all(?) support venues to provide the best support possible.
As noted above(?), the ideal solutions are the ones keeping the end-users the happiest. To which I’ll add, it is not about keeping the authors happy or about keeping dot-org happy. We should be happy to simply have the end-users choosing and continuing to use our works.
Thanks all for the comments.
I think some way to indicate a preferred support site would be the best.
Occasionally, I or one of my support community monitors will check out the WordPress.org support forums for my theme (Weaver), but mostly that forum gets ignored. Whenever it is checked, the answer is almost always to tell them to go to the “official” theme support forum.
As Edward just said, not much support for the .org theme forums. I’d like to see a preferred link, and maybe an auto-reply capability for anyone posting on wp.org to go to the official site instead. (Cause no matter what, if there are two paths, people will sometimes take the wrong one. And it seems that having the wp.org forum option should not go away.)
But as it stands, posting support questions for my theme (and I expect others with their own support sites) on the WP.org forum is simply an exercise in frustration for everyone involved – users because the answers are infrequent, and then just link to the official site, and the Weaver support community because we really feel bad about letting these people down (but checking out the WP.org forum is just not easy to do automatically.)
TRT aside for a moment: I have been pretty lucky as far as the “ghost” topics, moderators are either handling them directly, or users are being forwarded to my dedicated support, which is my preferred method and not just to keep all answers at the one place but also to reduce the load for WPORG too.
When my Theme was released I did subscribe to RSS feed to get any and all notifications once the questions are posted, at the same time Google alerts are big help as well. When somebody mentions my name, ThemeID or my Theme Google would notify me almost instantaneously and that’s how you can stay on top of it.
Each day I personally post anywhere between 80 to 100 posts on my forum, about the same comes from Ulrich and 10 to 30 from the rest of the TID community. There’s however only few on WPORG, mostly because mod sticked one of the topics and users are being redirected anyways.
This is me giving some metrics of my own
It seems pretty overwhelmingly clear that a Support URI option is well-needed. If one isn’t provided then a fallback .org forum is accessible instead seems to be the most logical solution.
So now, it’s just a matter of is this programmatically practical?
@Admins – Just a heads up that notifications for this forum/topic aren’t working.
There appears to be two plugins/features trying to handle this at once (possibly conflicting):
“Notify me of follow-up comments by email.
Notify me of new posts by email.”
(perhaps those actually apply to two separate things, but it just seems like a difference in phrasing)
Thanks, Bryan
I’m with Otto: no, for all the reasons he said, plus a few extra reasons around confusion for users about what is and isn’t official, consistency, etc.
But realistically, this is @matt‘s call. This kind of change shouldn’t be made without his okay.
Theme/Plugin developers aren’t required to provide support as a condition of having their code hosted by WPORG. Thus, my concern is that expecting (or implicitly requiring, as Otto has done via his comment) Theme (or Plugin) developers to provide support via the WPORG forums is impractical and unenforceable, and that it isn’t the developers, but rather the users, who get hurt in the end.
For many of us, providing official Theme support in the WPORG forums works just fine. But for others – especially, the developers of the most-popular/most-used Themes/Plugins – the WPORG forums don’t provide the necessary tools allow those developers to provide sufficient support to their users. If we aren’t going to facilitate developers to provide support via the medium of their own choosing, then we really need to take this discussion over to make/support, find out what tools/resources developers need in order to provide a sufficient level of support to their users, and then implement their suggestions.
@ipstenu would you be willing to lead such a discussion on the Make/Support site?
I was holding of on entering this one since I do have a viewpoint.
My opinion is that if you have your plugin or theme hosted here, users have a reasonable expectation that you provide some level of support on WPORG. Why? It’s the ‘official’ place to go. It’s already a pain to say “Oh sorry, that’s a paywall theme, you have to go there and ask.”
I’m willing to bring it up on make/support, and ask folks what they think would help them, but if I do, it’s best to limit the scope of that question.
“What tools in the WPORG forums would make it easier for you to support your plugin/theme that is hosted in the WPORG repositories? By this we mean moderation options, like closing posts, etc.”
Is that what you’re thinking?
Completely understandable and we’re just discussing that’s all. Nothing can be changed from our end anyways, we don’t have the “power” nor ability to do so
What about revisiting this in a different angle?
Instead of replacing “View support forum” or otherwise obfuscating or redirecting the .org forums in any manner, what about a Support URI simply offering up a new button above and in addition to “View support forum”, perhaps “Official support available” that links to the official forum?
That way, it informs the user AND gives them options. Since users are all that this issue concerns, there is simply no downside. I can’t see any reason why this wouldn’t be a perfect compromise.
Thanks, Bryan
In lieu of a readme.txt file, Themes are recommended to include a changelog, indicating version-to-version Theme changes.
In past months we noticed that Theme Authors either do not have the changelog or the changelog is very poor.
Changelog
Which does not really say anything at all.
Or the changelog that was not updated in months.
Luckily for us we have trac, but it would be easier for us and for users to know what is going on when author releases a Theme.
Please share your thoughts.
Thanks!
I think it should be required.
Personally I’d like to see this as a requirement, but only if the changelog is visible from the WordPress update screen (like they it is for plugins), which I don’t believe is currently the case. Short of that, I’d leave it as a recommendation.
I would happy to do that with my themes.
SVN discourages atomic commits (slooow), so most authors will commit a bunch of changes in one rush … and forget the details. I don’t say it would be better with a faster version control, but there were fewer excuses.
So, as long as SVN is the default don’t make it a requirement.
this is a good point @toscho.
For me, the fact that SVN (and our review system) encourages bundling a bunch of changes instead of atomic commits is *more* reason to require a changelog, not less. I can’t tell you how many theme updates I’ve seen that include a handful of bug fixes and several new features under “updates”, if they document it at all.
I see the need for good changelogs.
My point is: the current system makes it hard to write that automatically. You cannot say to people who are mostly not programmers: Hey, here is a task we made extra difficult, so it is a requirement.
In all honestly, SVN is used only as a *repository* for Themes; it clearly isn’t set up to be a proper version control system for WPORG-hosted Themes. (Commits are irrelevant, since Theme developers can’t SVN-commit directly – and given the way the Extend directory is set up, probably will never be able to do so.)
I strongly recommend using something like GitHub for proper VCS for Theme development. (Or, roll your own SVN/Git locally.)
All that said: I don’t think that themes.svn should be a determining factor here; we’re only using SVN serve tagged versions of Themes
I agree with John, This should be required and displayed as it is for plugins on the update screen. Users should really be empowered to know what is included in an update. This would be especially helpful if they done specific work in any area that has been updated or at least alert them as to where to look the closest to make sure nothing is broken post-update.
I would be more than happy to support a “RECOMMENDED” status for a changelog file, but I would not be willing to make it a “REQUIRED” theme element any time soon.
What I would recommend is theme authors, once they have successfully submitted a ticket for the latest version of their theme, is write (copy and paste works, too) a changelog as the first comment for the ticket. I see this as a more valuable addition than simply including a changelog file on its own.
Well, that provides a useful change log for the Theme *reviewer*, but doesn’t help the Theme *end user*. I think both use-cases are valid with respect to the benefits of having a change log.
From the perspective of ever-improving standards toward best practices, I support moving toward eventually making change logs *required*.
I think the first step is clarifying that a change log, whether as a stand-alone file or as part of readme.txt, is *recommended*.
Also, I think that we should clarify to Theme developers that including a detailed change log is a great way to expedite Theme reviews, because it is a tremendous aid to the reviewer in following/understanding the changes in any given Theme revision.
I agree completely. I think after strongly recommending a changelog, it might be best to have WordPress display changelogs for themes so it becomes a more useful feature. Then later start requiring them.
I think stand alone file would be better, no reason to have a file changing all the time when we could just have a specific file for it. Also we could define out some sort of formatting so that data can easily be harvested from the file.
I would love to have some integration with the WordPress.org page for themes which would give me a bigger reason to document and be more detailed with updates to the theme. Also while we are at it, why not add some information of the updates on the WordPress theme pages (where you pick and choose from the different themes on a WordPress Install)? Reason behind that is because I have people using my themes that have never even browsed to WordPress.org, surprisingly they stay on their website the entire time.
For standardization, and hopefully to facilitate integrating into the Themes’ Extend listing page, I strongly recommend using the Plugin Readme Standard.
If you split the change log into a separate file, I would recommend retaining the Readme Standard markup conventions for the change log itself.
(Note: I don’t follow this recommendation in my own Theme, because I have an HTML-formatted change log that renders in the contextual help tab for the Theme. I do have a Plugin Readme Standard-conforming readme.txt, but its change log is an overview of the more-detailed changes in the HTML file.)
Until themes are treated like plugins with a readme.txt file or a similar system, I’d only go with “recommended”. Outside of that, changelogs are practically useless for a large portion of users. Only those users who are “advanced” enough to look into the theme files will ever see it.
Is it safe to say that we can add a recommendation after the Theme was submitted? (Theme Check part).
That might encourage authors, or at least remind them about it.
And also what’s preventing Themes to be treated like Plugins? I am asking because I don’t really know what’s behind
That would be very helpful when Theme updates are available. Right now the link WordPress includes is leading to the listing in directory and that’s a dead end for users because we can’t include anything but description.
Thanks everyone,
Emil
I say we make it a recommendation but making it a strong recommendation. Have the reviewer state that for next submission strongly advice a changelog to facilitate the review process.
I would agree with @Emil Uzelac at most points specially with the point “themes to be treated like Plugins”.
Is not the formula of ‘what’s behind’ for WP plugins landing page is BBPress plus Plugins with function to pull data from separately hosted svn ? We basically can change ‘some data’ through that svn commit with our plugin repo. A plugin developer actually never get access to Plugins webpage directly (that is normal). The probable difference among the Themes and Plugins is the mother template. Themes has some extra and less things compared to that plugin page.
WPORG Git : Actually a WPORG hosted git would solve the versioning and detailed wiki issue. Usually people knows about git more than svn on average. The problem is not simple to solve. Github’s self hosted software costs a bomb and not under GPL 2.0 or later. Free Git softwares are actually not of same quality like GitHub Enterprise. Known free options are Gitorious and Gitolite.
So possibly changing that Themes template might work. May be a trial can be run to test.
Something to add, what would happen if a theme is completly rewritten and only say 20% if left from the last version? How would you put that in the change log?
most definitely, but maybe not all, link to svn might be good
How can we improve the Theme Review process? Please share your thoughts.
It has been awhile since I last did a theme review but I remember that though links are provided check what to clean there isn’t a single page that lists everything that needs to be checked.
I had to jump around between a few pages to make sure that everything was checked off and accounted for. Would make it a lot smoother if it was all on one page.
Hi Jarret,
You mean the trac itself?
Thanks for the comment,
Emil
I just looked at the theme review codex doc – http://codex.wordpress.org/Theme_Review
It looks like it has been improved quite a bit since I last reviewed a theme. I guess you can disregard my comment as I hadn’t checked it before commenting
No problem, I am glad that you commented
I think, for new reviewers, this is still a valid concern. The learning curve is still quite high. I’m beginning the process of creating a step-by-step how-to (and possibly even a review checklist) that will hopefully make the process much easier.
Where is the like button when you need it?
good stuff! and by the way, we can add like button as well 
Hi Emil,
I agree with Jarret. I was also theme reviewer in past and I always try to review themes now below are some key points I found while working in theme review process.
1. if somebody submitting theme for review after 1-2 attempt it should get priority instead of going back to the beginning.
2. Don’t de-prioritize existing themes in the queue upon newer submissions.
3. we can initiate a user rating system where if somebody has very good reputation in theme repository we can approve their theme quickly by giving them priority.
Thanks,
Himanshu
Hi Himanshu,
Thanks for the comment.
When Themes are submitted they will be in Priority #4 (New Themes, Never Reviewed) and if is not approved and upon second submission Theme goes to Priority #3 (Previously Reviewed, Not-Approved Themes), so they are not sent back.
As far as I know, there’s no de-prioritization, do you mind giving us some examples?
Reputation -vs newer submission is something I personally see as an unfair “rule”.
Just because “author A” is more reputable than “author B” should not give “author A” more “rights” in the trac
Emil
Actually, if Theme developers leave a comment pointing to a newer ticket, then the new ticket will take the old ticket’s place in the queue. That’s always been in place. That said, here are some ideas I’ve been considering to propose:
1) Have a reviewer follow a given Theme through the entire review process, from ticket to ticket.
2) Have some way of establishing *trust-worthiness*, so that developers who earn the privilege would have their tickets auto-closed as approved (leaving only the final, admin review upon pushing live in Extend).
I don’t know if I like having reviewers being dedicated to a theme. Sometimes a fresh pair of eyes can do wonders and I think various reviewers with their individual review-style improve the overall quality of the submitted theme.
I like 2). Though I fear it will increase the Admin-bottleneck as you will have to spend more time on reviewing those themes.
I always like when other people to go over my codes, just to make sure that I have not missed something, same applies for Themes as well. That’s bit harder when solo reviewer does it, don’t you agree?
“trust-worthiness” is definitely yes from me
I do like the idea of one reviewer throughout the entire process because then it can make seeing what was fixed that much easier.
As far as “trustworthiness” goes what would be the pre-requisites? Must have minimum three themes in the repository? Must be active within the community? Don’t get me wrong I love that idea but wonder what it would take to get that status as well as maintaining it.
I’d suggest not having one reviewer per ticket; I think it would result in 1) creating bottlenecks of volunteer availability, and 2) less active reviewers. When I decide to review a theme, it’s because I have a bit of time that day, for that one review. If I was “on the hook” for a full review process without knowing what that process would entail time-wise, I’d be a lot more reticent to grab tickets.
Hi Emil.
Thanks for your reply I agree with you on first point but some what disagree on reputation. It is not giving rights it is giving them credit for contribution to wordpress. we can process their submission in separate queue.
Meanwhile I found one 404 error while visiting following page
http://make.wordpress.org/themes/about/trac-ticket-request-queue/
the most recent queue hyperlink is pointing aug 2012 queue which doesn’t exist. I think we should replace that link with oct link.
Thanks,
Himanshu
I completely understand that. What I meant was not “rights” literally, however credits do sound better for sure.
We’re open here and also a team where decisions should be made collectively
I’m not fond of having a new thread every month, for that reason. And it must be more difficult to admin, as it means dealing with two threads at month.end.
SVN access to theme creators with stipulations (rules) or they lose access and have to return to the theme review process.
Theme Review team privately chooses who can get SVN access to their themes and ‘suggests’ to Otto who gets the deciding factor.
There are a couple dozen theme creators who I would suggest right away as they are upstanding members of the WordPress theme development community.
Hi Phil,
I guess this should be discussed with Otto, the subject is beyond TRT’s reach I’m afraid
Thanks,
Emil
You asked though
Yes sir I did
Direct SVN-commit access is possible, but due to the infrastructure between SVN and Extend, it wouldn’t actually work. With Plugins, SVN is scanned automatically on a periodic basis. The Theme directory, however, works differently. All of the synchronization scripts happen *only* when the “Make Theme Live” button is submitted in the Extend admin.
As an alternative: what if we could come up with a way to establish a “trustworthy developer” privilege, such that when such a developer uploads a Theme, the Trac ticket is created and then auto-closed as approved, leaving only the step of an admin making the Theme live in Extend?
D-v-l’s advocate on “trustworthy developer” status/privilege – Nobody’s perfect. Even wordpressdotorg’s theme team (take twenty twelve) expects Theme Reviewers to check for something they have overlooked and I consider them at the top of the “trustworthy developer” totem pole.
If themes submitted were perfect, it would take a few minutes to review. It’s the theme which has issues that take the most time to review – includes screenshots and code snippets where required.
It is already to the credit of the WPTRT admins that the review rules no longer require complete reviews before tagging the theme not-approved, since this has facilitated the review process. We could further streamline and expedite the process by prioritizing and setting review stages.
For example, some companies receive project proposals or feasibility studies by the hundreds each week and they need method/system to screen proposals or studies efficiently and effectively. The proposals/studies submitted should follow the guidelines set by the company in presentation and content which should include contact information. So here’s the process:
Stage 1
Stage 2
Stage 3
Stage 4 – etc.
“Ease of Access” for “trustworthy developers” has always been a touchstone ideal from the community … this definitely needs to be a take away point from this thread.
so how do we propose that? if everyone else agrees let’s add that our list!
I think it would be a great idea, if we have checkboxes in the trac for required thinks or something like this. We can check them and maybe a full review could be generated in a few minutes.
Funny you should ask! I’m working on just such a checklist (albeit not in Trac), to see if it would be feasible.
As Frederik mentioned at #wpcs, having a single trac number per theme would make it easier to follow the review and changes, but I don’t know what the logistics of that would involve. It seems like it would make it simpler, but perhaps there are programming limitations, otherwise, I’m sure someone would have done it by now.
As a theme developer (and a new one with only 2 themes in the repo) I personally would love to have a more thorough review. But again, Theme reviewers are volunteers and there’s only so much one can ask for.
I am a strong believer that single trac would speed up the process for sure, but at the same time new reviewers and old ones as well would need to pay extra attention, if this was new or previously approved Theme.
I have to admit to the status of lapsed theme reviewer and the reasons for that aren’t just due to a lack of time unfortunately. I mean this all with a positive mindset though and would love to be part of the solution not a nay sayer.
Aside from time the biggest hurdle for me getting back into reviewing themes was the biggest hurdle stopping themes. That is simply a lack of clear boundaries and vagueness on what should / shouldn’t pass. Yes, a lot of it is self education but there needs to be tightening up so it’s at least accessible to first users or those dipping back in.
Themes are not rigidly set and I don’t argue they should be so easy to pass / fail as a simple checklist. It’s a guide. But, the point is the guide was a bit too vague. It makes it both hard to start as a reviewer and to come back when you’ve had a period of absence (life and contributions to other areas happens).
I’d like to see some form of regular theme review meeting if possible. I know that’s not the easiest thing in world but it does help.
Perhaps also those of us who are lapsed but had review status could get a few mentoring checks on our reviews (or easily request it) when we do come back. I’d like to request that now for myself.
That is actually the point of our new posts, the idea is to bring reviewers step closer to each-other. And it’s understandable for volunteers to prioritize their day-jobs first
Actively following the theme-reviewers mailing list is also low-hanging fruit for keeping up with theme review. That’s where I would post questions, ideas or concerns about evolving best practices, etc.
I think the process we have now is fine, just need more volunteers.
Some sort of incentive to volunteers should speed things up
Thanks for the comment, more volunteers are always a big plus!
I think we all agree that more volunteers would be great, but speaking personally, it was the complexity and seeming vagueness of the review process that put me off getting involved for years. And having now completed a few reviews, I’m still not clear. What’s more, how do I know if things have changed since I last reviewed a theme? It feels to me that I’d need to do a few hours research just to get back up to speed before even requesting a theme (and then there’s a wait to be allocated, and by that time, the window of opportunity/motivation has perhaps passed… couldn’t this be automated?)
I’m pretty new to the theme review process, but here are my two cents anyway .
1) “Trustworthy” Developers
I’m actually not in favor of this, because I think that automatically auto-closing as approved is problematic because of two reasons.
First of all even veteran developers still can make mistakes like using a deprecated function, writing non valid HTML&CSS or forget to include licensing information for a bundled ressource. So a reviewer should look at the theme, and as they are good developers, these reviews would be done in a couple of minutes.
Secondly I also fear that this might divide the theme development community into those whose themes get approved on the fast track and those who have to wait in line.
2) Different review stages
I like mercime’s idea a lot, because the Stage 1 review of verifying the licensing, credit links, theme name and screenshot could be done really fast and it would help cleaning the theme review queue of spammers and other malignant developers.
The second step would be also pretty fast since it involves little work besides downloading the theme and doing the checks with the plugins. This would mean that only the “worthy” themes make it to stage 3, in which a reviewer has to dig in and do all the manual checks that are needed.
3) The learning curve
I agree with Chip that the learning curve is steep. My main motivation for joining the WPTRT was to learn more about themes, as suggested by Matt in the State of the Word 2012.
There is a lot of self-education involved which is great for improving your skills as a WordPress developer (which happens to be my main job ) but it also means that you have to search for information if you are not clear on an issue.
In all of the cases in which I wasn’t sure if something was permitted I found the answer in the WPTRT mailing list archives. But searching in those archives is a real pain, so some kind of FAQ style ressource with categories, tags and a search function would be a better place to store all this knowledge.
For beginners it might be good to get coupled with a mentor when they start out reviewing themes. This mentor would be an experienced theme reviewer that could then help out with the first reviews of his protégé. Just like Emil mentioned, coders appreciate peer review, so I think for reviewers this might work as well.
4) The community
Building a community is always a good idea! Besides Chip and Emil, I don’t even “know” any of the other theme reviewers.
So maybe we could interact a little more besides the mailing list, maybe by having a theme reviewer’s chat like the core developers do?
As far as the incentives mentioned by Tskk are concerned, I think the biggest reward might be appreciation from the community. If you are a core contributor, your name is in the release notes, if you are a plugin developer, your name is displayed in the plugin repository, etc. Not sure how this could work out for theme reviewers though.
The guidelines should have to be reduced. In half. Or at least the reviewers should follow more simple rules.
The current guidelines are very detailed, as such they are suitable for (kind of) WP standards for best practice but make review process slower and tedious. But to stimulate developers to stick to detailed guidelines there might be some ‘reward’ like badge in theme’s page or something else indicating that this theme is compatible with WP standards of good practice and quality.
How I imagine the process: It should be divided to two parts. The main will check for general basic things like the theme have to contain such and such files, such and such functions, no malicious/obfuscated code, no affiliate links and so… It it fulfilled the requirements make it pass.
The second part is up to author – if he/she wants may ask for full review according to WP guidelines, and if the theme pass it will receive a mark for quality.
More important than reducing the number of guidelines is identifying tiers of approval. Mercime suggestion is a great idea, especially for new theme submissions. At the present with some tickets waiting over a month for a review, I find it’s faster to just do a complete review since the developer’s been waiting for so long.
The majority of issues that I’ve found are for things readily discovered in the Theme Guidelines and Theme Unit Tests. So is it that developers are not reading the guidelines or are those guidelines confusing?
This is where a regular reviewer’s meeting would be good so we can discuss the common problems we’re having.
I’m not in favor of having a “Trustworthy Developer” tag which would allow the developer to auto-approve their themes. It’s always good to have a second or third pair of eyes on your code – everyone makes mistakes from time to time.
1) Create a list of guidelines and update it whenever a change is agreed upon in the mailing list.
2) Encourage reviewers by giving them priority review for 3 of their themes for say 100 themes they review
3) Theme authors to take a quiz of the guidelines to submit their theme. ( So the incoming themes will be already following the rules )
Number 1) is essentially what we have now: the Guidelines are established, with a defined update mechanism and frequency. (The Guidelines are updated with each major core update, unless security or a similarly major issue necessitates an immediate revision.)
Number 2) would be very difficult to implement without encouraging quantity-over-quality reviews. It also moves us into the subjective area of favoritism, and would need to be handled very carefully in that regard.
Number 3) is, for the most part, covered by the Theme-Check Plugin, which the uploader script runs against the Theme before completing the upload and creating a Trac ticket.
2) You can decide on boundaries for it, it will be nice to have that privilege
It will definitely encourage more reviewers but logistics has to be worked out.
Thank you all for comments, I see many great ideas around, should we maybe start getting the list together?
It’s interesting reading this thread and all the comments about how vague and confusing it is to do a review. I think it’s heavily tied in to the same frustrations that submitters face when trying to make their code acceptable to pass a theme review: documentation of the guidelines.
I think a lot of these issues would solve themselves if we could present the guidelines in a clearer, more effective way. Each requirement could have 1) “What”: what is the requirement? 2) “Why”: Why is this a requirement? Best practice? Security? Breaks plugins if not followed? 3) “How”: How do we properly write this code? Actual code examples and, eventually, a demo site and demo theme downloads that show these requirements in action. In my head I even see each requirement displayed with those three tabs, with the “What” tab showing by default.
My problem with the guidelines is the same as my problem with the Codex in general: a huge, dense, mind-numbing block of text that makes most people run screaming. The guidelines have served well up to this point, but for theme review to grow and sustain itself, an overhaul of how we present the guidelines would be a huge bonus to reviewers and submitters alike.
Is it necessary for a Person to Provide a Theme URL for the theme to be approved ?
No. You can optionally omit both Theme URI and Author URI, though it is *recommended* to include at least one or the other.
I love how extend/plugins gives “Requires” and “Compatible up to”. Reviewers would have a better idea of when to give an approved theme a closer look (changing guidelines, possible errors missed by previous reviewers), and users would have a better idea of the theme’s compatibility with their own install.
A small thing that would help a lot would be to have a link directly to the ticket in the emails we receive from Trac.
All WordPress.org passwords are reset due to suspicious commits earlier today. You will need to request a new one to regain an access. read more
Danielx64, 
unsalkorkmaz 11:48 am on February 28, 2013 Permalink |
Waiting.. Waiting is too long. I sent a theme for review and waited 3+ weeks. Theme got denied because of 1-2 little things that can easily fix in 1 hour… Even you fix it, you need to wait another 3+ weeks for second review.
At least there should be a chance to fix quick like in 12 hours ? I mean give theme author 12-24 hours before decline theme?
Chip Bennett 6:34 pm on February 28, 2013 Permalink |
But if they were such simple things, why were they not fixed before submission, or during the three-week waiting period?
One thing that the Admins are adamant about is that Theme Reviewers not be used as “quality control”. The vast majority of the time, simple things that obviously don’t conform to the Guidelines are indicative of a developer not doing due diligence before submission. So, that’s why we no longer require full reviews before closing a ticket.
Is it possible to overlook things? Sure. But as you’ve seen, the turnaround time for a review is already long enough as it is, and having to weed through the crap submissions just to find the Themes submitted by developers who honestly have made a good-faith effort to conform to the Guidelines is often like trying to find a needle in a haystack.
Allowing a grace period to fix and resubmit is an option entirely at the discretion of the Theme Reviewer. At this point, I don’t think it’s beneficial to *require* Theme Reviewers to provide a grace period, because it would get abused.
But if you’re able to address issues quickly, and re-submit, it never hurts to comment in the original ticket, indicating that all issues have been addressed, and linking to the new ticket.
Paul 7:30 pm on February 28, 2013 Permalink |
yes, I did that when I submitted my last theme. There were only a few minor issues that I was able to fix straight away.
Paul 12:10 pm on February 28, 2013 Permalink |
Well, if anything goes, then I don’t like the email list for communication. I’d prefer forums.
Another thing: when you submit a theme for review, there’s a period of waiting. If you upload a new version in the meantime, there’s no easy way to manage that.
An idea: how about a multiple choice quiz about the Theme development guidelines that any person submitting a theme should take. Like a test for being able to upload a theme for the first time.
Srikanth 12:24 pm on February 28, 2013 Permalink |
I suggested that once, it was shot down
SriniG 2:12 pm on February 28, 2013 Permalink |
Quiz or no quiz, I think there should be more information in the theme upload page than there currently is. Specifically, information that there could be a waiting time, probably a link to the themes trac and that newer versions of the same theme should be uploaded via the extend/themes/upload/ page rather than as attachments to the ticket to the first version, things like that. I see many new submitters getting confused on these points. Just a little information on the process followed can be helpful. Also emphasize the point that themes that don’t meet up with the requirements in the theme review page are subject to disapproval.
I don’t think there is any difficulty in managing that. Simply upload a new version and provide a link to the new ticket in the original ticket. But as I said above, it helps let the new submitters know about these processes and methods.
Mahesh 5:11 am on April 9, 2013 Permalink |
I agree with the “more information in the theme upload page”. As of now there isn’t much transparency for them to make an estimate of how much time it would take for the approval.
Chip Bennett 6:10 pm on February 28, 2013 Permalink |
Can’t the Trac ticket, with comments, be used for that purpose? Note that any reviewer can comment/provide input/feedback on any ticket.
ryanve 6:42 pm on February 28, 2013 Permalink |
@Paul Agreed. A forum would be better searchable in Google and more in the vein of opensource. IMO using Github issues would be ideal. I’d love to see WP ditch the trac and use Github (https://github.com/WordPress/WordPress) to greater effect.
Chip Bennett 6:57 pm on February 28, 2013 Permalink |
A mail-list isn’t ideal for everything, but it is useful for some things. For as much as is feasible, we’re doing our best to move toward most of the communication happening here on the Make/Themes site.
toscho 9:32 pm on March 1, 2013 Permalink |
The mailing list suffers from full quotes. These create many duplicates and make search unnecessary hard.
The Make-sites are too restricted. Very few people can open a new thread here.
Something more community-friendly would be nice.
Paul 7:32 pm on February 28, 2013 Permalink |
yes, it would be nice to have a pre-screening on Github! I like that idea. then once it’s ready, submit it to trac
Greg Priday 12:25 pm on February 28, 2013 Permalink |
Some things I’d really like to see:
== A Vote Based Review System ==
I’d like to move over to a system where each theme submission becomes a community discussion. This would allow anyone to point out issues, offer help and ultimately vote for or against inclusion in the directory. P2 would be a great platform for this.
Benefits:
Allows anyone to start checking new themes without committing to a full review.
More public theme review process would help educate reviewers and developers.
Faster feedback for developers. Waiting 1 month for each round of feedback can be frustrating.
== More Automated Tests ==
Theme-Check is great, but we could do with a few more automated tests. I’ve extended theme-check to include a whole bunch of extra tests. They aren’t perfect, but they’re a massive help. I used these tests to review ~60 themes on trac last week.
I think as a community we could automate 90% of the theme review guidelines. We could even include the unit tests with a little help from PhantomJS.
Paul 12:48 pm on February 28, 2013 Permalink |
maybe you should work together with Mario, he has a theme mentor plugin that adds extra checks
Chip Bennett 6:28 pm on February 28, 2013 Permalink |
My preference would be for any Theme-Check extension to be pushed back upstream into Theme-Check itself. Theme Mentor is great, but the more we can do with fewer Plugins for Reviewers to install, the better.
nofearinc 5:23 pm on March 1, 2013 Permalink |
(wrong account switched above by mistake with the reply below)
Chip, I replied back to your comment in my blog as for the reasons I would keep the Theme Mentor separate. It’s an open discussion, though I still believe these cover two different aspects of the review process.
Chip Bennett 7:34 pm on March 1, 2013 Permalink
No problems there; but anything that is developed using Theme-Check’s designed extensibility would be most logical pushed directly upstream to Theme-Check, rather than being ported to a separate Plugin, IMHO.
nofearinc 2:02 pm on February 28, 2013 Permalink |
I’m slowly working on the Theme Mentor as Paul mentioned above, it’s purpose is exactly pointing tests based on existing reviews with about 70% success rate (better to have ‘see this’ without guaranty instead of missing details while reviewing code).
The project is also on GitHub and collaborations are welcome.
Chip Bennett 6:25 pm on February 28, 2013 Permalink |
Vote-based system:
Implementing a vote-based system would bring things to a grinding halt.
Currently, community discussion is entirely possible – right in the Theme-Trac ticket generated for the submitted Theme. That process is entirely public, because anyone can read the tickets, and anyone with a WPORG username can comment on the tickets.
More people doing piecemeal reviews would introduce further delays, and would result in more issues falling through the cracks, because there’s no guarantee that they were ever checked.
More automated checks:
Patches welcome.
If you have extended Theme-Check, please push those improvements back upstream. It was written the way it was specifically with this purpose in mind.
Ideas for further automating or otherwise facilitating the Review process are always welcome.
Greg Priday 8:07 pm on February 28, 2013 Permalink |
Yeah, I think I have a fairly idealised view of how voting system would work. Alos, getting it right would also probably take a lot more effort than it’d be worth.
I’ll put some work into my theme checks and try submit a few patches.
konradS 1:38 pm on February 28, 2013 Permalink |
It should be possible to have several persons working on themes!
christine 5:17 pm on February 28, 2013 Permalink |
That might get a bit confusing.
Instead, I would love to work on abandoned themes. If the dev hasn’t updated the theme in more than 2 years, then I would love the ability to adopt it and resurrect it.
Stephen 2:36 pm on February 28, 2013 Permalink |
As an theme author, I would like to see same and consistent treatment regarding review guidelines. My theme was asked to remove Author URI as it against the policy – personal site or development site. I removed it in the latest release.
But this is still bother me as why many themes can have the companies as Author URI. Technically their Author URI is the URI for their Premium or PRO themes, not the URI for Lite theme in repo.
I am not here to question their business model. The key point is same treatment.
As a theme reviewer, I would like to see repo promotes Open Source not just GPL. I think the repo is slowly (or has ) become a marketplace.
christine 5:16 pm on February 28, 2013 Permalink |
I think that Stephen has a point. I’ve been asked to removed things from themes when previous themes were fine. I understand that reviewers are human and make mistake, but perhaps having a thorough list of review items and final score would be a good idea.
Say, there are 100 things that need checked, when you have your final review, you can see the full list with comments to go along it.
Chip Bennett 6:17 pm on February 28, 2013 Permalink |
Your point about consistent reviews is certainly valid. That’s why the Guidelines read as rigidly as they do – so that they can be as objective as possible, in the hopes of ensuring the most consistent and least subjective review possible.
But reviewers are human, and are all volunteers. If you disagree with a review comment, there are recourses (discuss in-ticket, escalate to theme-reviewers mail-list, etc.).
As for your specific issue: it was discussed in-ticket, and on the mail-list, and the reviewer made the correct decision. Author URI is reserved for the *personal/development* site of an individual developer who submits a Theme, or the Theme shop landing page for a Theme shop that submits a Theme.
Author URI should not be used for an up-sell page. Nor should the Theme URI be used for an upsell Theme promotion, for that matter.
Handling of credit URLs is one of the most difficult aspects of Theme review, both because of the inherently subjective nature of the criteria, and because credit links are the single most-likely aspect of a Theme to be intended for abuse.
Stephen 8:01 pm on February 28, 2013 Permalink |
My case was closed. I believe it is correct according to the guidelines. Can I implement it this way?
IBM, you are a consulting form, not a theme shop. Therefore, you cannot be the author of GPL-Licensed theme.
Satish Gandham 4:14 pm on February 28, 2013 Permalink |
Selection of featured themes should be transparent. It should be based on communities choice rather than one single person making the call.
Featured themes section should be updated regularly and frequently to give a fair chance to all the themes and developers.
christine 5:14 pm on February 28, 2013 Permalink |
I agree with Satish, the featured themes on wordpress.org is pretty poor. In fact much of the theme section on the wp.org site is poor and there was discussion of this at the community summit. WP.com has a great way to feature it’s theme which is way more visual stunning.
Having said that, this has nothing to do with theme review, so it’s probably best to continue this discussion at another time.
Chip Bennett 6:19 pm on February 28, 2013 Permalink |
We’re going to have a discussion on this. It is a lower priority at the moment, but we’ll get to it.
ryanve 6:50 pm on February 28, 2013 Permalink |
The prescreen is too strict. I think that the only required functions should be the hooks: wp_head() and wp_footer(). There are valid reasons that someone would want to avoid direct use of echoing functions like body_class(). These would be better as warnings rather than fails. Related: http://core.trac.wordpress.org/ticket/23537
Chip Bennett 6:55 pm on February 28, 2013 Permalink |
What are the “valid reasons” not to require body_class(), post_class(), comment_class() (via comment_form()), etc.?
Justin Tadlock 12:16 am on March 1, 2013 Permalink |
Frankly, the nit-picking of every little item and spending days arguing over them is tiring. It feels like we spend more time debating inconsequential things than reviewing. It’s a big turn-off from the review team.
For example, the theme name guideline we just discussed the other day. That should’ve been resolved by just using a little common sense. I see no reason why it warranted a major discussion in the mailing list.
Emil Uzelac 12:54 am on March 1, 2013 Permalink |
Human nature is to agree and disagree and that’s something we need to live with. Once this discussion is over we will gather your thoughts and act upon them accordingly and hopefully few things will change.
Discussions are good and never a turn-off
So far this is what we hear:
Wait Time
Forum -vs List for Communication – (I am in favor of this and something we talked with all Team Reps last month)
More Consistent Reviews
Featured Themes should be on the list, but I do agree with Chip that this is not priority. And just to note that one person does not make this call, it’s between all of us here.
Justin Tadlock 2:07 am on March 1, 2013 Permalink |
I’ve got nothing against discussions and didn’t claim that they were a turn-off.
It’s just tiring to see some of the seemingly endless debates about things that are of little consequence when it comes to whether a theme works or does not work.
Anyway, I’m just answering the question.
Emil Uzelac 2:14 am on March 1, 2013 Permalink |
I know
Daniel 2:47 am on March 1, 2013 Permalink |
Another example is branding/demo content, some common sense should have been used there as well.
Daniel 2:45 am on March 1, 2013 Permalink |
One thing that I would like to see if a checklist of things to check before uploading in a format where we can print then out.
Maybe it just me but I sometime find it easer to have a checklist on paper than trying to have both theme and checklist up on screen at the same time.