Ulrich 3:19 pm on March 27, 2017

Chat with Matt about the future of theme repo

On Friday evening, @jcastaneda, @poena and I met with Matt to discuss the future of the theme repository on WordPress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/. @greenshady had to drop out last minute since Matt wanted to do a voice call and Justin was unable to due to his internet connection. We asked Jose to join so that we would not need to delay the call any longer.

Matt wanted to know more about the Theme Review team. We mentioned our plans with automation, problems with the previews, and how the portability of content affects users. We also mentioned that the common issues in themes are security, code errors and prefixing.
A suggestion that we got back was that we should check if themes could be prevented from being activated if there are PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. fatal errors, like it is done with plugins.

The reason for having the meeting now and not at the community summit is so that we can start working on improvements before the community summit.

Matt’s goal for the theme repository is to make it the main place for users to search and find themes.
Matt is interested in seeing how the repository reacts if we remove the manual review process and switch to a process where user feedback helps rank the themes.
Amazon was an example where search results are influenced by different metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. data including reviews and how reviews affect users buying a product or not. On Wix you can search for nearly any term and a related theme is shown. That should be possible on WordPress.org too.

Another comparison was made between Google and Yahoo where Yahoo used to have a directory of links that were reviewed before being added to the list but Google indexes all links but shows the best at the top.

Allowing users to give feedback in different formats like adding tags or giving a tag a thumbs up or down if the tag was correct or not thus allowing bad themes to drop and good themes to rise. Users should be able to decide how much upsell is too much.
An example of this is how Google Maps asks users for feedback after visiting is place.

Matt asked us how the featured themes/homepage functioned and we explained that they were random themes that have been updated in the last two years. As this list of featured themes are not always the best we need to come up with ideas how to improve it.

As we are not sure if the process will function without manual reviews, we will start working on getting better user feedback on themes. Once we have a good infrastructure in place we can experiment with how the repository reacts with no manual reviews.

We discussed the process we would go about making decisions on changes to the theme repository and came to the consensus that a direct democracy is too fragile and representative democracy would be a better solution.

We were given a few tasks to complete and then get back to Matt with the results so that he can see where the resources are needed.

Tasks:

Matt has asked that we choose a few people who will represent the team and make the final decisions.
The representatives are people who can be contacted to get feedback from and provide feedback to.
To create a list of blockers or resources that are needed to improve the theme repo.
Once we have a list to contact him so that he can assign the necessary resources.

A few things that we discussed as potential places where we can improve on.

SVNSVN Apache Subversion (often abbreviated SVN, after its command name svn) is a software versioning and revision control system. Software developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS). WordPress core and the wordpress.org released code are all centrally managed through SVN. https://subversion.apache.org/. access
Multiple screenshots
Improved search to be able find a theme related to a certain topic for example “Thai Restaurant”.
Improved previews that display the theme at a basic level and allow theme authors to show their own theme demo.
Showing more information about the theme, like the readme and changelogs.
Allow a way for users to report bad themes, but also if a theme has a certain feature.
If we allow themes to have content then we should allow custom post types and taxonomies as they are better than pseudo custom post types.

Milana Cap 5:02 pm on March 27, 2017

..how the repository reacts if we remove the manual review process and switch to a process where user feedback helps rank the themes..

Hopefully this is not too ignorant question but, aren’t these two different things? Review and ranking?

I wonder how will this effect Review team. I just love reading #themereview slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel – so many things to learn every day. It would make me very sad if that would be lost with this switch.
- Ulrich 5:42 pm on March 27, 2017
  
  Hopefully this is not too ignorant question but, aren’t these two different things? Review and ranking?
  
  Yes they are different but ranking can hide bad themes.
  
  I wonder how will this effect Review team. I just love reading #themereview slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel – so many things to learn every day. It would make me very sad if that would be lost with this switch.
  
  This is something we would need to look into. There may be an optional review process that gives a theme better ranking.
  - Milana Cap 6:19 pm on March 27, 2017
    
    Maybe human interaction can switch focus to educating/tutoring authors. This would provide continuous sharing knowledge and experience and would be extremely beneficial for community. I know it would for me.
    
    Maybe “Upload” right should be earned.
Joy 6:08 pm on March 27, 2017

I wonder if themes are submitted in waves, a bunch knowing how to do things, and then a bunch not knowing. I looked at 160 new themes in the past month (actually running them), and it seemed like problems ran in waves. There were maybe three themes that did not have problems in use. Five had fatal errors.
Over that month I have seen the top part of that list start to be reviewed, and they all have the prefixing, escaping, sanitization errors that we look for. If all of those were uploaded into the repository without all those fixes, the user would have a terrible experience, and would definitely look elsewhere for themes.

My opinion is that switching to a theme hosting site instead of a theme repository would greatly reduce the value of WordPress as a solution. To quote the Developer’s Handbook, “This review ensures that WordPress users across the globe can download themes that are high quality and secure.” And the Getting Started page says “The goal of the theme directory isn’t to host every theme in the world, it’s to host the best open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. WordPress themes around. ”
I don’t see how you can live up to those standards without review.
- Ulrich 6:11 pm on March 27, 2017
  
  I don’t see how you can live up to those standards without review.
  
  We would not, that was how the theme repo was when it was small. We need too how the theme repo can scale for the future?
ThinkUpThemes 6:34 pm on March 27, 2017

Complete automation without any review sounds good on the surface, but it wouldn’t take long for the repo to be filled with spam. Even if we do go to full automation, I’d like to see some form of limitation to deter spammers and those who want to clog the repo. Limiting to 1 / month could be a crude way to address this.
vovafeldman 6:58 pm on March 27, 2017

I think that Matt’s call moving towards “crowdsourced” curation driven by users’ reviews is a great direction. It will make the Theme Review work so much easier and will also give theme authors more freedom.

One of the things that I keep asking myself is what’s the official affidavit of responsibilities of the themes review team. From what I observed, currently, the team is making sure the theme is secure, following coding standards, but also enforcing restrictions on content (e.g. the commercial related stuff – a practice that isn’t governed by the plugins review team).

Is it the team’s responsibility to make content related decisions based on opinion vs. delegating that to the community?

For example, last year a new pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party was released to the repo that adds an oEmbed support for adult sites. Some people in the community thought it’s inappropriate and that it should be removed from the repo. The decision that was made (and I totally agree with it) is that the plugin serves a purpose/need – whether we like it or not, there are many adult sites using WordPress. Therefore, there’s nothing wrong with that. If someone is offended by that, they can provide their feedback with a review.

Same thing regarding how aggressive developers with their in-dashboard upsell marketing. Many people in the community believe that Yoast has a very aggressive admin notices engine, having said that, there are over 6M active sites using Yoast and it has an avg. rate of 4.8 stars.
- Carolina Nymark 2:14 am on March 28, 2017
  
  The content creation requirement is affecting a very small number of themes considering the amount of themes submitted so it’s not a main concern or a focus right now.
Edward Caissie 7:52 pm on March 27, 2017

Although it’s great to see @matt taking some time to get involved with WordPress themes and the theme repository it saddens me to think it may be too little too late with the loss of so many great reviewers and advocates no longer being involved with the theme review team.

I do hope it improves the process enough to at least get rid of the “one theme per author in the review queue” rule as I will continue to stand by my position of not even providing an update for any of my existing themes (let alone publish any of the new themes I am working on) until such time that rule is repealed.
- Diane Wallace 8:26 pm on March 27, 2017
  
  I know that previously when automation of the theme review was discussed, it was suggested that users (and possibly themers) could tag a theme to say it needed a review, could that work to help keep the quality up?
Joy 9:30 pm on March 27, 2017

Part of the review process involves making sure the proper copyright and attribution is used for all the parts of a theme. There are so many that just copy code or fonts or images and change the names.
There are also a lot of themes that think they are plugins. I don’t see how the review could be removed altogether, since plugins have at least some review.
dmccan 10:22 pm on March 27, 2017
It seems like there are several things being discussed:

1. Theme standards
- I think we want the TRT to set the standards for the WordPress repository.
2. Review process
- Continue to optimize the process and experiment. I agree that some review is important.
3. User feedback / problem reporting
- In this context I think user feedback is in relation to theme standards, otherwise it will be all over the place. With standards you can have a “report issue” form with a drop down to identify the problem area.
4. Theme repository UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. (searching, better demos, and more theme info)
- Why are we expecting the TRT to design the UI and end user experience? I think we have tended to start from our current limitations and challenges when imagining the theme repository, and this has held us back. If Matt can contribute resources, then maybe a designer to do mock-ups of what it should look like and how it should work, would be a good start. Then you have specs and the programmers can make it happen and the TRT can update the requirements to reflect the changes.
5. Allowing custom post types
- Yes. Start with Portfolios and Testimonials. If we define the field names for “standardized” custom post types, do we solve the issue of theme portability (at least in these cases)? If your theme doesn’t have a Portfolio custom post typeCustom Post Type WordPress can hold and display many different types of content. A single item of such a content is generally called a post, although post is also a specific post type. Custom Post Types gives your site the ability to have templated posts, to simplify the concept. then install Jetpack or a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party that implements the standard.
- Carolina Nymark 1:57 am on March 28, 2017
  
  Admittedly it is a bit scary to remove the reviews.
  The theme directory can’t become
  “the main place for users to search and find themes” if authors keep submitting untested, low quality themes.
  Authors HAS to step up and improve their themes.
  Hopefully these new user ratings will motivate authors better than we have been able to.
  Today we see authors making required changes to for example security: but only to pass the review.
  Then they do the same mistakes in their next theme, and the next.
  They need to learn and understand that the problems with their themes affects users.
  
  Anyway I believe we will find a middle way where we have the new theme check pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party run on theme upload to check security and code quality, and then have a manual review of security, license and accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility).
  
  We also need a way to prevent all these theme copies from being submitted.
acosmin 7:09 am on March 28, 2017

Some spammers can be so creative with code, I know because I did a lot of review. If you want examples 🙂 you can check the SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel.

The review process should stay in place, with checks for security and license information. The rest of the issues should be solved by authors based on bad reviews or reports by users.

The Featured page should disappear or make it curated based on merit, by a team. Just do it, set some rules, because if you ask authors about it, they will never agree with any solution (no one wants to see a competitor on that list).

For the Popular page I have not solution 🙂 it’s a fiasco. A priority would be UUIDs for themes, solving the active installs issues.
williampatton 11:48 am on March 28, 2017

I have some questions specifically about the talk that I would like to ask for clarity. I’ll try to keep them short and to the point.

The discussion about automation: What was the general feeling about that? is it still something we are pursuing with any degree of importance?

No timelines were mentioned here other than ‘start working on improvements before the community summit’ – which means tackling some things in the next 2-3 months. Were any other timelines for individual items discussed aside from that?

Also regarding timeframes and the tasks that Matt wants us to complete. How quickly are we to come to a decision regarding those (team leads, identify blockers, find where best to allocate resources)?

Was there any mention of changing status of current projects? For example requirements rewrite and review flow projects seem like they wouldn’t currently fit into the new vision of a more open submission policy and the user based quality assurance system.
- williampatton 11:52 am on March 28, 2017
  
  Sidenote: could we possibly collaborate with the Plugins team and get some insight into how they manage SVNSVN Apache Subversion (often abbreviated SVN, after its command name svn) is a software versioning and revision control system. Software developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS). WordPress core and the wordpress.org released code are all centrally managed through SVN. https://subversion.apache.org/. access and detecting fatal errors in plugins before activation?
- Ulrich 12:39 pm on March 28, 2017
  
  We did not go into any detail on the automation other than it could help other companies with their code reviews.
  
  There are no timelines yet as we are still at the defining the problem stage of the project. A post will be published soon which will go into more detail on the team leads and the general team structure. I don’t expect the process taking more than a couple of weeks.
  
  Identifying blockers and getting back to Matt with a list of item should also not take more than a couple of weeks.
  
  We would also potentially to get someone who would lead the project and coordinate between the different projects but that is someone we can discuss once the goals for the project have been defined.
  
  We have not fully discussed the how this affects current projects. Some processes will stay the same other will change.
  
  I don’t think we need to be concerned how SVNSVN Apache Subversion (often abbreviated SVN, after its command name svn) is a software versioning and revision control system. Software developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS). WordPress core and the wordpress.org released code are all centrally managed through SVN. https://subversion.apache.org/. access is given as that would be managed by the people on the systems team.
  
  As how detecting fatal errors in plugins before activation, you have a look at the code in WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. which is OpenSource and public 😛
  - williampatton 2:05 pm on March 28, 2017
    
    Happy to hear the expectation is to move quickly on some items and to aim for a few weeks instead of a few months.
    
    I’m looking forward to the coming meetings where we’ll be able to make some decisions about how to move forward with the future direction 🙂
    
    P.S. I have looked at the code for how Plugins detect errors and it’s a simple idea (an output buffer that’s tested for content when it should be empty). Even though the idea is generally quite simple I would still appreciate their insight into this as they obviously have a lot more experience with it than I do.
Sallie Goetsch 4:18 pm on March 28, 2017

I’m all in favor of reducing the load on the theme review team, which should be possible by using appropriate tools to detect the most common (or at least the most egregious) errors.

If the developers submitting the themes used proper debugging tools, most of those would be caught long before the theme was submitted. I know there’s a theme check pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party aimed at helping theme developers; I wonder about adding tutorials for debugging to the theme developer handbook.

Crowdsourcing the review process is much more problematic, in my mind. It’s difficult to get users to engage in the review and rating process that we have now. Look at Twenty Seventeen: 1 million plus installs, and 39 ratings. That’s not a statistically significant sample, but more importantly, it’s a good indicator that there won’t be a lot of people jumping on board to help with quality control.

I do think that @dmccan‘s suggestion of a “report issue” button that gives users the ability to select among possible errors or violations is a good one, though the average end-user (or, at least, my average client) is not likely to understand where the problem comes from, especially if there’s no visible error message. That’s likely to mean s/he will either go to the support forumSupport Forum WordPress Support Forums is a place to go for help and conversations around using WordPress. Also the place to go to report issues that are caused by errors with the WordPress code and implementations. and say “this isn’t working” (harder to tabulate and darned hard to search) or go to a consultant for help.

But as a developer working on client sites, I might well take the time to report a specific issue I’ve noticed, since I’ve also consulted support forums on behalf of clients who have a problem with a particular plugin or theme.
Ulrich 7:41 pm on April 4, 2017

I just had an automation idea.

The automated system can detect potential issues but there still needs a manual review. The idea is that for each potential issue there is an ability for a reviewer to make that issue as a false positive and have it ignored for future updates. Then the theme author can see the actual list of issues.

We could use this list of issues to help lower the ranking of the themes without kicking them out of the repository.
- Joy 2:55 am on April 5, 2017
  
  That’s interesting. It would be like a history that stays with the theme. I suppose users could see it? or just the ranking?
  Wouldn’t this also be similar to what we have now, in the form of “exceptions”, as different reviewers mark different things as false positives based on their understanding of the guidelines?
- Sanyog Shelar 8:46 am on April 14, 2017
  
  Great idea

Welcome to the Themes team!

Handbooks and review requirements

Coding standards and plugins

Code Packages

Communication

Chat with Matt about the future of theme repo

Tasks:

Welcome to the Themes team!

Handbooks and review requirements

Coding standards and plugins

Code Packages

Communication

Tasks:

Share this: