Plugin Reviews Disabled (And More about the Support Forums)

Reviews will be broken until about September 5.

This is directly related to the support forum maintenance.

Per @jmdodd:

We’ll do our best to keep this window short, but for now the choice was between closing reviews for 4 days or closing all of the support forums for 24 to 48 hours.

The Meta team felt (and I personally agree) that it is far more important to have support forums than the reviews. And the support forums were unsustainable. So while this is a wrench in your plugins, it’s far far better than no forums at all for Labor Day Weekend.

Updated Sept 2 0233

From @otto42

Consider this an announcement: all plugin/theme connections to the forums are currently considered broken. We expected that. It will take a few days to restore this, and that’s considered acceptable losses, for now. We will be working to fix these issues over the next few days, and it will be corrected as we get to it. In other words, we are aware of the issues and working to fix them.

Updated Sept 2 1628

You may have noticed you can’t do some things in the forums anymore. This is known. Please read Forum Bugs and Broken Things before you complain. Here’s a list of what you’re probably trying to figure out. ALL of these are being working on. Don’t fret. Enjoy your weekend.

  • Plugin authors can’t sticky
  • Plugin committer/author support views don’t work
  • Plugin authors can’t resolve threads
  • Pinned topics are unpinned in plugin forums
  • Plugin Authors aren’t labelled as Plugin Authors
  • Cannot subscribe to plugin forums

#forums, #reviews

Reminder: Do Not Compensate Reviewers

It was brought to our attention that some plugin developers on WordPress.org have used various third party services to find new users for their plugins and to have them leave reviews on our site.

It’s time for a reminder.

We do not allow for compensated reviews to be on our site, by any means whatsoever, and consider those reviews to be disingenuous.

The WordPress.org plugin and theme directories are for users to write their experiences, not for companies to use market their products. A compensated or recruited review should be posted on someone’s own site, the reviewers own site, or the 3rd party site itself.

While you may not consider getting a product free (or at a discount) to be compensation, we do. It messes up the system, which really is meant for people who legitimately use a plugin to leave a review of their experience. It’s also misleading, in our eyes, because it was not made by an actual user of the product in question.

Asking an existing user to leave a review is one thing. Emailing your user base, while possibly annoying to many people, is totally fine. Reaching out to new people and saying ‘please try and review’ inflates the number of reviews in an unnatural manner.

You may have heard about how Amazon does permit reviews like that, as long as the reviewer “clearly and conspicuously disclose[s] that fact” in their review. We’re not Amazon, and being a much smaller community, we’re able to monitor our reviews in a tighter manner. Paid reviews, compensated reviews, or recruited reviews are all the same idea. You’re ‘paying’ someone to review.

The Consumerist has a long article about this, asking Is Amazon Doing Anything To Fight Latest Wave Of Fake, Paid-For Reviews? This article illustrates the issues these kinds of reviews cause, primarily they break the trust a reader has in any review. Also keep in mind that companies like Yelp hire people to blacklist companies who reward people for leaving reviews.

This is just something you should avoid and reviews that are found to have been compensated for will be removed.

#bribery, #reviews

Plugin Review “Inconsistencies”

A few people have complained that they feel the review process is inconsistent. I’d like to take a moment to explain exactly why that happens. The tl;dr is, of course, humans make mistakes. But if you really want to understand what’s going on, read on!

There is no automated review process

This is the big thing. Every single plugin is opened and read by a human being. We download the plugin, read it, and try to catch the myriad things that are wrong, insecure, not permitted, etc. And we’re humans. We do our best to scan/grep for things we know are easy to find (like I love checking for wp-(con|load|blog) to see if that’s being called). But a lot of times things are buried or hard to catch.

This means mistakes are made. We don’t claim to be perfect. We claim to try our best to give you the best review we possibly can for your sake, as well as your users.

Some replies are canned, the process is not

I’m sure a lot of you have gotten an email starting with this:

There are issues with your plugin code. Please read this ENTIRE email, address all listed issues, and reply to this email with your corrected code attached. It is required for you to read and reply to these emails, and failure to do so will result in your plugin being rejected.

Yes, that’s a canned auto-reply. In order to get through reviews faster, we have replies for the common issues. Right now I have 60 in A-Text. That means there are at least 60 problems with plugins I see every single day.

This makes us able to keep up with reviews. It’s impersonal, we know, but we try to cite examples from your plugin to help you understand what needs your attention.

We don’t test your plugin on all environments

Sometimes we do. But really that’s your job, not ours. We do if we notice things that are weird and we think may be problematic. Some days we test on VVV with PHP 5.6, sometimes it’s HHVM, and sometimes its PHP 5.2. Why? It depends on what we have available just then.

This means sometimes we catch that you coded something for PHP 5.3 and up and sometimes we don’t.

Every new version is checked top to bottom

Think about that for a second, please. If you submit a plugin and we pend it for changes, and you send us the new version, we read the whole thing all over again. Every. Single. Time. We check to make sure you did your changes first, yes, but then we go back and re-read everything to make sure we didn’t miss anything, or you didn’t accidentally add in something new.

This is why, sometimes, you get an email that starts with “We missed this before…” or “This is also not permitted because…” We’re giving you the best review we can.

No, we don’t list everything wrong

It’s not what you’re thinking. Every time we do a review, we list everything we see that’s wrong. We do not list out, for example, every instance of a non-sanitized/validated POST call. We do not list out every single usage of script tags instead of enqueues. We will give you an example, especially if you miss some on your first edit, but we expect you to know how to search your own code.

This helps you learn how to better vet and review your own code. Also it saves us a little time.

There are multiple people doing reviews

Some of us are better at some thing than others. When we find a plugin we don’t feel confident in reviewing on our own, we raise a flag and ask our cohorts to spot-check our work.

This also lets us hand off troublemakers. Let’s be honest here, folks, we don’t all get along with everyone. When it’s clear we’re at an impasse with someone, we ask each other for help.

Our goal is protecting users first, then you

The people we care about the most are the users who can’t code or who don’t understand the severity of things like offloading CSS. You may think it’s trivial and makes your plugin smaller. Someone in another country could find them sued for not disclosing it. Or your plugin may not work because Google is blocked where they are.

We care about protecting the users from XSS and SQL injections. We care about protecting their information. We care about keeping them safe. But we care about you too! We’re so techy about you documenting ‘This plugin calls service XYZ’ because, yes, the users have a right to know where their data is going, but also because you deserve not to have a slew of angry 1-star reviews that you didn’t tell them.

This is a tricky road to walk. Some people may get exceptions. Some people may teach us more about code! Some people may be told ‘no’ flat out.

Guidelines evolve over time (so do security best practices)

We’re constantly looking over the guidelines and evaluating them for clarity, consistency, supportability, and real-world applicability. Have you read our Detailed Plugin Guidelines lately? You should. Similarly, our security checks have gotten better over time. We used to allow you to call wp-config.php directly. We don’t anymore. The more a specific vulnerability is targeted, the harder we are on your code to ensure you are not the weakest link.

This is for your protection! We’re doing our best to make sure you don’t get dog-shamed for being the reason sites go down.

Remember: We are mortal

I said this to start off this post and I’ll say it again. We, your review team, are human beings.

We make mistakes. We miss things. We read code incorrectly. We don’t test everything as fully as we should. We screw up. We never miss things out of maliciousness or an intent to blacklist you from the repository. We believe you submit your plugins in good faith, and we respect you enough to treat you as adults and point out what you missed or explain how you can do things better.

This means you should give us the same benefit of doubt we give you.

#reviews

Ratings Rebuilt

Did your ratings suddenly change dramatically? Hopefully not, but if they did, it’s because the ratings for all plugins were recently reset and rebuilt earlier this week. All ratings now correspond exactly with existing, non-deleted, reviews.

As Otto put it:

Back when we launched the review system 2.5 years ago, we tied ratings to reviews. However, up until that point, we had existing ratings in the system. At the time, some argued that the ratings should be wiped and everybody start fresh. I argued for the opposite, that we should leave the existing ratings in place until such time as we had enough reviews in the system to build up a good body of ratings.

That time has finally come. What you see now is the ratings that correspond to your reviews. The data comes directly from the reviews themselves, and is accurate. Any ratings previously left over from the pre-review world are no longer available.

Additionally, the ratings now will accurately reflect the actions of the moderation team. If a review is deleted for whatever reason, then the associated rating for it will not be reflected in the results.

Please keep in mind, this means that all of the people who thought making sockpuppets to spam the reviews with 5-stars on their own plugins (or 1-stars on their competitors) have had the biggest swings. It should go without saying that you should never leave multiple reviews on your own product (we’re pretty sure you like it 😉 ) and you should never attempt to hide behind proxies and fake accounts to leave reviews. Be honest. It works out better.

#directory, #repository, #reviews