The number one question asked at the summit of me was “Can I join the plugin review team?” I told everyone “Follow make/plugins and I’ll post there by [last] Friday.”
Sorry about the delay, we had coordination issues which ironically is why the current answer is “No, we’re not adding anyone new to the review team.”
State of Things
The way the review of new submissions is sorted right now, it’s a single-thread system. There is a single queue that contains all submitted plugins, and it can only be viewed by one person at a time – or we run the risk for two people reviewing a plugin at once, which can end with one rejecting and one approving simultaneously. In order to avoid this, we are constantly asking each other which of us is currently in the queue. Even then, the system is archaic and has issues. So yes, it’s entirely a technical limitation and it’s one we ARE actively addressing. We’ve all talked (in one-offs) about what we want and need, and we have it spelled out. A lot of this is because we were intentionally waiting for the inevitable bbPress 2.x upgrade, but since that’s not happening any time soon, we’re going to have to make an interim plan.
What We Do
But there IS a future where we will want more people to help out in various roles and it’s with that in mind I want to talk to you all about what we actually do.
Review New Submissions
This means we download a submission, check it for any violations against the guidelines, test it on a sandbox of our own, and make sure there isn’t anything egregiously wrong. We also have to check for licensing and trademarks, which leads to fun things like the time I rejected the Official Facebook plugin because they used a gmail email address and a dropbox URL for the zip.
Right now, the check is 100% manual. We’re developing a Plugin Checker (like the Theme Checker) but it’s much harder since themes are pretty standard when you compare them to how crazy plugins get. We have, finally, boiled down to what we know we can auto-reject and what we need to warn/inform people about, so we’re making progress on that end.
One thing we don’t do is put our own feelings into a plugin review. If the code is good and there’s nothing ‘morally offensive’ about it, it comes in. That’s why we have a bajillion twitter plugins. Determining what is and is not offensive is hard, though. We don’t allow things we determine to be black-hat SEO (“This plugin will improve your SEO by 1000%!”) and we don’t allow things we feel would be detrimental to the community, but we do allow things we know will offend some people. It’s a fine line.
Handle Guideline Violations
Every single email you send to plugins AT wordpress.org saying ‘So and So’s plugin puts in powered by links!’ has to be verified. Usually this is easy, but once you report one user, we check all of their plugins. This can take a while and it gets worse when we get a submission like “Joe’s twitter plugin emails him when installed!” Sounds easy, right? Go on and figure out how many twitter plugins that might actually refer to. I reply to those a lot and ask “WHICH plugin? Please link to the repository page.”
What we really need is simple.
1) A link to the plugin page (ex: wordpress.org/plugins/evil-twitter/
2) A clear explanation as to what’s wrong (ex: The widget puts in a link for non logged in users)
3) Optional: A link to where the evil code is (ex: https://plugins.trac.wordpress.org/browser/evil-twitter/trunk/index.php#L2 )
With that it speeds up everything.
Handle Security Reports
Everything we do in the guideline violations has to be done here, but worse, we have to reproduce the bug and give suggestions/information about possibly fixing it. Why? Because not everyone actually understands why they have to sanitize, or why their plugin which we approved 4 years ago, calling wp-load.php directly, needs to remove that now. The guidelines and standards change over time, and while we don’t expect people to keep up with them 100%, when they do change, it’s a waste of time to argue with us that they changed… Which bring us to the number one thing we actually do.
Be patient with angry people
If you’re not good at handling support tickets or forum posts, I have news for you. You will not survive the plugin team. Getting sent the dread “Your plugin has been removed…” email is possibly the worst day for a plugin developer. It’s earned us a lot of anger from the community, from people who feel we single them out or that we specifically hate them. We don’t.
Just because you’re the most awesome person when it comes to reverse engineering security issues and solving them doesn’t mean you’re great at explaining to people why they can’t phone home or why something that was okay 4 years ago isn’t now, or even teaching them how to fix an issue even if it’s not actually our responsibility. And yes, people absolutely lose their minds to the plugin team fairly regularly. Buy me a coffee, I’ll tell you about the guy who tried to impersonate me by sending emails ‘from me’ telling other devs their plugins were removed, because I’d closed his.
The point here is that we really need people who either are great communicators from day one, or who are comfortable asking for help when they know someone’s gone off the rails and can’t be reasoned with by them. If you’re this guy, you’re not ready:
So … now what?
Well now we just want this post to get you all on the page we are. And we want you to understand that until we fix the technical issues, we can’t actually address the training people up to help out. I promise you, I’m just as riled up about not having more people on the team as you are, because right now if two of us go away for a week, we have a massive queue which is just depressing. Trust me, we’re all in agreement here. But since they won’t let me reboot the plugins directory, we’re going to have to take this seriously and careful, and I beg of you to be patient.
And that’s what we need most of all. Be patient. Stick around here. Be understanding. Don’t nag. Seriously, that never helps. We know who’s interested, and maybe we’ll come up with some quizzes and tests to see ‘Would you approve this plugin?’ and sort folks out even more. But it’s not today, and it’s not because we don’t want more people. It’s because more people would break a broken system worse.
And that is your state of the plugin review team at this moment.