We’ve moved the SVN and Trac firehose mailing…

We’ve moved the SVN and Trac firehose mailing lists to lists.wordpress.org, from the legacy lists.automattic.com. If all goes well, we’ll move the rest of them over as well.

The one side effect is this is likely to break your existing mail filters. The new mailing list emails are wp-trac@lists.wordpress.org and wp-svn@lists.wordpress.org. If you’re using Gmail’s mailing list filters, those would now look like list:wp-trac.lists.wordpress.org.

For those of you who use Gmail, I’ve also added basic styling to the SVN commit emails. Hooray for reading diffs with red and green. (For those who don’t use Gmail, you’ve already been seeing styling that Gmail strips out.)

Just a reminder you don’t *need* to use the Trac firehose. You can also subscribe to individual tickets, milestones, components, focuses, new tickets, etc.

Another note: WordPress.org is now forced SSL. Announcement and details here.

gmail-diffs

#mailing-lists, #notifications, #svn, #trac

A New Frontier for Core Development

In a little over a decade, we’ve made twenty five thousand commits to WordPress. WordPress (along with the web itself) has come a long way, but our development workflow has remained largely the same.

As a part 3.7, I’ll be leading an effort to revamp and streamline our development workflow. We’re going to bring all of our core components—our code, our tests, and our tooling—under one roof. Developers will be able to use and improve the tools we’re already working with day-to-day, and we’ll be able to add new tools to make working with WordPress even easier.

We’re also making sure that any changes are compatible with our current workflow, so you won’t have to change the way you work. These changes won’t break any existing checkouts or scripts that use core.svn.wordpress.org. Now that we’ve cleared that up…

develop.svn.wordpress.org

This week, we’re creating a new repository: develop.svn.wordpress.org.

The develop repository will be the new home for all core development. We’re going to sync over all of the commits from core.svn.wordpress.org and will reorganize this new repository to use a new directory structure:

The src directory

This is the WordPress core source. All of the existing WordPress core files will be moved into this directory. You’ll develop and test against this directory as usual, but the real power will come from using the rest of the tools in the repository. The contents will be optimized for development, not production (this means uncompressed files—including our internal libraries).

The tests directory

We’re going to pull our internal unit tests into this repository as well. This will make testing a cinch, and allow us to commit patches that include changes to both the source and the unit tests. We’ll also add JavaScript unit tests during the 3.7 cycle.

The tools directory

Our tools directory will include all of the little things that make our lives as developers easier. To start, we’ll include our internationalization tools, which will allow us to generate pot files from within core. We’ll also include a handful of scripts that some of us use to interact with Trac, like scripts to upload and apply patches.

The root directory

This frees up the root directory to include configuration files, allowing WordPress to interact more seamlessly with other applications and services like Travis CI and Composer.

A build process

We’ve had an implicit build process in the WordPress codebase for a considerable time now. Most of that process is now contained within bumpbot, a little script that lives on the WordPress.org servers. Whenever we update CSS or JavaScript files, bumpbot comes around, minifies them, and commits the result. Despite bumpbot’s efforts, this process is less than ideal. Our source files (in this case, JavaScript and CSS) live alongside our production-optimized files, which causes confusion and makes contributing to core considerably more difficult. Our built output—the files optimized for machines—shouldn’t live alongside the source code that’s optimized for developers, and beyond that, shouldn’t be tracked in source control.

We’re adopting Grunt to write a real build process and help us manage and run our various tools. Grunt is a JavaScript task runner that has a robust ecosystem of plugins focused around building modern web applications. We can include tools such as integrated testing, minification, linting (for PHP, JS, and CSS), image optimization, and LiveReload—and that’s just the beginning.

Using Grunt will be an optional but encouraged part of WordPress development. These tools are all built atop Node.js, which Grunt needs in order to function.

So what’s happening to core.svn.wordpress.org?

In a word, nothing. The contents of the core repository have always been optimized for production, and that’s not going to change. core.svn.wordpress.org will now serve as the repository for our built output: every time we make a commit that changes the build, the core build will be updated with a corresponding commit. As mentioned earlier, scripts and checkouts will continue to function as expected. When you download a zip of WordPress, it will still be generated from this repository.

Let’s get started

The develop repository will go live soon. For now, I’ve made a preparatory repository available to illustrate the structure I’ve described above, and allow us to experiment with this new workflow. You can learn more about how the tools are installed and used on the repository page. Please provide feedback on this post—the goal is not to build all of the code there first, but to provide insight into the new process.

Lastly, I’d like to say thank you to all of the contributors who helped shape this idea over the past few months, especially at WordCamp San Francisco’s hack day. Together, we’ve created a structure that elegantly addresses many of our existing problems and opens the door to countless possibilities for better development practices.

#svn

For those of you who receive wp svn…

For those of you who receive wp-svn commit emails: I’m considering a simple change to reduce some of the noise on that list. The change would be to not email any commits by bumpbot or potbot.

One of the original goals of bumpbot was to reduce the noise when doing code review on a commit that involved JS or CSS. But these commits can occur fairly often and still cause noise.

The concern is two-fold:

  1. Will skipped commit numbers cause people to look around for them, so instead of saving time, it actually increases time spent on these?
  2. If a bot screws up and no one is around to hear it, does it make a sound?

I think #1 is surmountable. #2 can be solved in a few ways. One, the commits will still be posted to IRC by svn-bot, and viewable in Trac’s timeline. Two, we could continue to have them sent to a few people, if necessary. Also, these bots have made few mistakes over the years, as every time there’s been a bug, we’ve fixed it. Of course, it’s all just VCS, so any issue can be reverted. And we have protection in place around releases to make sure the bots did their jobs and “run clean”.

Anyone have a strong opinion either way?

#bots, #mailing-lists, #svn, #wp-svn

Our main SVN server is undergoing maintenance early…

(Maintenance started around 8:18 AM UTC and finished around 9 AM.)

Original post. Our main SVN server is undergoing maintenance early shortly. It should take less than an hour (once it starts). During that window, no WordPress.org Subversion repositories will respond. The exception is http://plugins.svn.wordpress.org, which is on a different server.

The 3.5 Beta 1 and Twenty Twelve releases are delayed until after this is done, so see you all late Thursday morning.

#svn

Plugin committers now receive svn notify emails with…

Plugin committers now receive svn notify emails with every commit to their plugin. This is something we’ve been planning to do to assist with collaboration, but we added it today without extra things like being able to sign up for other plugins. (Look for that soon, though.)

And not to sound like the PA in a subway or at an airport, but if you see something, say something. Say things to security@wordpress.org.

#plugins, #svn, #wordpress-org

Commit access has been restored for plugins If…

Commit access has been restored for plugins. If you get a 403 error, you need to go reset your password. Please, to something new.

#plugins, #svn, #wordpress-org

I just moved WP.org plugins SVN and Trac…

I just moved WP.org plugins SVN and Trac to a new server. What does this mean for you?

1) If your DNS resolver doesn’t honor TTLs, you may see a “down” message for http://plugins.trac.wordpress.org and you may get an error message when committing to http://plugins.svn.wordpress.org

2) Core WP trac shouldn’t be so slow anymore. About 80% of the resources on the server were being used by plugins core + trac. I plan on moving core WP trac to a new server soon-ish, but in the mean time, it should be much snappier.

3) The SVN server version is now 1.5 instead of 1.4 (yes, I know – still behind. May upgrade later)

4) I may have completely broken something 🙂 I don’t think I have, but if you find plugins SVN or trac completely broken, please let me know here!

#infrastructure, #plugins, #svn, #trac