WordPress.org

Ready to get started?Download WordPress

Make WordPress Core

Updates from August, 2013 Toggle Comment Threads | Keyboard Shortcuts

  • Andrew Nacin 12:13 pm on January 29, 2014 Permalink
    Tags: ,   

    Please propose agenda items for the development meeting today at 21:00 UTC. Let’s try to keep it to under an hour, like last week’s.

    Some to start us off:

    • Consider the Widgets Customizer plugin for 3.9 merge. If you haven’t looked at it yet, please spend some time today before the meeting. Obviously, we all like the idea, but we need to study the implementation and provide feedback. Please comment on that post if you haven’t already.
    • I’d like to propose an initiative for cleaning up a group of 556 tickets during the 3.9 cycle. (It has details, will explain in the meeting.)
    • Quick situation reports for various 3.9 tasks. If you’re working on something, please come armed with a sentence so we can do these rapid-fire — or post it in the comments thread. Some things are moving forward quickly, some aren’t. Where can we help? Here’s a list of what was proposed for 3.9 (see also the comments, which were updated after the last meeting). We need to take the “proposed” label off soon…
    • I’d like to get moving on the Trac component re-organization; I’ll see if there are any questions. We will need some help to move around tickets (ideally adding feedback to them along the way), so I’ll be looking for volunteers and talking about what is needed.

    Aren’t familiar with how these meetings work? They’re especially good for breaking logjams — whether that means talking through a major sticking point, finding volunteers to help with something. We also use them to plan out schedules and tasks, make sure everyone is on the same page, and do release post-mortems to see how things went. (As 3.8.1 was released on Thursday, could be good to discuss this and 3.8.2, for example.)

    The meeting takes place in IRC on chat.freenode.net in the #wordpress-dev channel. See you then!

     
    • William Bowles 12:27 pm on January 29, 2014 Permalink | Log in to Reply

      I like it! One of the problems with WP has always been the amount of time spent moving between the various components that make up the site. I’d like to see a lot of the backend stuff move to the Customize sidebar and the popout access to other widget tools seems to work perfectly.

    • Tom Auger 2:06 pm on January 29, 2014 Permalink | Log in to Reply

      I have moved forward with https://core.trac.wordpress.org/ticket/21811. Still trying the time to upload an actual patch, but could use some eyes on the last two plugin ZIPs that were uploaded to see whether there’s any buy-in on my proof-of-concept approach to modularizing the Image Editor.

    • Tom Lynch 2:43 pm on January 29, 2014 Permalink | Log in to Reply

      I’d suggest considering talk around some of the older parts of wp-admin that have static coded HTML tables that don’t have any hooks or filters, nor use the Settings API to actually make it possible for people to develop management plugins.

    • John Blackbourn 3:11 pm on January 29, 2014 Permalink | Log in to Reply

      We should have a think about how we can start to build this new “easy first bug” report on Trac.

    • Jen Mylo 3:28 pm on January 29, 2014 Permalink | Log in to Reply

      GSoC 2014. I’ll post about it on this blog before the meeting, but would like to take a few minutes during the chat to discuss areas of interest for projects and people who’d be interested in mentoring/co-mentoring so I can get the application started.

    • Andrew Ozz 4:16 pm on January 29, 2014 Permalink | Log in to Reply

      Decide on JavaScript actions and filters, https://core.trac.wordpress.org/ticket/21170. We need JS hooks in core, either our own implementation or standardize on jQuery( document ).trigger( 'event-name' ). I know @koop had more thoughts on this but he hasn’t been around for a while.

    • tomdryan 4:17 pm on January 29, 2014 Permalink | Log in to Reply

      I may not be able to make it today, but I would like to propose creating a new task force that can look at existing plugins that would be good candidates to get moved into core. Most of the core efforts appear to be in developing new functionality, which is critical, but there are existing plugins that also fix gaps in the core feature set. The goal would be to identify those plugins and approach the authors to see what there interest level is in contributing their code to core. This approach unitizes WP excellent plug in architecture and plug in community to move WP core forward more rapidly.

      I love to hear comments about this idea (post here). I’ll try to make it to today’s meeting, but if not, I love to talk about this subject in the near future.

    • Bryan Petty 6:07 pm on January 29, 2014 Permalink | Log in to Reply

      I have an upstream update to PHPMailer in #25560. There are a couple decisions that still need to be made: include the full library, or strip it down without docs, tests, and examples (both patches are there), and a quick review of the approach on back-compat. This does need to in early in the cycle though, so about now is a good time, and hence why I bring it up now.

  • Andrew Nacin 9:01 pm on January 27, 2014 Permalink
    Tags: ,   

    Proposed Trac component reorganization 

    Warning, this long. tl;dr: I propose a reorganization of our Trac components. 34 top-level components with two dozen subcomponents. New tree at the bottom. First, an overview of some of our problems.
    (More …)

     
    • Helen Hou-Sandi 9:22 pm on January 27, 2014 Permalink | Log in to Reply

      I am particularly excited about how groups and maintainers will (hopefully) form around components and focuses. Anybody can comment on a Trac ticket or pitch an idea or create a potential roadmap, but there’s a real sense of empowerment when you really feel like you’ve got your head and hands wrapped around a specific area, and are perhaps even recognized for doing so.

      I know for me personally it’s been really gratifying to be entrusted to run with a particular area (what we can now define as the UI focus) and has made me feel more comfortable with interacting and pushing core forward. And not just when it comes to UI development and conception, but in other areas as well. I also might just dare hope that we can stop the worst of the ticket rot, with having both more bodies as well as a clearer idea for hopeful contributors as to where to go for more feedback or help.

    • nofearinc 9:23 pm on January 27, 2014 Permalink | Log in to Reply

      I’m curious about the Query branches of Comments and Users – too many tickets for these types of Queries or plans to extend a lot furthermore?

      Great list though, and the “hall of fame” with notes is awesome.

      • Andrew Nacin 9:28 pm on January 27, 2014 Permalink | Log in to Reply

        I don’t think either is necessarily the case. The idea was to create a specific subcomponent under Users that those focused on Query (in general) could also specifically follow. Same for Comments. So in this case, it’s about granularity, not necessarily roadmap or ticket counts. Query is goofy, as I mentioned in the post — it’s a cross-section that would normally be good as a focus, but it actually covers functional areas of core (there just happen to be a lot of areas).

    • StyledThemes 9:24 pm on January 27, 2014 Permalink | Log in to Reply

      Give me a BIG Pot of coffee and I will ready this shortly…however, I saw the part of Bootstrap which caught my eye as I build my themes with it (well, parts of it). But from what I see in the list, looks interesting overall.

      • Andrew Nacin 9:25 pm on January 27, 2014 Permalink | Log in to Reply

        Bootstrap is not Twitter Bootstrap, it’s the “loading” process of WordPress. I may rename this to Bootstrapping if it’s a problem.

        • StyledThemes 9:34 pm on January 27, 2014 Permalink | Log in to Reply

          ah… how about WordStrap :)

        • nofearinc 9:37 pm on January 27, 2014 Permalink | Log in to Reply

          That’s the initial association of many folks btw, even knowing what Bootstrapping is, the recent popularity of the Twitter’s thing is overlapping actively, just my 2¢

        • jack96161 10:26 pm on January 27, 2014 Permalink | Log in to Reply

          Agreed, Twitter commandeered “Bootstrap” for too many in the community these days — greybeards like me know what a real bootstrap is, but renaming it will eliminate more questions like this in the future. I always liked the “Progenitor Process”, we used in an early operating system at HP, but it will probably end up as something like “startup”, “init”, or other bland 2-syllable invention …

          • jack96161 10:34 pm on January 27, 2014 Permalink | Log in to Reply

            Just noticed on the 3.9 Trac pages that “Bootstrap/Load” is used as a component name — I’ll vote for that one and the Twitterites can just deal with it!

    • Aaron Jorbin 9:52 pm on January 27, 2014 Permalink | Log in to Reply

      One additional focus that may make sense is Unit-Tests. The use case I’m thinking of is a patch that just adds unit tests. It would go in the component that it most directly deals with as I think the test tools component is more focused on the tools of testing rather than the actual tests and receive the focus of Unit-Tests (and optionally the JavaScript focus if it is a javascript unit test)

      • Andrew Nacin 10:07 pm on January 27, 2014 Permalink | Log in to Reply

        Ah, yeah, I forgot to mention this. This is also on my list. I’m still trying to figure out how it overlaps with keywords (needs-unit-tests and has-unit-tests), so I saved it for later. Does needs-unit-tests and has-unit-tests simply trigger the unit-tests focus automatically? If a ticket is opened specifically to provide unit tests for something, is that more of a has-patch situation? etc.

        • Aaron Jorbin 11:27 pm on January 27, 2014 Permalink | Log in to Reply

          I think needs-unit-tests and has-unit-tests should trigger the unit tests focus.

          Has-patch makes sense in the just adding unit tests case since it is fixing the reported issue (not having unit tests) while I think needs-unit-tests and has-unit-tests is more workflow oriented.

    • Jon Brown 12:15 am on January 28, 2014 Permalink | Log in to Reply

      This looks great. You’ve done a great job of making the parents obvious (fwiw, I’d keep bootstrap as bootstrap).

      Two things seemed non-obvious to me, and maybe that’s just unfamiliarity on my part, but worth mentioning.
      First – Formatting-Shortcodes/Charset. This seems non-obvious to me but I’m guessing this is processing raw content, essentially wpautop, it’s brethren and associated filters. Formatting sounds more like CSS to me than filtering/processing content.
      Second – There are separate top level categories for HTTP API & XML-RPC. It seems to me this could be a single top level “Remote/External APIs” component with sub-components for HTTP, XML-RPC, JSON, etc… Maybe that’s makes no sense though though since code wise they’re entirely separate.

      Again, great work sifting all this into these components, these are just comments as I read through and understand what’s group where, why and what each component covers.

      • Andrew Nacin 7:01 pm on January 28, 2014 Permalink | Log in to Reply

        Formatting has been called Formatting, well, for as long as I’ve been around. It also centers around formatting.php. It might not be the best name, but yeah, it’s for processing raw content. We’ll make sure the description and search keywords for it are solid.

        For right now, HTTP and XML-RPC have pretty much zero overlap in form or function. We’ll need to revisit everything here anyway when the REST API comes into core. (Gut reaction would be a component for the server, and a focus to handle APIs specific to another component.)

    • Robert Dall 12:30 am on January 28, 2014 Permalink | Log in to Reply

      I am not sure if we want to confuse the themes & bundled theme anymore then they already are. When I gave a talk on trac and I explained why it is called bundled themes a couple light bulbs went off in the crowd. But yet they knew what default themes meant. It’s all semantics, but now that understand why themes and bundled themes are different.

      What I *REALLY* like is having a subcomponent for each theme in development. We could still use use the title to differentiate between Twenty-Thirteen and Twenty-Fourteen as is the current standard but having this and the easy of use sounds whole lot better and easier for query’s of trac etc…

      • Andrew Nacin 1:12 am on January 28, 2014 Permalink | Log in to Reply

        I’ve got no issue renaming Bundled Theme to Default Themes, and we can definitely nest individual components underneath it for each theme. The issue I see would be that if we added subcomponents for each default theme directly under the Themes component, where would a ticket go that affects more than one theme? (This isn’t entirely uncommon.)

        • Sam Sidler 4:19 pm on January 28, 2014 Permalink | Log in to Reply

          In the main Bundled/Default Theme component? I assume the components are still usable even when subcomponents exist.

          • Andrew Nacin 6:47 pm on January 28, 2014 Permalink | Log in to Reply

            Sorry, I should have been more specific. Option 1, which is totally fine;

            Default Themes

            • Twenty Ten
            • Twenty Eleven

            Option 2, which I’d prefer (and which I thought RDall was hinting at):

            Themes

            • Appearance
            • Widgets
            • Nav Menus
            • Twenty Ten
            • Twenty Eleven

            My point was I’d like to merge “Bundled Theme” (or “Default Themes” or whatever) with “Themes” to make it even less confusing. But then I’m not sure where general default theme tickets (affecting multiple themes) would go — getting dumped in “Themes” means they may get lost.

        • Robert Dall 6:20 pm on January 28, 2014 Permalink | Log in to Reply

          Does it need a subcomponent if it effects all themes? Or are subcomponents required when they exist?

  • Andrew Nacin 5:36 pm on August 15, 2013 Permalink
    Tags: ,   

    JavaScript meeting on Monday. @koop @azaozz @carldanley @adamsilverstein @kadamwhite will be meeting on Monday at 9 p.m. EDT (Tuesday, August 20 at 01:00 GMT) in the #wordpress-dev IRC channel. I’ll let them build out an agenda in the comments, but here’s some potential topics I know they’ve been kicking around:

    • JS style, structure, and design patterns, and what new code should look like
    • JS unit testing, including candidate frameworks and where to start
    • APIs, such as WP-style hooks (actions/filters), what form they should take if they should exist
    • Which areas of our JS need refactoring and overhauls, and what that might look like
    • Which JS-heavy features need eventual iteration (often “version 2″)
    • What action items could be done for WP 3.7 and 3.8
     
    • Beau Lebens 5:54 pm on August 15, 2013 Permalink | Log in to Reply

      I’ll try to be there.

    • Marko Heijnen 6:41 pm on August 15, 2013 Permalink | Log in to Reply

      I would love to join but unsure yet. The time to me is wrong choosen.

    • adamsilverstein 12:27 am on August 17, 2013 Permalink | Log in to Reply

      I am looking forward to contributing to the JS effort in 3.7 and will be at the meeting!

    • raam86 12:34 am on August 18, 2013 Permalink | Log in to Reply

      I would be delighted to help in anyway possible.

    • Andrew Ozz 5:31 pm on August 19, 2013 Permalink | Log in to Reply

      One more item for the agenda: upgrade TinyMCE to 4.x (3.7 or 3.8)?. Will break many WordPress plugins that add TinyMCE 3.x plugins (JS errors are silenced so TinyMCE would continue to work but without the 3.x plugin). All custom plugins need a rewrite, including ours.

      There are tons of improvements in TinyMCE 4.x:

      And a lot more both in the UI and back-end.

      • Aaron D. Campbell 5:45 pm on August 19, 2013 Permalink | Log in to Reply

        I’m really interested in this one. I’d be happy to help wherever. I’d love to see it in 3.7, but that’s mostly selfish. The current TinyMCE is the biggest struggle for many of our clients, and while 4.x isn’t perfect it DOES seem to be a serious improvement.

      • WraithKenny 6:16 pm on August 21, 2013 Permalink | Log in to Reply

        I’d like to see it upgraded to 4.x for 3.7 (or for 3.8 if 3.7 is not doable).

    • Josh (Ult. Tinymce) 6:33 pm on August 21, 2013 Permalink | Log in to Reply

      Hi, I am the developer of Ultimate Tinymce. I would LOVE to help out with this, and be included in the development.

      As Andrew mentioned, a ton of plugins (and themes) are going to need to be re-written to handle the new Tinymce 4.x interface. (Andrew… much respect!!)

      I was wondering if someone could point me in the direction of learning how to use the “#wordpress-dev IRC channel”, so that I may also be present in the meeting.

  • Andrew Nacin 7:35 pm on August 14, 2013 Permalink
    Tags:   

    WordPress 3.7 organization 

    With WordPress 3.7 and 3.8, we’re a project in major transition. Version 3.7 aims to solve for a lot of things that are weighing us down. (See the initial kickoff post.) What do you want to do? In terms of process, here is what is being worked on for 3.7:

    Processes, Tools, Workflows

    • New development tools — develop.svn.wordpress.org — run by @koop. If you are interested in Grunt, unit testing (including JS testing), and how we can streamline and modernize our workflows, this is for you.
    • Code reference — inline documentation efforts — run by @rzen, @ericlewis, and @drewapicture. If you are interested in inline documentation, this is for you. See this post to get started.
    • Process changes (including improvements to Trac and how we organize around components). If you want to write some Python for Trac, this is for you. If you’re just interested in helping with component reorganization, stay tuned here — there will be a post soon outlining a new component tree.

    Security, Stability, and Updates

    We have a few other focuses that all deal with a general theme of security and stability.

    • Passwords. We are aiming to improve the adoption of best security practices by assisting with password generation. #24633@duck_ is also working on some changes to strengthen cookies. #20276. If you are interested in security, this is for you.
    • Updates. @dd32 and @pento will be heading up automatic updates for minor releases, as well as improving the trustworthiness of our upgrader. If you love updates and stability, this is for you.
    • Language packs. We need to separate language files from plugins, themes, and core, allowing them to be maintained and updated independently (and, ideally, automatically). #18200. If you want to help WordPress’ global reach, hook up with me, @dd32, and @markoheijnen.

    General Triage

    Finally, we have 3,800 (already only 3,600) open tickets on Trac. There are dozens of components with many, many tickets. Already, there have been a few components with a strong base of contributors working on them, including:

    • Multisite, currently being smashed by @jeremyfelt and others. If you like multisite and want to make it better and more stable, this is for you. There are currently 122 open tickets.
    • JavaScript. A number of you have expressed interest on working on JavaScript in core. Whether that is shoring up the stability of existing features, improving existing JS, or working on a JS testing framework, this should be a great working group. @carldanley, @adamsilverstein, @nbachiyski, @kadamwhite, and others expressed interest.
    • Query and Taxonomy. These two advanced areas of core kind of go hand-in-hand, not in the least because @wonderboymusic smashes query tickets with his left hand and taxonomy tickets with his right. If you’re interested in bringing down these tickets (68 and 93 open), this is for you.
    • General triage. Folks like @c3mdigital and @avryl have been already going through old tickets either closing them out or finding diamonds in the rough. Or maybe you find that one of the many open components catches your interest. (Go here and choose any component from the drop-down to see all open tickets.) We’ll be coordinating efforts to work together both in IRC (especially during the weekly meetings) and here on make/core.
    • Components with a lot of open tickets: General (376 tickets), Administration (302 tickets), Media (221 tickets), Template (161 tickets), Comments (135 tickets), Users (107 tickets), Themes (105 tickets), Formatting (98 tickets), Menus (96 tickets), Widgets (90 tickets), Plugins (90 tickets), Editor (85 tickets), Upgrade/Install (84 tickets), Import (70 tickets). If you want to do general triage, General and Administration in particular need a lot of work!

    Today during the weekly meeting, we’ll be talking about ongoing efforts, rallying the troops and helping to assemble working groups, and setting up some times for regular triage meetings.

    So, what do you want to work on? Let’s start coordinating in the comments.

     
    • WraithKenny 7:49 pm on August 14, 2013 Permalink | Log in to Reply

      I’m interested in “New Development Tools” and JavaScript Triage. I’ll help as much as time permits and see if I can get a coworker to help out too.

    • MartyThornley 7:51 pm on August 14, 2013 Permalink | Log in to Reply

      I’m always up for multisite. Just submitted a revised patch for http://core.trac.wordpress.org/ticket/20774, based on feedback and taking a look now to see what else I might be able to jump on.

    • George Stephanis 8:00 pm on August 14, 2013 Permalink | Log in to Reply

      I take it core won’t be getting two-factor authentication in 3.7, then?

    • nofearinc 8:07 pm on August 14, 2013 Permalink | Log in to Reply

      I would like to work in “Updates” and help in General and Administration.

    • Konstantin Obenland 8:10 pm on August 14, 2013 Permalink | Log in to Reply

      I’d like to help clean up the themes component. I started with pushing some tickets to the 3.7 milestone and I think there are some quick wins there.

    • K.Adam White 8:30 pm on August 14, 2013 Permalink | Log in to Reply

      I’m guilty as charged: JavaScript! I’d love to have a conversation in #wordpress-dev to discuss the JavaScript changes. I know @carldanley has begun to work on some proposals; it should be a lively conversation!

    • Mike Schroder 10:01 pm on August 14, 2013 Permalink | Log in to Reply

      I’d like to help with “Updates”!

      Otherwise, I’m likely just to spend some time triaging tickets, likely mostly in the ‘media’ component. Another project needs assistance? Let me know!

    • lucasstark 10:16 pm on August 14, 2013 Permalink | Log in to Reply

      Regarding Process. Is there any way we could flag and filter tickets that are “plugin territory”.

      • Andrew Nacin 3:00 am on August 15, 2013 Permalink | Log in to Reply

        Good question. I think we’d hesitate to add yet another resolution. We have pretty much always used “wontfix” for this — as in, we are acknowledging the ticket is valid, but we’re declining to address it.

        By all means, feel free to close ‘plugin territory’ tickets with a comment stating such!

    • Ryan McCue 3:57 am on August 15, 2013 Permalink | Log in to Reply

      Development tools, process, passwords and updates sound good to me (and of course, background work on the code reference as needed). Process and tooling will probably be the main ones for me given time constraints (Python’s always fun).

    • Travis Northcutt 12:47 pm on August 15, 2013 Permalink | Log in to Reply

      Is there any reason to think that a plugin that prevents WP from sending passwords in plaintext, and instead requires a new user to set their own password, would be a good idea?

      I don’t know enough to know the specifics of how that would work internally, but it’s something that interests me.

    • Nashwan Doaqan 5:31 pm on August 15, 2013 Permalink | Log in to Reply

      I am really want to help with Language packs, this will save my life !

    • Mustafa Uysal 12:45 pm on August 16, 2013 Permalink | Log in to Reply

      I’d like to help language packs.

    • doughamlin 4:00 pm on August 19, 2013 Permalink | Log in to Reply

      Are the two items listed above for passwords the only things being worked on? I think it’d be smart to see both limited login attempts (probably not technically password-related) and strong password enforcement built into core.

  • Daryl Koopersmith 4:19 pm on August 6, 2013 Permalink
    Tags: develop-svn   

    A New Frontier for Core Development 

    In a little over a decade, we’ve made twenty five thousand commits to WordPress. WordPress (along with the web itself) has come a long way, but our development workflow has remained largely the same.

    As a part 3.7, I’ll be leading an effort to revamp and streamline our development workflow. We’re going to bring all of our core components—our code, our tests, and our tooling—under one roof. Developers will be able to use and improve the tools we’re already working with day-to-day, and we’ll be able to add new tools to make working with WordPress even easier.

    We’re also making sure that any changes are compatible with our current workflow, so you won’t have to change the way you work. These changes won’t break any existing checkouts or scripts that use core.svn.wordpress.org. Now that we’ve cleared that up…

    develop.svn.wordpress.org

    This week, we’re creating a new repository: develop.svn.wordpress.org.

    The develop repository will be the new home for all core development. We’re going to sync over all of the commits from core.svn.wordpress.org and will reorganize this new repository to use a new directory structure:

    The src directory

    This is the WordPress core source. All of the existing WordPress core files will be moved into this directory. You’ll develop and test against this directory as usual, but the real power will come from using the rest of the tools in the repository. The contents will be optimized for development, not production (this means uncompressed files—including our internal libraries).

    The tests directory

    We’re going to pull our internal unit tests into this repository as well. This will make testing a cinch, and allow us to commit patches that include changes to both the source and the unit tests. We’ll also add JavaScript unit tests during the 3.7 cycle.

    The tools directory

    Our tools directory will include all of the little things that make our lives as developers easier. To start, we’ll include our internationalization tools, which will allow us to generate pot files from within core. We’ll also include a handful of scripts that some of us use to interact with Trac, like scripts to upload and apply patches.

    The root directory

    This frees up the root directory to include configuration files, allowing WordPress to interact more seamlessly with other applications and services like Travis CI and Composer.

    A build process

    We’ve had an implicit build process in the WordPress codebase for a considerable time now. Most of that process is now contained within bumpbot, a little script that lives on the WordPress.org servers. Whenever we update CSS or JavaScript files, bumpbot comes around, minifies them, and commits the result. Despite bumpbot’s efforts, this process is less than ideal. Our source files (in this case, JavaScript and CSS) live alongside our production-optimized files, which causes confusion and makes contributing to core considerably more difficult. Our built output—the files optimized for machines—shouldn’t live alongside the source code that’s optimized for developers, and beyond that, shouldn’t be tracked in source control.

    We’re adopting Grunt to write a real build process and help us manage and run our various tools. Grunt is a JavaScript task runner that has a robust ecosystem of plugins focused around building modern web applications. We can include tools such as integrated testing, minification, linting (for PHP, JS, and CSS), image optimization, and LiveReload—and that’s just the beginning.

    Using Grunt will be an optional but encouraged part of WordPress development. These tools are all built atop Node.js, which Grunt needs in order to function.

    So what’s happening to core.svn.wordpress.org?

    In a word, nothing. The contents of the core repository have always been optimized for production, and that’s not going to change. core.svn.wordpress.org will now serve as the repository for our built output: every time we make a commit that changes the build, the core build will be updated with a corresponding commit. As mentioned earlier, scripts and checkouts will continue to function as expected. When you download a zip of WordPress, it will still be generated from this repository.

    Let’s get started

    The develop repository will go live soon. For now, I’ve made a preparatory repository available to illustrate the structure I’ve described above, and allow us to experiment with this new workflow. You can learn more about how the tools are installed and used on the repository page. Please provide feedback on this post—the goal is not to build all of the code there first, but to provide insight into the new process.

    Lastly, I’d like to say thank you to all of the contributors who helped shape this idea over the past few months, especially at WordCamp San Francisco’s hack day. Together, we’ve created a structure that elegantly addresses many of our existing problems and opens the door to countless possibilities for better development practices.

     
    • Aaron Jorbin 4:26 pm on August 6, 2013 Permalink | Log in to Reply

      This is incredibly exciting and awesome. It really opens the door for things like #22862 and other modernizations for how WordPress is developed. For now if we want to help, should we open issues and pull requests on the preparatory repository?

      • Andrew Nacin 4:31 pm on August 6, 2013 Permalink | Log in to Reply

        develop.svn.wordpress.org will be going live soon, likely today. We turned issues off on that repository so the conversation wouldn’t be fractured — any general feedback code can happen here for now, while Trac will (upon launch) be the proper place to contribute code.

    • Peter Chester 4:29 pm on August 6, 2013 Permalink | Log in to Reply

      Very cool! Excited to learn more about how we can use Grunt to do this (I’ve never worked with it).

    • Boone Gorges 4:31 pm on August 6, 2013 Permalink | Log in to Reply

      This is extremely exciting. I can’t wait to see how this core change trickles down through the WP dev ecosystem, especially the increased emphasis on automated testing. Thanks to all for your work so far!

    • Andrew Nacin 4:32 pm on August 6, 2013 Permalink | Log in to Reply

      I think the coolest thing about this project is we’re doing it in a backwards compatible way.

    • Ryan Duff 4:36 pm on August 6, 2013 Permalink | Log in to Reply

      I think it’s safe to say, the future is here.

      Awesome news and thanks to everyone involved’s effort to make this happen.

    • Jeff Bowen 4:37 pm on August 6, 2013 Permalink | Log in to Reply

      Very exciting. grunt is super powerful. I’m excited to see how it gets used here.

    • Amy Hendrix (sabreuse) 4:38 pm on August 6, 2013 Permalink | Log in to Reply

      So very excited!

    • Mike Schinkel 4:38 pm on August 6, 2013 Permalink | Log in to Reply

      This is excellent news.

    • Franz Josef Kaiser 4:39 pm on August 6, 2013 Permalink | Log in to Reply

      Yes!

    • Andrew Ryno 4:40 pm on August 6, 2013 Permalink | Log in to Reply

      Been waiting for these changes to occur for a while. Glad 3.6 is finally how so they can happen :)

    • Ryan Duff 4:41 pm on August 6, 2013 Permalink | Log in to Reply

      I do have 1 question, and because someone is bound to ask, we might as well discuss the elephant in the room– why stick with SVN?

      Since core.svn.wordpress.org is staying around and just being pushed to, can’t core development be rolled into git at this point and just use git-svn to push those build commits back to core.svn.wordpress.org?

      Seems like the perfect timing if it were to ever happen.

      • Ryan Duff 4:44 pm on August 6, 2013 Permalink | Log in to Reply

        And sorry, I wasn’t trying to be pushy. Just curious.

        I know SVN is familiar to most. Is there some other technical reason?

      • Alex King 4:49 pm on August 6, 2013 Permalink | Log in to Reply

        • chriscct7 4:51 pm on August 6, 2013 Permalink | Log in to Reply

          Then manually convert the patches and diffs to SVN style isn’t exactly fun

          • Andrew Nacin 6:30 pm on August 6, 2013 Permalink | Log in to Reply

            SVN and Git use the same diff format, and both can be applied with patch.

            • Jonathan D. Johnson (jondavidjohn) 6:58 pm on August 6, 2013 Permalink

              I don’t think he’s saying he wants to use git formatted patch files. I believe the correct sentiment here is that pain of juggling / refreshing patch files period. Creating, Passing Around, and applying patch files feels exceedingly ancient to anyone who cut their teeth on git.

            • chriscct7 10:56 pm on August 6, 2013 Permalink

              I can’t reply directly to Jonathon but exactly what I mean. In an ideal world WordPress would use an open source version of a Github style program with pulls and the issue tracking. I really dislike the vast limitations of Trac.

            • Ryan McCue 2:13 am on August 7, 2013 Permalink

              (They are slightly different; git prefixes paths with `a/` and `b/`, which I don’t think `patch` handles correctly.)

            • Bryan Petty 2:16 am on August 7, 2013 Permalink

              It does handle them correctly as long as you use the right strip level value, but on the other hand, every patch I submit is built from git, except I use diff.noprefix in my repo config that prevents that “a/” / “b/” prefix entirely with every diff, and hopefully everyone is just using that instead, and saving everyone the hassle altogether.

            • Andrew Nacin 3:11 am on August 7, 2013 Permalink

              We’ll be adding a patch application script to the new develop.svn repo that will handle all sorts of incoming formats, including git diff with a/ and b/ prefixes, Trac URLs, core.svn versus develop.svn /src patches, GitHub URLs, etc. We’ll probably also add a diff-and-upload-to-Trac script.

        • Andrew Nacin 4:53 pm on August 6, 2013 Permalink | Log in to Reply

          Congratulations, anyone who runs this, you’ve just cloned the WordPress for Android source. :-) Will fix that.

      • Andrew Nacin 4:51 pm on August 6, 2013 Permalink | Log in to Reply

        Actually, I’m surprised it took this long for a “git” mention in the comments. :-) I had odds on comment one.

        The differences between core.svn and develop.svn — one being a built source as it is now, the other being a full repository with raw source, tools, and configuration — has very little to do with the version control system in use. You can check out our syncing script which would be just as complicated (actually, more so) if it involved Git or git-svn. Nothing here prevents, impedes, or for that matter helps any switch to git. It’s a different conversation.

        • scribu 5:24 pm on August 6, 2013 Permalink | Log in to Reply

          Hi!

          If the development repo would be under git, you’d have to first ask the question:

          Do we want to preserve the commit history (i.e. commits in the production repo should roughly match commits to the src/ directory in the dev repo)?

          If the answer is yes, you will indeed have to do some git-svn gymnastics. If the answer is no, you wouldn’t need git-svn at all. Just copy the files into the prod svn repo and make a commit with a generic message using svn directly.

          But, the question “Why not move directly to git?” is still relevant:

          If you switch to git now, you will only have to maintain core.svn.

          If you switch to git later, you will have to maintain both core.svn and develop.svn.

          • Andrew Nacin 6:01 pm on August 6, 2013 Permalink | Log in to Reply

            Yes, we wanted to maintain history.

            “Moving” implies we’d drop SVN for a new Git repository, which is not desirable. This project in particular was carefully designed to be a simple and painless transition. Switching version control systems would have made adopting this new structure more difficult.

            And more to the point, I’ve mentioned elsewhere that we should aim to become VCS agnostic. That means regardless of what we do now or in the future, there will need to be a develop.svn — either as the canonical repository that is mirrored to Git, or as a mirror of a Git repository. Which is why I said nothing here either impedes or helps Git.

            • scribu 6:45 pm on August 6, 2013 Permalink

              I agree that this step is a big improvement over the previous workflow.

              I don’t have much faith in the idea of being VCS agnostic though, since it implies restricting the workflow to the lowest common denominator, which is SVN.

              It’s like the politically correct way of saying “we use SVN, but have official Git mirrors”.

            • scribu 7:17 pm on August 6, 2013 Permalink

              To clarify, being truly “VCS agnostic” would mean that you’ll also accept pull requests made on the git (not necessarily github) repo and they will have equal visibility with svn patches.

              If you can make trac work seamlessly for both, then great, but I think it’s a waste of resources.

              “This concludes our ranting hour.”

            • Andrew Nacin 7:57 pm on August 6, 2013 Permalink

              Except that the difference between “we use SVN, but have official Git mirrors” and “we use Git, but have official SVN mirrors” is a flipped switch that mostly just affects a dozen committers.

            • Andrew Nacin 8:03 pm on August 6, 2013 Permalink

              But if we moved to Git, we’d need to adapt our workflow for pull requests (to a Git repo or to a GitHub mirror). And we’d need to adapt Trac to fit a change in workflow. Incidentally, we’d have to do all of this to become “agnostic”. You’d also have to consider the amount of time lost by existing contributors trying to immediately shift their current workflows (rather on their own terms), and potentially the number of contributors we lose for what could amount to a major roadblock for contributing. All of that and more easily justifies some resources. They aren’t a waste.

            • wycks 8:30 pm on August 6, 2013 Permalink

              Maybe have a look at something like http://subgit.com/ (bidirectional Subversion to Git replication)

            • scribu 9:00 pm on August 6, 2013 Permalink

              I find myself agreeing with you, Nacin. There still are a lot of people that know SVN, but don’t know Git and there still is this impression that git is hard to learn and/or to use.

              So, yeah, getting to a point where you can accept both svn patches and git pull requests seems like a more pragmatic approach than making a sudden switch from Subversion to Git.

              By the way, Github has great Subversion support:

              https://github.com/blog/966-improved-subversion-client-support

              https://github.com/blog/1178-collaborating-on-github-with-subversion

          • wycks 8:32 pm on August 6, 2013 Permalink | Log in to Reply

            Nevermind it’s It’s closed source, didn’t realize.

            • Andrew Nacin 8:45 pm on August 6, 2013 Permalink

              It also doesn’t work. (We’ve tried.)

            • chriscct7 10:58 pm on August 6, 2013 Permalink

              Github? WordPress could use Enterprise Github or an open source version like Gitlab

          • iamntz 4:12 pm on August 7, 2013 Permalink | Log in to Reply

        • Ryan Duff 9:30 pm on August 6, 2013 Permalink | Log in to Reply

          Ok, so I’ve been pondering this all afternoon and while most questions have been answered in one form or another, what’s the new workflow?

          You’re ditching the traditional svn model of /branches /tags /trunk and going to a more git-like repo structure. And while that’s fine in git because of how branching works, it doesn’t play nice in SVN.

          develop/src essentially becomes trunk. What happens when you need to push a security patch on 3.5? Where and how does that happen?

    • Robert Dall 4:42 pm on August 6, 2013 Permalink | Log in to Reply

      If I understand this correctly dev.svn is where all the beta development will take place but core.svn is where the confirmed code will actually live.

      • Daryl Koopersmith 5:38 pm on August 6, 2013 Permalink | Log in to Reply

        develop.svn is where all of the source code will live—it will be the repository we commit to and build on. The core.svn repository becomes the home for the processed output: the code in core.svn will be generated from the code in develop.svn, and will include production-optimized code (i.e. minified files).

    • Jeremy Felt 4:42 pm on August 6, 2013 Permalink | Log in to Reply

      Freaking awesome! Can’t wait to start using this. Any thoughts from @koop, @nacin, @markjaquith on adding a Vagrantfile to the develop repo as well?

      • Mark Jaquith 4:50 pm on August 6, 2013 Permalink | Log in to Reply

        Indeed, I brought that up while we were discussing this at WordCamp San Francisco. Absolutely something that could be in the cards.

        • Jeremy Felt 4:54 pm on August 6, 2013 Permalink | Log in to Reply

          Beautiful. I will gladly dive into this.

          • Bryan Petty 1:46 am on August 7, 2013 Permalink | Log in to Reply

            I’ve briefly been looking at this myself, and I think there might be some benefit to re-using Puppet Labs’ pre-built vagrant boxes, and using their same process (Veewee) for building new boxes we need that they’re missing (which includes a PHP 5.2 box right now). Puppet’s existing Veewee configurations could easily be cloned with minor modifications too. It’s all flexible, well defined, reproducible, and well automated.

            It should give everyone a great way to quickly spin up esoteric (but supported) hosting environments to hunt down bugs and test new features regardless of what hardware everyone has.

            • Jeremy Felt 4:56 pm on August 7, 2013 Permalink

              I have some (quickly outdated) thoughts that I wrote up on what a core Vagrantfile would accomplish. Need to rewrite that a bit now that all of this magic has been released. I personally don’t care for Puppet and would rather WP.org start with a blank box of our own, though it’d be interesting to compare the benefits. http://jeremyfelt.com/code/2013/07/28/a-wordpress-core-vagrantfile/

      • rilwis 3:44 pm on August 8, 2013 Permalink | Log in to Reply

        Quite interesting idea! All weapons in hand now!

    • Greg Rickaby 4:44 pm on August 6, 2013 Permalink | Log in to Reply

      +1 Ryan Duff

    • Weston Ruter 4:48 pm on August 6, 2013 Permalink | Log in to Reply

      Awesome. And this is the gateway that will allow WordPress to be VCS-agnostic. Can’t wait to submit pull requests to https://github.com/WordPress/WordPress :-)

      See @nacin: https://twitter.com/nacin/status/361200912678662146

    • tomaugerdotcom@yahoo.ca 4:50 pm on August 6, 2013 Permalink | Log in to Reply

      Very exciting guys. I hope this new change will facilitate a general cleanup of tickets in Trac and will make it easier for the community to contribute and get patches committed!

    • Gregory Cornelius 4:55 pm on August 6, 2013 Permalink | Log in to Reply

      I am super excited about this and appreciate the clever approach used to make this shift.

    • Tracy Rotton 5:05 pm on August 6, 2013 Permalink | Log in to Reply

      So does this mean we’re one step closer to using CSS preprocessors? Maybe? A little?

    • Eric Mann 5:08 pm on August 6, 2013 Permalink | Log in to Reply

      Adding JS tests to core is a huge win in my book. Not only does it help make our code more robust, it helps document what the code does. I know many newer devs who struggle to grasp some of the fancier things we’re doing with JS in core – many senior devs, too.

      I would love to help with the JS testing. Just let me know what’s needed :-)

      P.S. Can I make a recommendation for QUnit?

      • Andrew Nacin 5:17 pm on August 6, 2013 Permalink | Log in to Reply

        An earlier revision of this post actually said they’d likely be “QUnit-powered.” It was removed because we didn’t want to suggest we were prematurely choosing a framework. That’s something that should be discussed openly and with all other options considered, rather than being the product of some unilateral decision. (For me at least, our existing relationship with the jQuery project certainly makes QUnit appealing.)

        • Eric Mann 5:25 pm on August 6, 2013 Permalink | Log in to Reply

          Agreed. I’d be open to discussing other frameworks. Just leaning towards QUnit both because of the existing relationship with jQuery and with the easy integration it has with the aforementioned Grunt process.

      • Aaron Jorbin 5:56 pm on August 6, 2013 Permalink | Log in to Reply

        I’m also excited for this part and a fan of QUnit, but it has also been a little while since I looked at all the options out there. I’m planning on spending some time playing with as many as I can later this week/early next week just to see what is out there.

        I think most of the frameworks have grunt integration, so that point is null.

        I think what is going to matter the most is robustness (what will make most of our code testable), ease of use, and cross browserness.

        A jshint file and making it so all of our code follows that will help a lot as well.

        • K.Adam White 8:49 pm on August 6, 2013 Permalink | Log in to Reply

          Aaron’s right, Grunt support is not a significant distinguishing factor anymore. I originally asked Koop & Nacin about QUnit as well, both because of the jQuery connection and the assertion-style syntax’s closer similarity to PHPUnit vs a BDD syntax, but I’m +1 for trying out the options. I’m also excited about looking into running tests in multiple browsers, à la what the jQuery team does with http://swarm.jquery.org/, but landing on a library and getting some tests in certainly comes first!

    • Robert Chapin 5:13 pm on August 6, 2013 Permalink | Log in to Reply

      Is this going to invalidate all pending patches, or can you still accept patches that were generated from core paths?

      • Andrew Nacin 5:19 pm on August 6, 2013 Permalink | Log in to Reply

        Yes, we’ll continue to accept patches that were generated from core paths. (Our patch application scripts will likely end up in this repository, to boot.) So yes, you can continue to make patches off core.svn, though it might be easier if you switch to develop.svn. (If you don’t want to mess with the whole build process, you can check out just the “src” directory and create patches from there.)

        • Jeremy Felt 5:24 pm on August 6, 2013 Permalink | Log in to Reply

          I’ll admit the syncing of commits confuses me. Why not do an SVN external of core.svn into the /src directory?

          • Bryan Petty 5:38 pm on August 6, 2013 Permalink | Log in to Reply

            It would make it impossible to generate patches from SVN across the entire repository if that happened for one, but also it would negate a major benefit of including the unit tests in the same repo – Travis CI wouldn’t work properly that way.

          • Andrew Nacin 5:42 pm on August 6, 2013 Permalink | Log in to Reply

            Avoiding externals was a major goal here. (@koop may be able to elaborate further on those discussions.) We want develop.svn to become the canonical place for all development: code, tests, and tools. As Bryan pointed out, one patch — and one commit — can now include both code and associated tests. It

            core.svn will simply be a build repo, and we are implementing commit syncing because it’s much nicer than having “dumb” syncs. It didn’t take a lot of work for core.svn’s own logs to still be useful, and we felt that was worth it.

    • Terry Sutton 5:23 pm on August 6, 2013 Permalink | Log in to Reply

      Just curious if this will make it easier for beginners to get started or will result in more overhead and make it harder?

      • Andrew Nacin 5:34 pm on August 6, 2013 Permalink | Log in to Reply

        The goal is for this to be no harder. You could still run core.svn, or even just develop.svn’s /src directory — no build process required. We’re not sure it’ll make it easier to get started, but once you do, I reckon it’ll be a lot easier to contribute, in part because of how much more power these tools will bring to existing workflows.

    • Jon Brown 5:32 pm on August 6, 2013 Permalink | Log in to Reply

      This sounds like a good move. What does this mean for SCRIPT_DEBUG? I’m a little unclear how this makes it easier to do development on core since the un-minified/compressed files are no longer handy in a standard install. Again, sounds like a good move, just a little unclear how this changes running trunk.

      Perhaps outside the scope of this topic, but what’s happening with Westi’s WP Beta Tester plugin? Will that install built nightly’s w/o dev files or will that include the un-minified/compressed files?

      Finally, I echo Ryan’s comment, why not switch to git but in conjunction with this even if it’s a separate issue it seems the logical time do it..

      • Amy Hendrix (sabreuse) 5:42 pm on August 6, 2013 Permalink | Log in to Reply

        If I’m understanding all this correctly, develop.svn’s src directory will have development versions of everything, and core.svn will have production files — so if you develop against a src checkout, all the unminified files are right there.

      • Andrew Nacin 5:44 pm on August 6, 2013 Permalink | Log in to Reply

        The un-minified files will still be handy on a standard install (core.svn, a download zip, etc). The difference is that the *minified* files will no longer be in version control (in develop.svn, at least).

        While SCRIPT_DEBUG will still exist, if you are running develop.svn’s “src” directory, it’ll simply be on automatically. (We may turn WP_DEBUG on by default too. wp-dev-config.php? Who knows.) This will make it easier for people to contribute. By not tracking built files, it makes it much more obvious which files to edit.

        • Amy Hendrix (sabreuse) 5:53 pm on August 6, 2013 Permalink | Log in to Reply

          Ooh, I like the wp-dev-config.php idea (or some kind of default development profile, whatever that evolves to be) — I can’t count the number of times I’ve pointed someone toward those two constants, and I’ve forgotten one or the other myself at least as many times.

          • nofearinc 7:59 pm on August 6, 2013 Permalink | Log in to Reply

            I definitely like that idea as well, all debugging snippets could be placed there to facilitate debugging scripts, queries, log data and so forth – all in!

        • Jon Brown 7:31 pm on August 6, 2013 Permalink | Log in to Reply

          Awesome, that all makes great sense. Personal preference wp-config-dev.php, not wp-dev-config.php.

    • Gage Morgan 7:09 pm on August 6, 2013 Permalink | Log in to Reply

      I was at WordCamp Columbus last week, and since being told that I should join the core team, I realize I don’t know how to use PHP. Once I learn, how do I join?

      • K.Adam White 9:18 pm on August 6, 2013 Permalink | Log in to Reply

        Gage: First of all, PHP’s not a requirement — there’s lots of work to do with CSS, JavaScript, documentation, images, and so on, so you’re welcome whether or not you’re a PHP whiz! (Many of us are not). Second, we’re working on some “getting started w/ contributing” guidelines here: http://make.wordpress.org/core/handbook/ Basically, keep an eye out on these make.wordpress.org blogs for interesting issues, then jump in to the discussion!

    • Andrey "Rarst" Savchenko 7:11 pm on August 6, 2013 Permalink | Log in to Reply

      I am not sure about this making it easier to support Composer? Are you considering for Composer package to include this whole new repository with tests and everything, not just core?

      • scribu 7:24 pm on August 6, 2013 Permalink | Log in to Reply

        To be fair, most Composer packages do include their respective test suites.

        • Andrey "Rarst" Savchenko 7:29 pm on August 6, 2013 Permalink | Log in to Reply

          Yes, that was my first thought that by analogy it would make sense… However most Composer packages:

          (1) are much smaller than WP core and there isn’t much of test stuff accordingly (or maybe I just wasn’t paying attention to volume)

          (2) do not expose their folder structure in URL, core doesn’t work like that – any extra levvel in folder structure would be reflected in admin URL

          • Bryan Petty 2:10 am on August 7, 2013 Permalink | Log in to Reply

            I’m not sure there is a hosting company out there that doesn’t let you customize the webroot directory, which they’ve technically already done when they mapped your domain to a default folder. In fact, if that root directory is exposed in your URL, you did something very wrong to begin with because you’ve also now exposed everything else to your webserver (tests, tools and scripts, etc).

            Not that it matters all that much anyway though because anyone using Composer is typically the type of audience that is building out custom servers. For your typical one-click-installers, they’ll still be powered from zip builds off core.svn, not including tests or the extra subdirectory, and they’ll still handle upgrades the same way they always have (along with whatever new automatic security upgrades happen past 3.7).

            For those advanced users making use of Composer, not only can that process of explaining which directory needs to be the webroot be documented, but I think most Composer users are used to configuring that anyway with most new projects built on popular frameworks. It’s “public” for Laravel, “app/webroot” for CakePHP, “web” for Symfony, etc…

            On the other hand, I do understand your concerns here. I’m more worried about how the existing “wordpress-plugin” / theme / muplugin installer types will now be in the wrong location by default unless we also broke wp-content out into the root of the repo alongside “src”, and was simply linked to during dev env config, and copied into the appropriate location in packages during releases. Of course, this is also why composer-installers is not built-in functionality in the first place, and is discouraged when possible, and end-users can still fix this themselves in their project root.

    • adamsilverstein 8:35 pm on August 6, 2013 Permalink | Log in to Reply

      Very exciting, great introductory post & comments – thanks for all the thought that went into the is process already!

    • K.Adam White 9:13 pm on August 6, 2013 Permalink | Log in to Reply

      I’m curious for thoughts on a related subsidiary file re-organization — specifically, moving front-end assets (JS and CSS) out of the root of wp-includes/ and wp-admin/, and into an intermediary folder e.g. wp-includes/static/ and wp-admin/static. I’ve used this structure on a number of standalone projects and themes, and find it provides a useful separation between server-oriented code and browser-oriented code.

      • Andrew Nacin 6:46 am on August 7, 2013 Permalink | Log in to Reply

        Because these are public files, anyone can just reference them in a script src or link href attribute, or use them as part of a script/style enqueue, etc. So we’ll probably hold onto wp-admin/css/$file or wp-includes/css/$file, which is much better than wp-admin/some-file.css, at least.

    • Jeff Waugh 9:31 pm on August 6, 2013 Permalink | Log in to Reply

      Terrific news! :-D

    • Japh 11:44 pm on August 6, 2013 Permalink | Log in to Reply

      Wow, this is super exciting news! Looking forward to jumping back in for 3.7/3.8 development, especially now.

    • Chuck Reynolds 4:15 am on August 7, 2013 Permalink | Log in to Reply

      yuuusssss.. exciting times now through dec. :)

    • Lester Chan 6:34 am on August 7, 2013 Permalink | Log in to Reply

      Any thoughts of including imagemin task in Grunt as well to optimize images?

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel